About this integration
The dagster-meltano library allows you to run Meltano using Dagster. Design and configure ingestion jobs using the popular Singer.io specification.
Note that this integration can also be managed from the Meltano platform using meltano add utility dagster and configured using meltano config dagster set --interactive.
Installation
pip install dagster-meltano
Example
An example of running an abitrary Meltano run command:
from dagster import repository, job
from dagster_meltano import meltano_resource, meltano_run_op
@job(resource_defs={"meltano": meltano_resource})
def meltano_run_job():
tap_done = meltano_run_op("tap-1 target-1")()
meltano_run_op("tap-2 target-2")(tap_done)
@repository()
def repository():
return [meltano_run_job]
About Meltano
Meltano provides data engineers with a set ot tools for easily creating and managing pipelines as code by providing a wide array of composable connectors. Meltano's 'CLI for ELT+' lets you test your changes before they go live.
About Quantile
Quantile is an expert data consultancy based in the Netherlands. You can find out more at quantile.nl.