About dagster-clickhouse
The dagster-clickhouse library provides an integration with ClickHouse using the native protocol (clickhouse-driver). Use a resource for ad hoc SQL, I/O managers for Pandas or Polars DataFrames, and components to wire templated SQL into Dagster projects.
Installation
pip install dagster dagster-clickhouse
Example
import pandas as pd
from dagster_clickhouse_pandas import (
ClickhousePandasIOManager,
)
import dagster as dg
@dg.asset
def iris_dataset() -> (
pd.DataFrame
): # asset name is the table name; I/O manager `schema` is the ClickHouse database
return pd.read_csv(
"https://docs.dagster.io/assets/iris.csv",
names=[
"sepal_length_cm",
"sepal_width_cm",
"petal_length_cm",
"petal_width_cm",
"species",
],
)
defs = dg.Definitions(
assets=[iris_dataset],
resources={
"io_manager": ClickhousePandasIOManager(
host="localhost",
port=9000,
database="default",
schema="iris",
)
},
)
About Clickhouse
ClickHouse is a column-oriented OLAP database designed for analytical queries over large datasets.