Announcing Dagster 1.0. - a stable foundation for building the orchestration layer for modern data platforms.
We are extremely excited to announce the release of Dagster 1.0.
1.0 doesn’t include any seismic changes - rather, it’s a marker that indicates we’ve put the finishing touches on Dagster’s core abstractions. Data teams who want to access what’s unique about Dagster now have a stable foundation to build on.
Dagster is different from other data orchestrators. It’s the first orchestrator built to be used at every stage of the data development lifecycle - local development, unit tests, integration tests, staging environments, all the way up to production. And it’s the first orchestrator that includes software-defined assets - it frees up teams to think about critical data assets they’re trying to build and let the orchestrator manage the tasks.
The Dagster journey started four years ago, when Elementl’s founder and CEO Nick Schrock laid down the first line of code. Since then, Dagster has had 463 releases, and it’s grown to include a production-grade scheduler, schematized config system, rich logging, asset catalog, lightweight sensors, thirty three integration libraries, best-in-class web UI, and a whole lot more.
Through all this, we held off on declaring Dagster 1.0, because we believe that iteration results in better software - if you stick with the first approach that occurs to you, it often means you’re not exploring the full space of possibilities, and you’re likely to end up far away from the global optimum. Over that time, we made a number of tweaks to Dagster’s core APIs that substantially improved their ergonomics and utility. Eventually, we found that we were no longer making breaking changes.
Last year, we arrived at stable versions of Dagster’s core computational abstractions - ops, graphs, jobs, schedules, and sensors. This year, we did the same with the asset layer - software-defined assets, materializations, and asset partitions. With all of Dagster’s core abstractions now stable, it was time to make things official and declare 1.0.
1.0 represents the work of over 200 people who contributed code and hundreds more who contributed feedback, bug reports, and encouragement. We’re incredibly appreciative of everybody in the community who has helped Dagster get here.
Of course, there’s still tons left to do. Our core engineering team is already mapping out what we’ll build next on top of this foundation. We’re excited to lean deeper into declarative asset orchestration, to expand and harden Dagster’s integrations, and keep widening the capabilities of Dagster’s orchestration layer.
For more details on this release, check out the change log, release notes, and migration guide.
