Learn best practices for writing Pythonic tests for Dagster.
We are happy to announce a new addition to Dagster University with our new course: Testing with Dagster!
Jump to the Testing with Dagster course page ↗
Learn
Testing is often overlooked in data engineering. However, the only way to properly scale a data platform is to move beyond constant maintenance and troubleshooting issues in production.
In order to build with confidence, you need assurances that your code works as expected before it ships. That means having tests in place to validate new features or ensure changes do not have unintended consequences.
Testing with Dagster is a six-lesson course, each focused on a different aspect of testing. If you've never written tests before, this course provides a structured introduction to test design and an overview of testing in Python. If you're an experienced Python and Dagster user, you’ll find best practices and techniques to streamline your testing suite.
Testing in Dagster
At Dagster, we believe strongly in the power of testing. The only way we can release a new version of Dagster every week is by ensuring everything works as we develop. We want our users to have that same level of confidence in the code they build.
This module covers:
- The fundamentals of unit testing and writing asset tests in Dagster.
- Strategies for handling external dependencies in your Dagster deployment while maintaining full control in a testing environment.
- Best practices for integration testing to ensure your tests mirror real-world production scenarios.
- Proven Dagster-specific testing tips to help you maintain and optimize your project.
Example: Mocking API calls
@patch("requests.get")
def test_state_population_api_assets_config(mock_get, example_response, api_output):
mock_response = Mock()
mock_response.json.return_value = example_response
mock_response.raise_for_status.return_value = None
mock_get.return_value = mock_response
result = dg.materialize(
assets=[
lesson_4.state_population_api_resource_config,
lesson_4.total_population_resource_config,
],
resources={"state_population_resource": lesson_4.StatePopulation()},
run_config=dg.RunConfig(
{"state_population_api_resource_config": lesson_4.StateConfig(name="ny")}
),
)
assert result.success
assert result.output_for_node("state_population_api_resource_config") == api_output
assert result.output_for_node("total_population_resource_config") == 9082539
Enroll Today
Like all Dagster University courses, Testing with Dagster is free and available to everyone. Simply sign up at Dagster University to get started. Once enrolled, you can track your progress and learn at your own pace.
