Amazon MSK - Managed Streaming for Apache Kafka
A comprehensive reference for Amazon MSK: managed Kafka clusters, event streaming patterns, and integration with AI/ML data pipelines.
Data-Pipelines
Amazon MWAA - Managed Workflows for Apache Airflow
Amazon MWAA is a fully managed service that runs Apache Airflow on AWS, providing workflow orchestration for data pipelines, ETL jobs, and …
Apache Airflow - Workflow Orchestration Platform
Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring data workflows and ETL pipelines.
Apache Kafka - Distributed Event Streaming Platform
Apache Kafka is a distributed event streaming platform used for high-throughput, fault-tolerant real-time data pipelines and streaming …
Great Expectations - Data Validation and Quality
Great Expectations is an open-source Python library for validating, documenting, and profiling data to ensure data quality in pipelines.
Prefect - Modern Workflow Orchestration
Prefect is an open-source workflow orchestration framework that makes it easy to build, observe, and react to data pipelines using Python.