Data-Engineering
All articles
24 total
AI Data Cleaning and Normalization
AI detects and fixes data quality issues - inconsistent formats, …
Ideas
Added 28 Mar
·
Upd 30 May
·2 min
Amazon Kinesis
What Amazon Kinesis is, how it processes streaming data in real time, …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Amazon OpenSearch Service
Search and Analytics for AI
Data
Added 25 Mar
·
Upd 14 Jun
·6 min
Apache Kafka
What Kafka is, how it provides distributed event streaming, and when to …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Change Data Capture
What change data capture (CDC) is, how Debezium and AWS DMS enable …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Data Catalog
What a data catalog is, how metadata management and data discovery tools …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Data Contract
What data contracts are, how schema-first agreements between data …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Data Contract Pattern for AI Systems
Implementing schema contracts between data producers and AI consumers: …
Patterns
Added 28 Mar
·
Upd 30 May
·3 min
Data Lake
What a data lake is, how it stores raw data at scale, and when to use a …
Glossary
Added 28 Mar
·
Upd 30 May
·2 min
Data Pipeline Patterns for AI/ML Workloads
Practical patterns for building reliable data pipelines that feed AI and …
Patterns
Added 24 Mar
·
Upd 30 May
·4 min
Data Preparation for AI Projects
A Practical Guide
Guides
Added 24 Mar
·
Upd 30 May
·4 min
Data Quality
What data quality means for AI systems, the dimensions of data quality, …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Data Quality Validation for AI Systems
How to implement data quality validation for AI workloads using Great …
Guides
Added 28 Mar
·
Upd 30 May
·3 min
dbt vs AWS Glue for AI Data Transformation
Comparing dbt and AWS Glue for data transformation in AI pipelines, …
Comparisons
Added 28 Mar
·
Upd 14 Jun
·6 min
Delta Lake vs Apache Iceberg for Lakehouse Architecture
Comparing Delta Lake and Apache Iceberg as open table formats for …
Comparisons
Added 28 Mar
·
Upd 14 Jun
·6 min
Designing a Data Lakehouse for AI/ML Workloads
A practical guide to designing and implementing a data lakehouse …
Guides
Added 28 Mar
·
Upd 30 May
·4 min
ELT
Extract, Load, Transform
Glossary
Added 28 Mar
·
Upd 30 May
·2 min
ETL
Extract, Transform, Load
Glossary
Added 28 Mar
·
Upd 30 May
·2 min
Feature Store
What a feature store is, how it serves as a centralized repository for …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Feature Stores for Machine Learning
A Practical Guide
Guides
Added 28 Mar
·
Upd 30 May
·5 min
Implementing a Data Catalog for AI Teams
How to implement metadata management with DataHub or OpenMetadata: …
Guides
Added 28 Mar
·
Upd 30 May
·4 min
Medallion Architecture
Bronze, Silver, Gold Data Quality Layers
Frameworks
Added 28 Mar
·
Upd 30 May
·4 min
Real-Time Data Pipelines for AI Workloads
Implementation guide for real-time streaming data pipelines: four-layer …
Guides
Added 28 Mar
·
Upd 30 May
·4 min
Stream Processing
What stream processing is, how Flink, Spark Streaming, and Kafka Streams …
Glossary
Added 28 Mar
·
Upd 30 May
·3 min
Open source projects