Skip to main content

CDC vs ETL: What's the Difference and When to Use Each

· 8 min read
Elliot Henderson
Organized by AI

ETL Gives You Yesterday's Data. CDC Gives You Now.​

The first fork in any data pipeline design is fundamental: do you move data in scheduled batches (ETL) or capture changes as they happen (CDC)?

These are not interchangeable tools. They solve different problems, impose different infrastructure requirements, and have wildly different cost and latency profiles. The shift toward real-time analytics has turned what was once a simple scheduling question into a critical architectural decision. For the full architectural picture of CDC itself, see What is Change Data Capture (CDC)?

ETL: The Batch Workhorse​

ETL (Extract, Transform, Load) connects to your source database on a schedule, pulls data, transforms it, and loads it into a warehouse. Run it nightly, hourly, or every 15 minutes: whatever your SLA requires.

ETL is synchronous and deliberate. You control what moves, when, and how. Tools like Airflow, dbt, Informatica, and Talend orchestrate the process.

Where ETL wins:

  • Simplicity. Mature tooling, abundant documentation, well-understood patterns. You're not solving novel problems.
  • Complete transform control. Business rules, data quality checks, and schema evolution are explicit, version-controlled, and auditable.
  • Lower infrastructure. Jobs run, finish, and consume zero resources while idle. No 24/7 stream processor required.
  • Predictable costs. Four batch jobs a day on a 16-core server for 30 minutes each? You know the bill.

Where ETL breaks down:

  • Latency. If your ETL runs daily, your analytics are 24 hours stale. Even hourly jobs mean hours of lag, unacceptable for fraud detection, inventory systems, or customer-facing dashboards.
  • Database load. ETL scans large tables even when few rows changed. A daily job extracting 100 million rows to find 10,000 changes puts real stress on your source.
  • Missed deletes. Standard ETL struggles to capture row deletions cleanly. You need soft deletes, temporal tables, or manual tracking logic.
  • Schema brittleness. Source schema changes break extraction logic, transforms, and downstream schemas. Teams become cautious about evolving schemas.

CDC: The Real-Time Stream​

Change Data Capture captures inserts, updates, and deletes as they happen at the source database and streams them downstream.

Log-based CDC reads directly from the database's transaction log (PostgreSQL WAL, MySQL binlog, Oracle redo logs); see database CDC sources for the mechanics. Query-based CDC polls for changed rows using timestamps, the approach behind a direct pull source. Log-based is more complete and lower-latency because it captures every change as the database writes it.

Where CDC wins:

  • Sub-second latency. Changes hit downstream systems within milliseconds. Real-time dashboards, event-driven workflows, and immediate reactions to business events become possible.
  • Minimal source impact. Log-based CDC reads from the transaction log the database is already writing. No extra queries, no full-table scans.
  • Full change history. Price changed three times in one second? You see all three events, including each step. Not only the final value.
  • Native delete capture. CDC captures DELETE operations as naturally as inserts.
  • Event-driven architecture. New customer triggers onboarding. Order state change triggers fulfillment. Inventory update triggers availability check. CDC maps directly to this pattern, and the outbox pattern is the reliable way to publish those events without dual-write bugs.

Where CDC adds complexity:

  • More infrastructure. Message broker (Kafka, Pulsar), CDC connector, stream processing logic, and downstream consumers. More moving parts to monitor.
  • Schema evolution handling. Schema registry updates, versioning, and downstream compatibility require deliberate management.
  • At-least-once semantics. Distributed streaming systems can deliver changes multiple times by default. Downstream systems must be idempotent.
  • Raw changes only. CDC captures what changed, not your business logic. Calculated fields, lookups, and aggregations still need a transformation layer.

Head-to-Head​

AspectETLCDC
Data FreshnessHours to daysSeconds to milliseconds
Source Database ImpactHigh (full scans)Low (reads transaction logs)
Captures DeletesRequires special logicNative
Resource UsageBursty (during job window)Continuous low-level
Schema HandlingRigid; changes break jobsFlexible with schema registry
Best ForHistorical analysis, warehousingReal-time analytics, event-driven systems

When ETL Is Right​

Data warehouse loading. Scheduled daily or hourly ETL jobs feeding a dimensional warehouse are simpler to operate than continuous CDC. If you're building for analytics and reporting, not driving real-time decisions, ETL works.

Complex transformations. Joining multiple sources, calculating metrics, applying ML models: ETL's explicit transform layer is easier to reason about and test than stream processing. Develop and test in SQL and dbt before production.

Lower operational overhead. Schedule a job with Airflow, run it, done. No message broker, no stream consumer, no exactly-once semantics.

Cost constraints. ETL runs during off-peak hours on spot instances. CDC requires always-on infrastructure.

When CDC Is Right​

Real-time dashboards. A live business metrics dashboard showing revenue, signups, and active users needs CDC. ETL's hourly freshness feels broken for operational visibility.

Event-driven systems. Triggering workflows, notifications, or updates based on specific data changes (new customer, shipped order, low inventory) maps naturally to CDC's architecture. An inbound webhook source covers the reverse direction, other systems pushing events into your pipeline.

Microservices sync. Multiple services staying in sync based on changes in a central database. CDC streams the source of truth. See the connector reference for which databases support this today.

Compliance and audit. CDC's immutable change stream provides a natural audit trail. Complete record of who changed what and when.

The Hybrid Approach: Use Both​

Most mature data organizations run ETL and CDC side by side.

Pattern 1: CDC for ops, micro-batch for warehouse. Stream real-time changes for dashboards and event-driven workflows. Aggregate the same CDC stream into 5-minute micro-batches for your warehouse. Fresher than daily ETL, simpler than powering a warehouse directly from real-time events.

Pattern 2: Split by data type. CDC for operational databases (customers, orders, inventory) where real-time sync matters. Traditional ETL for analytical loads where latency is acceptable.

Pattern 3: CDC to data lake. Capture everything with CDC, stream to a data lake in Parquet format (timestamped, partitioned by change date). Downstream consumers query the lake directly. No ETL job required. Completeness and freshness of CDC, simplicity of a data lake.

Real-World Example: E-Commerce​

With ETL: Nightly job extracts yesterday's orders, normalizes categories, converts currencies, deduplicates customers, loads to warehouse. Inventory manager checks the dashboard each morning. Low stock triggers a manual reorder.

With CDC: Each order flows to Kafka as it happens. A real-time consumer updates inventory immediately and auto-triggers reorders below threshold. Order data streams to a SaaS connector like Salesforce within milliseconds. Customer service sees live information: most recent order, current loyalty balance, recent messages.

Hybrid: CDC streams for real-time operations. Every 5 minutes, those changes also aggregate into the warehouse. Analysts still get their warehouse, just 5-minute fresh instead of daily. Operational systems get real-time.

How to Decide​

Ask five questions:

  1. What latency do you need? "Within 24 hours" = ETL is simpler. "Within seconds" = CDC is necessary.
  2. How much source load can you tolerate? Full-table scans during peak hours are expensive. CDC has lower impact.
  3. What operational overhead can you support? Streaming infrastructure (Kafka, consumers, monitoring) requires expertise.
  4. What's downstream? Data warehouse = ETL is native. Event-driven services = CDC is native.
  5. What's the budget? Factor infrastructure costs, engineering time, and operational burden. Simple expensive ETL sometimes beats complex cheap CDC.

Most teams won't answer "either/or." The real question is "which primary approach, with what secondary support?" A warehouse-heavy company might be 80% ETL with CDC filling real-time gaps. An event-driven company might be 80% CDC with small ETL jobs for batch exports.

The Bottom Line​

CDC and ETL solve different problems. ETL remains foundational for batch analytics and warehousing. CDC is the modern tool for real-time systems, operational dashboards, and event-driven architectures.

The shift toward CDC reflects the industry's move toward real-time data. But ETL isn't obsolete. It means choosing the right tool for each part of your pipeline, and increasingly, using both. If you want to see log-based CDC configured end to end on a real database, the PostgreSQL CDC tutorial walks through it, and Best CDC Tools in 2026 compares the platforms that implement it.

If you're evaluating real-time CDC, SchemaBounce offers sub-second streaming with a free tier. No enterprise pricing commitment required to start experimenting.

Tags:
  • cdc
  • etl
  • data-pipelines
  • batch-vs-streaming