Outbox Pattern

The Outbox Pattern provides reliable event streaming for databases without native CDC support, or when you need transactional consistency between your business logic and event publishing.

Works on all 13+ database providers. The outbox pattern is the universal fallback when native CDC is unavailable.

How It Works

The outbox pattern uses a two-phase approach: trigger-based capture and polling-based delivery.

1. Database trigger captures changes. AFTER INSERT/UPDATE/DELETE triggers write to the outbox table.
2. Bridge polls the outbox table. The SchemaBounce Bridge reads unprocessed events in batches.
3. Events are delivered to sinks. Events route through the pipeline to configured destinations.
4. Events are marked as processed. The processed_at timestamp updates and a cleanup job removes old events.

Application Write --> Your Table
Your Table --[Trigger]--> Outbox Table
Outbox Table --[Polling (500ms)]--> SchemaBounce Bridge
SchemaBounce Bridge --> Sinks (Kafka, Webhooks, etc.)

Standardized Outbox Schema

All providers use a unified outbox table structure for cross-provider consistency:

CREATE TABLE kolumn_stream_outbox (
    id BIGSERIAL PRIMARY KEY,              -- Auto-incrementing ID
    event_time TIMESTAMPTZ DEFAULT NOW(),  -- When event occurred
    op TEXT,                                -- 'i' (insert), 'u' (update), 'd' (delete)
    table_name TEXT,                        -- Source table name
    pk JSONB,                               -- Primary key of affected row
    data JSONB,                             -- Full row data (NEW for i/u, OLD for d)
    txid TEXT,                              -- Transaction ID
    origin TEXT,                            -- Provider identifier
    partition_key TEXT,                     -- Optional partitioning hint
    processed_at TIMESTAMPTZ                -- NULL if unprocessed
);

note

CockroachDB uses UUID with gen_random_uuid() and STRING types instead of BIGSERIAL and TEXT for optimal distributed performance.

Kolumn HCL Configuration

1. Create Outbox Infrastructure

create "stream_outbox" "default" {
  table_name                = "kolumn_stream_outbox"
  tables = [
    { schema = "public", table = "orders" },
    { schema = "public", table = "customers" },
  ]
  mode                      = "trigger"    # trigger | job
  retention_days            = 7            # Keep unprocessed events
  processed_retention_days  = 1            # Delete processed after 1 day
  notify_channel            = "kolumn_stream_outbox_events"  # Optional
  batch_size                = 500
}

2. Define Stream Route

create "stream_route" "orders_to_kafka" {
  source = {
    type             = "outbox"
    schema           = "public"
    table            = "orders"
    poll_interval_ms = 500     # Poll every 500ms
    batch_size       = 500
  }
  sink                = "orders_kafka"
  mode                = "outbox"
  outbox_table        = "public.kolumn_stream_outbox"
  notify_channel      = "kolumn_stream_outbox_events"

  # Optional filtering
  filter              = "operation != 'DELETE'"
}

Database Trigger Examples

Each database requires specific trigger syntax to capture changes into the outbox table.

PostgreSQL

PostgreSQL supports both trigger-based capture and NOTIFY for real-time delivery. The trigger writes to the outbox table, and an optional NOTIFY wakes the poller instantly.

-- Trigger function: capture INSERT events into outbox
CREATE OR REPLACE FUNCTION outbox_capture_insert()
RETURNS trigger AS $$
BEGIN
  INSERT INTO kolumn_stream_outbox
    (op, table_name, pk, data, txid)
  VALUES (
    'i',
    TG_TABLE_NAME,
    jsonb_build_object('id', NEW.id),
    row_to_json(NEW)::jsonb,
    txid_current()::text
  );
  -- Optional: wake poller immediately
  PERFORM pg_notify('kolumn_stream_outbox_events', TG_TABLE_NAME);
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

-- Attach to your table
CREATE TRIGGER orders_outbox_trigger
  AFTER INSERT ON orders
  FOR EACH ROW EXECUTE FUNCTION outbox_capture_insert();

MySQL

MySQL uses AFTER triggers to capture changes. Since MySQL lacks NOTIFY, the bridge polls the outbox table at a configurable interval.

-- Trigger: capture INSERT events into outbox
DELIMITER $$
CREATE TRIGGER orders_outbox_insert
AFTER INSERT ON orders
FOR EACH ROW
BEGIN
  INSERT INTO kolumn_stream_outbox
    (op, table_name, pk, data, txid)
  VALUES (
    'i',
    'orders',
    JSON_OBJECT('id', NEW.id),
    JSON_OBJECT(
      'id', NEW.id,
      'customer_id', NEW.customer_id,
      'total', NEW.total,
      'status', NEW.status
    ),
    NULL
  );
END$$
DELIMITER ;

SQL Server

SQL Server uses AFTER triggers with FOR JSON to serialize row data. The bridge polls the outbox table using the MSSQL outbox poller.

-- Trigger: capture INSERT events into outbox
CREATE TRIGGER orders_outbox_insert
ON dbo.orders
AFTER INSERT
AS
BEGIN
  SET NOCOUNT ON;

  INSERT INTO dbo.kolumn_stream_outbox
    (op, table_name, pk, data)
  SELECT
    'i',
    'orders',
    (SELECT i.id FOR JSON PATH, WITHOUT_ARRAY_WRAPPER),
    (SELECT i.* FOR JSON PATH, WITHOUT_ARRAY_WRAPPER)
  FROM inserted i;
END;

CockroachDB

CockroachDB is PostgreSQL-compatible and supports NOTIFY for real-time delivery. It uses UUID primary keys and STRING types for optimal distributed performance.

-- Trigger function: capture INSERT events into outbox
CREATE OR REPLACE FUNCTION outbox_capture_insert()
RETURNS trigger AS $$
BEGIN
  INSERT INTO kolumn_stream_outbox
    (op, table_name, pk, data, txid)
  VALUES (
    'i',
    TG_TABLE_NAME,
    jsonb_build_object('id', NEW.id),
    row_to_json(NEW)::jsonb,
    NULL  -- CockroachDB uses different txn IDs
  );
  -- Wake poller immediately via NOTIFY
  PERFORM pg_notify('kolumn_stream_outbox_events', TG_TABLE_NAME);
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

-- Attach to your table
CREATE TRIGGER orders_outbox_trigger
  AFTER INSERT ON orders
  FOR EACH ROW EXECUTE FUNCTION outbox_capture_insert();

MariaDB

MariaDB is MySQL-compatible and uses the same trigger syntax. The bridge polls using the MySQL outbox poller.

-- Trigger: capture INSERT events into outbox
DELIMITER $$
CREATE TRIGGER orders_outbox_insert
AFTER INSERT ON orders
FOR EACH ROW
BEGIN
  INSERT INTO kolumn_stream_outbox
    (op, table_name, pk, data, txid)
  VALUES (
    'i',
    'orders',
    JSON_OBJECT('id', NEW.id),
    JSON_OBJECT(
      'id', NEW.id,
      'customer_id', NEW.customer_id,
      'total', NEW.total,
      'status', NEW.status
    ),
    NULL
  );
END$$
DELIMITER ;

MongoDB

MongoDB uses Change Streams for real-time event capture. Requires a replica set (even a single-node replica set works).

// Enable replica set (if not already enabled)
// In mongod.conf: replication.replSetName: "rs0"
// Then initialize:
rs.initiate();

// Watch for changes on a collection
const pipeline = [{ $match: { operationType: { $in: ['insert', 'update', 'delete'] } } }];

const changeStream = db.collection('orders').watch(pipeline, {
  fullDocument: 'updateLookup',
});

changeStream.on('change', event => {
  // SchemaBounce Bridge handles this automatically
  // Events are forwarded to your configured sinks
  db.kolumn_stream_outbox.insertOne({
    op: event.operationType === 'insert' ? 'i' : event.operationType === 'update' ? 'u' : 'd',
    collection_name: event.ns.coll,
    pk: event.documentKey,
    data: event.fullDocument || event.documentKey,
    event_time: new Date(),
    processed_at: null,
  });
});

Provider Support Matrix

Database	Native CDC	Outbox Support	Notes
PostgreSQL	Yes	Full	NOTIFY/LISTEN for low latency
MySQL	Yes	Full	Triggers + polling
SQL Server	Yes	Full	AFTER triggers + polling
MongoDB	Yes	Full	Change Streams (requires replica set)
CockroachDB	Yes	Full	PostgreSQL-compatible, NOTIFY support
MariaDB	Yes	Full	MySQL-compatible, binlog + polling
SQLite	No	Primary	Only option for CDC
DuckDB	No	Primary	Only option for CDC
Snowflake	Yes	Optional	Scheduled query fallback

Performance and Tuning

We have not published measured throughput or latency figures for outbox capture. What we can tell you is what sets the numbers on your database. Wake latency is bounded by the notification mechanism: PostgreSQL and CockroachDB use LISTEN/NOTIFY, so a poller wakes on the commit rather than on the next tick. MySQL, MariaDB, and SQL Server use triggers plus polling, so wake latency is bounded by poll_interval_ms. MongoDB uses change streams. SQLite and DuckDB have no notification channel, so they poll on a fixed interval. Sustained throughput is bounded by your database's write capacity for the outbox table, the size of each row, and the batch size below.

Measure it on your own hardware with your own row shapes before you size anything.

High-Throughput Configuration

create "stream_outbox" "high_throughput" {
  retention_days             = 1     # Shorter retention
  processed_retention_days   = 0     # Delete immediately
  batch_size                 = 1000  # Larger batches
  notify_channel             = "outbox_notify"  # Wake pollers
}

create "stream_route" "high_throughput" {
  source = {
    poll_interval_ms = 10    # Poll every 10ms
    batch_size       = 1000
  }
}

When to Use Outbox Pattern

Use outbox when:

• Your database lacks native CDC (SQLite, DuckDB)
• You need transactional consistency
• Native CDC is misconfigured or unreliable
• You want cross-provider uniformity
• A managed database restricts log access

Use native CDC when:

• Maximum throughput is required
• Sub-50ms latency is needed
• The database supports it natively
• Zero trigger overhead is preferred

For a broader comparison between the two approaches, see CDC vs ETL and what CDC is.

Related

• Database CDC: the lower-latency alternative when your database supports native log-based capture.
• SaaS Connectors: sync data from 100+ SaaS platforms with OAuth.
• Connector Reference: the full catalog of supported SaaS connectors.
• Inbound Webhooks: receive push events in real time.
• Direct Pull: query-based syncing when triggers aren't an option either.
• Pipeline overview and how the pipeline works.
• Bridge documentation for how polling and delivery are implemented.
• Troubleshooting if outbox events aren't draining.
• See the best CDC tools in 2026 for how outbox-based approaches compare to dedicated CDC tools.