Retained benchmark, July 2026

1,071,440 events per second ingested. Sustained. Zero measured log loss.

Production-shaped AWS, one-kibibyte events, and an acknowledgement only once each record sat on three brokers. Every event the producers offered is in the log, and the arithmetic that shows it is below.

Book a time How it was measured

Events in the replicated logCumulative, read from the high-water offsets every ten seconds

The dashed line is the 1,000,000 per second target. Ten-second readings swing with the flush timing of 112 independent shards, including one dip to 354,976. The figure that matters is the cumulative rate across the whole run: 1,071,440 a second.

Every reading, as a table

Elapsed	Durable events	Interval rate
0:10	16,187,948	1,180,049/s
0:20	28,110,700	1,188,711/s
0:30	37,126,988	895,939/s
0:40	49,118,996	1,196,911/s
0:50	60,985,288	1,185,232/s
1:00	69,631,280	863,833/s
1:10	81,504,324	1,186,375/s
1:20	93,515,340	1,197,838/s
1:30	104,760,192	1,122,993/s
1:40	114,318,828	955,107/s
1:50	126,343,740	1,201,126/s
2:00	138,354,412	1,200,081/s
2:10	147,417,388	905,094/s
2:20	157,606,168	1,018,092/s
2:30	169,639,344	1,201,348/s
2:40	181,633,924	1,197,925/s
2:50	191,217,180	955,405/s
3:00	203,131,948	1,189,478/s
3:10	215,229,716	1,209,078/s
3:20	225,904,132	1,065,722/s
3:30	238,000,280	1,207,747/s
3:40	249,055,244	1,104,682/s
3:51	259,984,760	1,084,069/s
4:01	271,945,700	1,194,141/s
4:11	284,051,032	1,208,111/s
4:21	295,362,848	1,128,943/s
4:31	300,497,132	510,341/s
4:41	305,387,476	488,207/s
4:51	312,468,880	706,949/s
5:01	324,470,584	1,198,427/s
5:11	336,520,132	1,203,585/s
5:21	347,260,956	1,072,186/s
5:31	359,056,168	1,178,513/s
5:41	371,113,340	1,204,518/s
5:51	383,161,556	1,203,205/s
6:01	394,465,268	1,128,890/s
6:11	405,340,112	1,085,632/s
6:21	417,430,616	1,207,866/s
6:31	429,529,712	1,207,096/s
6:41	441,230,572	1,169,237/s
6:51	452,135,396	1,089,707/s
7:01	464,204,292	1,205,936/s
7:11	476,017,636	1,180,440/s
7:21	488,101,528	1,206,931/s
7:31	497,920,008	981,108/s
7:41	509,983,980	1,205,044/s
7:51	522,116,484	1,211,488/s
8:01	534,197,248	1,206,306/s
8:11	544,405,852	1,019,526/s
8:21	556,359,332	1,194,352/s
8:31	568,437,216	1,206,417/s
8:41	575,144,492	669,406/s
8:51	580,013,764	486,203/s
9:01	583,578,448	354,976/s
9:11	590,926,084	733,388/s
9:21	603,158,784	1,214,025/s
9:31	615,231,768	1,205,851/s
9:41	627,236,784	1,199,511/s
9:51	638,434,892	1,115,309/s
10:01	643,577,856	511,909/s

Offered and acknowledged643,577,856

Durable in the log643,577,856

Lost0

Topology: AWS us-west-2, the AWS_PRODUCTION_LIKE profile: 8 c7g.4xlarge producer hosts running 112 shards into Amazon MSK, 6 express.m7g.8xlarge brokers, 256 partitions, replication factor 3.
Payload: 1 KiB (1,024 bytes) canonical events, four to an operation, compact wire encoding, no compression.
Durability: acks=-1 with an idempotent producer: an acknowledgement is one durable record replicated to three brokers.
Completion: Rate and loss were read from the topic’s high-water offsets, not from producer self-reports: 160,894,464 committed batches × 4 events = 643,577,856, exactly the number offered, with every one of the 112 shards having run to completion.

This measures replicated Kafka/MSK ingress on Storm’s own managed AWS profile, read from the log rather than from the producers. It is not a complete commercial-path result and not a customer-hosted result, and real-time consumer keep-up at this rate is a separate, still-open measurement.

How it was measured

One replicated log, read at its own offsets.

The problem this tier exists for, the path each event took, and what it ran on. Nothing below is a claim about the deployment you would run. It is a claim about this run.

The problem

An AI engineering product emits telemetry at the pace of its models, not its users. A solver sweep, a simulation, or a fleet of agents can produce more events in a minute than most metering systems see in a month, and every one of them is a fact about work that was done.

A meter that drops events does not meter less. It meters wrong, and nobody can say which line was affected. So the ingest tier has one job: take every event once, make it durable before the product moves on, and keep the database that holds money out of that path entirely.

The path an event took

Producers112 shards on 8 hosts
Kafka producers in TypeScript on c7g.4xlarge instances, sending one-kibibyte canonical events four to an operation, idempotently, with no compression.
acks=-1
Replicated logAmazon MSK, 6 brokers, 256 partitions
Replication factor 3, so an acknowledgement means the record is on three brokers. Both the rate and the loss count were read here, from the high-water offsets, not from the producers.
Measured here
fan-out
Consumers
- Raw archiveAmazon S3, event-level segments, every line re-verified on write
- Hot storeClickHouse, a rebuildable read model of the current period
- LedgerPostgreSQL, never on the acknowledgement path

The stack

Runtime: TypeScript on Node.js, with kafkajs as the Kafka client. The producers, the consumers, and the verifier every batch passes are the same language as the rest of Storm.
Producers: 8 × c7g.4xlarge, 112 shards, concurrency 6, 256 batches per send, idempotent, acks=-1, no compression.
Durable log: Amazon MSK Express, 6 × express.m7g.8xlarge, 256 partitions, replication factor 3. About 40% peak broker CPU at this rate, so the log was not near its ceiling.
Consumers: 4 archive and 3 hot-store workers on c7g.4xlarge, draining a prior run at the same time. The archive is event-level segments in Amazon S3; the hot store is ClickHouse on r7g.8xlarge.
Ledger: PostgreSQL on Amazon RDS, db.r6g.large. Money lives here, and it is the one component that is never on the raw-event acknowledgement path.
Provisioning: Terraform, checked into the repository, with the teardown scripted.

Retained artifact docs/benchmarks/results/2026-07-24-aws-sustained-1m-10min.json, sha256 66dd17aa19ac3662a00918864a18334f84a121db7be0548368ee22edea3a9131. Ask for the file by its hash and check it yourself. The reproduction is the same driver in duration mode against a fresh topic, with consumers primed on every partition before the window opens, and the rate and the loss read from the topic’s high-water offsets rather than from anything the producers report.

Elapsed

Durable events

Interval rate