6. Latency and scale

Measured, not estimated. Anything not measured says so.

How these were taken

test/load-tenant.js, against one server process on a 10-core, 16 GB laptop, over loopback. Re-measured 31 August 2026; every figure below comes from that run.

Not test/load-crowd.js, and the difference matters. That one drives role=view, a viewer on our own platform, which needs our video: with nothing publishing, every viewer is closed with 4004 and the run reports zeros that read as a clean result. This one drives role=crowd, which is the shape a licensee integrates and takes no video at all. Viewers connect as role=crowd: a licensee's audience, carrying no video, which is the shape an integration actually runs. Each viewer acts about once a second, which is a hot crowd rather than an average one.

Loopback means no network in these figures. Real latency is these numbers plus the round trip to the viewer, which is tens of milliseconds within a region and over a hundred across an ocean. The server's own cost is what is measured here, and it is the part we control.

Latency

Two different numbers, and they answer different questions.

Act ack is the viewer pressing something and being told what it cost them. This is the responsiveness a viewer feels, and it grows slowly with the crowd.

Batch lag is the same press reaching the game in its next tally. It is essentially one tick and it is flat from 1 000 to 10 000 viewers, because batching is on a timer rather than on the queue: the crowd getting larger makes each batch bigger, not later. TICK_MS is 250, so this number is a uniform draw from 0 to 250 ms and a p95 of ~250 is the design working, not a queue forming.

Add the network and a viewer press lands in the game in roughly 200 to 350 ms end to end.

Against the video delay

Twitch and comparable platforms put 2 to 8 seconds between a moment happening and a viewer seeing it. Interaction is an order of magnitude faster than that, so the interesting problem is the reverse of the one people expect: not that the click is slow, but that the viewer is always acting on a stale picture.

The engine is deadline based for that reason. A round carries a server-side deadline, and the game is expected to render the countdown into the video itself, so what the viewer sees and what the server believes are one clock. Betting closes on that deadline, server-side, and settle() closes the round before it pays, so a stake cannot be placed against a known outcome.

Throughput and what degrades first

At 10 000 viewers, one process carried 9 124 intents per second and refused none of them. At 16 337 it carried 15 044 per second, also refusing none.

Read that as offered load rather than as a ceiling, because that is what it is. The test drives each viewer at roughly one intent per second, so this number is the crowd size and not a limit the server was pushed to. What it says is that the server kept up with everything sent, which is the useful claim; what it does not say is where it would stop.

What degrades first is not latency, it is the per-tick cap, on purpose. Each action declares maxPerTick, and intents beyond it are counted and dropped rather than queued, so the game is never handed more than it agreed to consume. At 10 000 viewers with a 1 000 cap, 63 715 of 196 188 intents were dropped that way. The game still gets the shape of the crowd, and it never gets a flood. A game that wants more raises its own cap.

The ceiling we actually hit

At 20 000 requested viewers, 16 337 stayed up and the server served them at a p95 act ack of 37 ms. At 30 000 requested, only 6 832 opened: 23 168 connections failed and the attempt took 20 seconds.

That was the test machine, not the server, and the run says so itself. With 6 832 viewers up the server's latency improved, to a p95 of 13 ms, because it had fewer connections to serve while the client could not open them fast enough. A saturated server gets slower under load; this one got faster when the load stopped arriving. macOS also caps the accept backlog at 128 (kern.ipc.somaxconn), so a burst of simultaneous connects is refused by the kernel before the process sees it.

So the honest statement is: measured to 16 337 concurrent on one process, with nothing refused and the server showing no sign of strain. Where it actually breaks is still not known, because this test still could not reach it.

50 000 and 500 000

50 000 on one machine is plausible and untested. The measured figures extrapolate to roughly 600 MB of memory. Node runs the socket loop on a single thread, so one core is the real ceiling, and neither TLS nor real network jitter is in the measurement. Nothing here should be read as a promise at that size.

500 000 is not possible on today's architecture, and not because of any of the above. One process holds the channel registry, every open round and the event log's sequence, and fly.toml says to run exactly one machine for that reason. A second machine today is two half platforms: a viewer routed to the wrong one sees a live stream as offline and a bet settles against a different pool.

Getting past it is understood rather than done: a shared channel registry, a message bus between processes, and the event sequence handed out centrally instead of held in memory. Sharding by channel is the natural first cut, because a round never spans two channels.

State this plainly in partner conversations. The engine's per-viewer cost is low enough that one machine goes a long way, and the honest number today is five figures per channel, not six. The measurement puts that at sixteen thousand and rising rather than at eleven.

The cash-out market under load

Measured on the same machine as the rest, August 21st. The LMSR maker prices and books ~480,000 trades per second on one core in isolation (10,000 trades across 500 funded viewers in 21 ms, engine and in-memory ledger included). Pricing is never the bound; the socket layer is, and its numbers are the ones above. 2,000 interaction-only viewers connect in 1.3 s (~1,500 sockets/s fan-in).

One machine, on purpose, and the path out

Today the deployment is a single machine, and that is a choice rather than an accident: rounds, rate limits and OAuth handshakes live in process memory, which is exactly what makes them fast, and the durable ledger is written through to Postgres so a restart loses no money. Shutdown drains every open round: stakes are refunded before the process exits, so a deploy never eats a bet.

What that buys: no coordination cost, no cache invalidation, no split-brain settlement. What it costs: one region, and a ceiling in the five figures of concurrent viewers per channel.

The path out, when a deal needs it, in order:

1. Video off the box first. HLS behind a CDN already exists for exactly this; the WebSocket player stays for the low-latency tier. 2. Shard by channel. A round never spans two channels, so channels are the natural unit: a router pins a channel to a machine and nothing about the engine changes. 3. Shared state last. Rate limits and sessions move to a shared store only when step 2 is not enough, because that is the step that buys scale at the price of latency.

None of this is speculative work to do now. It is the answer to give when a partner asks, and the reason the answer is credible is that steps 1 and 2 require no engine changes at all.

Part of the IMPAKT crowdplay integration pack. All documents.