Streaming demo: one engine ingests, materializes and serves
You are looking at Binance live market streamlive Binance aggTrades as 64-byte containers, durable on disk the instant they land in one Semurg substrate. Ask it to append a live burst and recompute the materialized view concurrently, one engine and it runs the
real query live on this box, on data you can
download and re-run yourself.
Click an example in the agent rail, or ask to run (appends a live burst), the page loads read-only until you do. Every run is measured end to end, on
this box, on the real dataset. Nothing is pre-computed.
The Binance live trade stream (aggTrades, BTC/ETH) flowing into the substrate. We append a burst of trades into the live shard and recompute the materialized aggregate over the same resident stream at the same time. One durable copy, no log, no processor, no separate serving store. Measured live.
the stream-processor leader
THREE SYSTEMS, THREE COPIES. A live feed needs a log to ingest, a stream-processor to maintain the view, and a serving store to answer reads, copied across all three with a consistency lag between them.
semurg, one engine
1 engine, not 3. Ingest, materialize and serve are the same engine on one durable copy, so the append, the running aggregate and the read run concurrently with no connector, no copy and no lag.
Two whole systems and every cross-copy disappear.
The live Binance aggTrade stream (BTC/ETH). Measured live on this box. Re-subscribe on the Binance socket and re-run it yourself.
8.9Mtrades/s materialized (read-only, ask to run to append)
0trades appended
8.9Mtrades/s materialized
2.4Mview scope
100.0%resident (hot)
8cores
1engine (not 3)
read-only view recompute (no append on page load) · materialized view = a fanned warm scan, no inverted index, no separate view store · aggregate swept in 265.3 ms
the answer
recompute the materialized view over the resident stream (read-only)
the streaming answer
live result
trades appended (live burst)
0
trades folded into the view
2.4M
trades/s materialized
8.9M
engines in the loop
1 (not 3)
ingest, materialize and serve on one durable copy, no connector, no cross-copy lag
8.9Mtrades/s
materialized the view over the resident stream, one engine, no separate view store
10xLeading
collapse: ingest, materialize and serve on one engine
one engine, not three, no cross-copy lag
hit-rate + reads/s · cold / warm / hot
cold 66.3M/s
warm 8.9M/s
hot 4.26B/s
reads/s at each tier · cold straight off NVMe · warm from the RAM ring · hot from CPU cache
RAM utilization · working-set vs dataset
5.1 GB anonymous RAM (the pipe) · 5.3 GB resident incl. reclaimable page cache · 1.4GB on disk · RAM is the pipe, not the store
where the data was when the CPU read it
COLD on disk, straight off the NVMe
WARM in the RAM pipe (the conveyor ring)
HOT already in CPU cache
Nearly all hot and almost no cold, with a tiny resident working-set over a large
on-disk dataset, is the whole point: disk-capacity storage running at cache speed. RAM
here is a pipe, not the store. A normal engine only reaches this speed by first loading the
entire dataset into expensive RAM. This one keeps the truth on disk and streams only what the
query touches, so the resident bar above stays small while the reads stay hot.
This is a COLLAPSE, not a raw-speed claim. A dedicated stream-processor can sustain a high append rate too. The win is that ingest, materialization, and serving are the SAME engine on ONE durable copy, so the eliminated tax is the whole second and third system: no connector from log to processor, no copy into a serving store, no consistency lag from stream to view to read. One engine, one copy, and the same substrate also serves graph, full-text and vectors.