Linear Leios endorser blocks: forging, propagation, acquiring, certifying

• cardano-node 2026w45-eze (2a1ad)
• Dijkstra limits: 100,000 B of references (maxEndorserBlockReferencesSize), 1,000,000 B of transactions (maxEndorserBlockTxsSize)
• 2-in 2-out transactions, 2 destination keys, 374 B each
• Offered load (token bucket configured limit, not the one the cluster achieved): 3.28 tx/s per target (1.23 KB/s), 167.1 tx/s aggregate (62.48 KB/s) over 51 targets
• 51 nodes · AP (Sydney) 17 · EU (Frankfurt) 17 · US (N. Virginia) 17
• Mempool timeout overrides: 5.0 s capacity (MempoolTimeoutCapacity), 1.0 s soft (MempoolTimeoutSoft), 3.0 s hard (MempoolTimeoutHard)
• TxSubmissionLogicVersion = TxSubmissionLogicV2
• AWS EC2 c5d.2xlarge, x86_64 · memory 16,285,147,136 B (15.2 GiB) · kernel 6.12.30 · 8 vCPUs
• fleet CPUs: Intel Xeon Platinum 8124M @ 3.00GHz, 8223CL @ 3.00GHz, 8259CL @ 2.50GHz, 8275CL @ 3.00GHz

Setup

Topology

51 pools · 3 regions: AP 17, EU 17, US 17 · torus-dense · 306 producer entries = 204 links (102 reciprocated) · 153 intra-region links, 51 inter-region · every pool has 12 links of its own, 6 out and 6 in, 4 of them reciprocal, so 8 distinct partners · diameter 4, radius 4, mean shortest path 2.48 hops · peer sharing off (PeerSharing false in the node's configuration), so a node talks only and always to its listed producers

Full graph

Intra-region links only

Inter-region links only

Node positions are identical in all three views; hover a node to highlight its links everywhere and read its degrees. Region colours: EU green, US red, AP blue. Links are drawn undirected; direction and each pool's producer list are in the adjacency table.

Adjacency table

Node	Region	Out intra	Out inter	In intra	In inter	Producers (outbound)
node-0	EU	4	2	4	2	node-1, node-2, node-3, node-48, node-15, node-33
node-1	US	4	2	4	2	node-2, node-0, node-4, node-49, node-16, node-34
node-2	AP	4	2	4	2	node-0, node-1, node-5, node-50, node-17, node-35
node-3	EU	4	2	4	2	node-4, node-5, node-6, node-0, node-18, node-36
node-4	US	4	2	4	2	node-5, node-3, node-7, node-1, node-19, node-37
node-5	AP	4	2	4	2	node-3, node-4, node-8, node-2, node-20, node-38
node-6	EU	4	2	4	2	node-7, node-8, node-9, node-3, node-21, node-39
node-7	US	4	2	4	2	node-8, node-6, node-10, node-4, node-22, node-40
node-8	AP	4	2	4	2	node-6, node-7, node-11, node-5, node-23, node-41
node-9	EU	4	2	4	2	node-10, node-11, node-12, node-6, node-24, node-42
node-10	US	4	2	4	2	node-11, node-9, node-13, node-7, node-25, node-43
node-11	AP	4	2	4	2	node-9, node-10, node-14, node-8, node-26, node-44
node-12	EU	4	2	4	2	node-13, node-14, node-15, node-9, node-27, node-45
node-13	US	4	2	4	2	node-14, node-12, node-16, node-10, node-28, node-46
node-14	AP	4	2	4	2	node-12, node-13, node-17, node-11, node-29, node-47
node-15	EU	4	2	4	2	node-16, node-17, node-18, node-12, node-30, node-48
node-16	US	4	2	4	2	node-17, node-15, node-19, node-13, node-31, node-49
node-17	AP	4	2	4	2	node-15, node-16, node-20, node-14, node-32, node-50
node-18	EU	4	2	4	2	node-19, node-20, node-21, node-15, node-33, node-0
node-19	US	4	2	4	2	node-20, node-18, node-22, node-16, node-34, node-1
node-20	AP	4	2	4	2	node-18, node-19, node-23, node-17, node-35, node-2
node-21	EU	4	2	4	2	node-22, node-23, node-24, node-18, node-36, node-3
node-22	US	4	2	4	2	node-23, node-21, node-25, node-19, node-37, node-4
node-23	AP	4	2	4	2	node-21, node-22, node-26, node-20, node-38, node-5
node-24	EU	4	2	4	2	node-25, node-26, node-27, node-21, node-39, node-6
node-25	US	4	2	4	2	node-26, node-24, node-28, node-22, node-40, node-7
node-26	AP	4	2	4	2	node-24, node-25, node-29, node-23, node-41, node-8
node-27	EU	4	2	4	2	node-28, node-29, node-30, node-24, node-42, node-9
node-28	US	4	2	4	2	node-29, node-27, node-31, node-25, node-43, node-10
node-29	AP	4	2	4	2	node-27, node-28, node-32, node-26, node-44, node-11
node-30	EU	4	2	4	2	node-31, node-32, node-33, node-27, node-45, node-12
node-31	US	4	2	4	2	node-32, node-30, node-34, node-28, node-46, node-13
node-32	AP	4	2	4	2	node-30, node-31, node-35, node-29, node-47, node-14
node-33	EU	4	2	4	2	node-34, node-35, node-36, node-30, node-48, node-15
node-34	US	4	2	4	2	node-35, node-33, node-37, node-31, node-49, node-16
node-35	AP	4	2	4	2	node-33, node-34, node-38, node-32, node-50, node-17
node-36	EU	4	2	4	2	node-37, node-38, node-39, node-33, node-0, node-18
node-37	US	4	2	4	2	node-38, node-36, node-40, node-34, node-1, node-19
node-38	AP	4	2	4	2	node-36, node-37, node-41, node-35, node-2, node-20
node-39	EU	4	2	4	2	node-40, node-41, node-42, node-36, node-3, node-21
node-40	US	4	2	4	2	node-41, node-39, node-43, node-37, node-4, node-22
node-41	AP	4	2	4	2	node-39, node-40, node-44, node-38, node-5, node-23
node-42	EU	4	2	4	2	node-43, node-44, node-45, node-39, node-6, node-24
node-43	US	4	2	4	2	node-44, node-42, node-46, node-40, node-7, node-25
node-44	AP	4	2	4	2	node-42, node-43, node-47, node-41, node-8, node-26
node-45	EU	4	2	4	2	node-46, node-47, node-48, node-42, node-9, node-27
node-46	US	4	2	4	2	node-47, node-45, node-49, node-43, node-10, node-28
node-47	AP	4	2	4	2	node-45, node-46, node-50, node-44, node-11, node-29
node-48	EU	4	2	4	2	node-49, node-50, node-0, node-45, node-12, node-30
node-49	US	4	2	4	2	node-50, node-48, node-1, node-46, node-13, node-31
node-50	AP	4	2	4	2	node-48, node-49, node-2, node-47, node-14, node-32

Latency

cardano-cli ping round trips over the node-to-node handshake, measured at cluster start by every pool against its producers: 918 samples over 6 region pairs. Milliseconds.

Not an ICMP ping: cardano-cli ping opens a TCP connection and runs the node-to-node handshake, so each sample is an application-level round trip through both nodes' network stacks and mux, which is what the Cardano protocols actually experience. The TCP connect column is the bare TCP round trip from the same tool, the closest stand-in for a raw ping. SEM = SD / sqrt(n). The spread is 406x, from 0.62 ms within AP to 250.45 ms between AP and EU, and it is the floor under every interval in this report.

tx-centrifuge

The load generator, a closed-loop driver. A workload is a self-contained circuit: its own disjoint partition of funds, single-use UTxOs, its own builder turning funds into transactions, its own token-bucket rate limit, its own injection targets and its own observer following the chain. Every transaction spends funds and creates outputs, and an output becomes spendable again only once that workload's observer has seen its transaction reach the confirmation depth, so the transactions form chains gated by the chain itself and the funds budget is the buffer bridging confirmation latency. With on_exhaustion = error the generator fails the run the moment a node asks for transactions and it has none to hand over, whatever left it empty: too small an initial budget, funds not coming back from confirmation, a builder that cannot keep pace, or simply a target pulling faster than the loop can refill. It is one guard against the whole class of not keeping up, and it is loud rather than a quiet slowdown.

• 51 workloads and 51 observers, one of each per pool: every workload drives its own pool and no other (51 of 51), and its observer follows that same pool rather than a shared one (51 of 51)
• builder value, every transaction 2-in 2-out with 2 destination signing key per builder, fee 1,000,000
• rate limit: token_bucket, scope per_target, at 3.28 tx/s per target, 167.1 tx/s offered in aggregate (62.48 KB/s at the 374 B measured transaction size)
• recycling on_confirm at confirmation depth 3, over a nodetonode observer
• initial funds from genesis-funds (genesis_utxo_keys), scope per_workload, so a fund returned by a workload can only be taken up by that same workload
• on_exhaustion = error · max_batch_size = 0

The generator runs on its own machine in eu-central-1, which forges nothing, so every transaction crosses the network before the pool that must place it in a block ever sees it: the round trip a workload pays is the EU row for its target's region in the table above. Each workload holds two buffers of its own, funds waiting to be built and built transactions waiting to be delivered.

1. Forging, generated load

1.1. Totals

Every EB forging found, nothing filtered. A block that lost its height battle, that was rolled away from, or that no other node ever adopted still counts.

By region

Endorser block contents

What an endorser block holds. An endorser block is a list of (tx hash, size) references and nothing else, so its size is that list's size.

The transaction closure is what binds this run. Its transactions are 374 B each, so its largest endorser block holds 2,673 of them: 98,904 B of references against the 100,000 B cap (98.9%), and 999,702 B of transactions against the 1,000,000 B the closure allows (100.0%). The one that reaches 100% is the one that stopped the endorser block from growing.

The other run of this pair, 2026-11-06-20-39-a0c13-w45notxsub-value-leios-workload-all-n2n-100k1m-2i2o2w-nomadperf-dira-nom, is the control for exactly this: its transactions are 374 B, its largest endorser block holds 2,673 references for 98,904 B (98.9% of the reference cap), and their closure is 999,702 B (100.0% of the closure cap). With 374 B transactions the closure fills first, with 374 B transactions the closure does, off the same two caps. So the same cap binds in both, and the transaction size did not move it.

1.2. Start Leadership Check

A slot's start is fixed by the protocol, the system start plus the slot length, and everything here is measured against it. A node traces every slot it enters, so a slot with no trace is one it never entered, and each node is counted over its own window, from the first slot it entered to its last.

Forging in the slot

One value per block, from the start of its own slot to its forging: every ranking block in the first row, every forging that carries an endorser block in the second, so a hash forged twice counts twice. Section 6 measures that second stage again from the same origin but counts each hash once, at its first forging, which is why its samples are fewer.

Per node

Every row holds one value per node, counted over that node's own window, so the samples are nodes and the spread is across the cluster rather than over time.

By region

1.3. Timeseries

The load as it ran, per minute: the generator's own side first, then what the nodes' mempools did with it. Every row is split, the first hour on the left and the rest of the run on the right, each half on the scale its own window needs. A region's band runs from the lowest of its 17 nodes to the highest, the line inside it is their mean and the dashed line the mean over all 51. Hover a row for the values at that minute.

Workload, tx-centrifuge

The generator's own traces: the closed loop of funds, what it delivered, how fast, and what each target asked for and did not get.

Node mempool

The nodes' own traces: what each mempool held, how fast transactions entered and left it, and what it turned away.

2. Propagation, fetching the EB references

Three instants are used:

• The forger forges the endorser block (Consensus.LeiosKernel.BlockForged)
• The forger announces it over the network (LeiosNotify.Remote.Send.BlockAnnouncement)
• A peer finishes receiving the endorser block's bytes over the network (LeiosFetch.Remote.Receive.Block)

Where an endorser block's hash was forged more than once, and so referenced by more than one ranking block, the first forging is the one used and the later ones are ignored: the instants this section reads are the first for that hash and node, so they belong to the first forging.

2.1. Announcing

The forger alone: from forging the endorser block to sending its first announcement of it.

2.2. Fetching

From the forger's first announcement of the endorser block to that endorser block first arriving at a peer. Repeated deliveries of the same endorser block are counted under Activity, never timed here.

By region

Spread over the entire cluster

One value per endorser block: the delay by which that share of the other nodes had first received it, measured from the forger's first announcement as above. The 100% row holds each endorser block's slowest node, so it sits above the table of every node and endorser block.

2.3. Activity

The same events counted instead of timed, and counted per peer and endorser block: for one endorser block, how many announcements that peer sent out, and how many times it fetched the endorser block. Anything above one is a repeat the protocol paid for. The two announcement figures below count every node, each forger's own sends included; the regional table counts the receiving peers only, which is why its rows add to one node less.

Times a node fetched the same endorser block

Inbound: the node pulling the endorser block from a peer. One means it fetched it once, more than one means it fetched the same endorser block again.

By region

3. Acquiring, the EB written down

A delivered endorser block is not yet held: until its write is durable the body still reads as not-held, which is why the same endorser block can be fetched again meanwhile. Consensus.LeiosKernel.BlockAcquired is the moment that ends: the node has the reference list stored and claims to hold it, registering which referenced transactions it still needs. This is the step from arrival to that claim, which includes waiting for the single writer, and how often the same endorser block was stored again.

3.1. Writing

By region

Spread over the entire cluster

One value per endorser block: the delay by which that share of the other nodes had first stored it, each measured from that node's own fetch of the endorser block. The 100% row holds each endorser block's slowest node, so it sits above the table of every node and endorser block.

3.2. Activity

Counted per peer and endorser block, like section 2: how many times a peer wrote the same endorser block down. This one is local, nothing crosses the network.

Times a node wrote the same endorser block

By region

4. Closures, the transactions an EB references

An endorser block is a list of references. The transactions themselves are a separate transfer, and a node has the closure once it holds all of them: Consensus.LeiosKernel.BlockTxsAcquired. Counted per peer and endorser block, on the same endorser blocks as sections 2 and 3.

4.1. Obtaining

A voter has until 7 s after the announcing block's slot onset to hold and validate the transactions: lHdrWait 3 s then lVoteWindow 4 s, both fixed in LeiosVoting.hs rather than read from genesis. 70 of 151,330 closures were obtained later than that, counted from that slot's own start.

By region

Spread over the entire cluster

One value per endorser block: the delay by which that share of the other nodes had first obtained the closure, each measured from that node's own fetch of the endorser block. The 100% row holds each endorser block's slowest node, so it sits above the table of every node and endorser block.

4.2. Activity

Times a node obtained the same closure

By region

4.3. Body hits

When a node takes an endorser block it checks the transactions it names against its own mempool, and the kernel reports what it found (Consensus.LeiosKernel.BodyHits): one case per peer and endorser block, 154,358 of them over 3,027 endorser blocks.

Both rows are the same division: transactions the node had, over transactions the block names. For several cases together, add the tops, add the bottoms, divide once.

Do not add the two rows. Held in the mempool plus fetched is already all of them, in every one of the 154,358 cases, so the cache row counts transactions the mempool already had.

One case is one peer and one endorser block, and the share is against that block's own reference list, 2,673 references at the median.

By region

Forger region against peer region tracked in the cache / held in the mempool

Each cell is for endorser blocks forged in the row's region and checked by a node in the column's, so the diagonal is same-region. Both shares are totals over totals, every case of that pair pooled, not a mean of the cases.

Timeseries

The same shortfall over the run, per minute. The first chart is one dot per endorser block, the share of what it named that its peers were missing. The other three take the peers of one region and split that share by the forger's region: each line is that minute's cases pooled and the band runs from the lowest of the seventeen peers to the highest. Hover a row for the values at that minute.

5. Certifying

5.1. Totals

Counted on the blocks the whole cluster took. A block qualifies when all 50 nodes other than its forger adopted it, every peer it has of the 51 in the cluster: 2,938 of the run's 3,056 walkable blocks. Both sides of the ratio are held to it, so an announcement made by a block only part of the cluster held, and a certificate carried by one, are both out. No battle or fork filter is applied: nothing here asks what happened at a contested height. The blocks come from every forging the logs can place on a chain, both sides of a fork included, and the test above is adoption by every peer, not having won the height.

By region

5.2. What happened to every endorser block

Hover a stage for what it counts and which trace says so. The stages down to Never reached quorum count endorser blocks; the rows under it count voter decisions, one per trace event, which is why they sum past 100%. Nothing is gated here, not even the run's opening epochs: every endorser block any ranking block announced is counted, which is why this table is wider than the cards above. 118 endorser blocks were announced only by blocks that part of the cluster never adopted, and the two carried counts differ by 10, certificates whose carrier was not adopted by every peer. tooLate is the voter's own refusal, counted once per voter and endorser block and only on the ones that never reached quorum: a node logs it when the window had already shut before it began, whether or not it ever held the transactions, so it is not section 4's late closures.

6. Against the slot clock

6.1. RBs

Every ranking block against every peer that adopted it: from the block's slot start to that peer's own adoption, one value per pair. Nothing here is a per-block figure, so a block the whole cluster took contributes as many values as it had peers.

Adoptions

Left out: 960 block and peer pairs the block never reached, and 1,640 with no adoption instant.

Spread over the entire cluster

One value per ranking block: the delay by which that share of its own peers had adopted it, from the block's slot start. One table per population, named in its first column.

6.2. EBs

The same stages as the sections above, each from the start of the slot the endorser block belongs to instead of from the stage before it, so they read against each other and against the protocol's windows. A slot is 1.00 s.

A row naming each peer holds one value per peer and endorser block, never the first peer alone; one naming the last peer holds one per endorser block, that block's slowest. Every endorser block the run forged is in, counted once.

A stage past 1.00 s is a stage the next slot had already begun. 7 of 3,027 endorser blocks were forged past it and 9 announced past it, and 3,022 of them reached their last peer past it. Left out for want of an instant: 7 cases where a peer never fetched the block, 8 where it never held the closure, and 0 blocks the forger never announced.

By region

7. Forks

Every time a node abandoned a chain it already had for a better one, counted once for each node that did it (ChainDB.AddBlockEvent.SwitchedToAFork). A contest is one pair of tips, so a contest the whole cluster lived through counts once. Both rates below are contests per 1,000 blocks on the chain, so runs of different length compare directly.

The slot rate is the control and the height rate is the signal: two leaders in one slot is fixed by activeSlotsCoeff and cannot be engineered away, while a height battle is a node forging on a parent a sibling already had, which is propagation losing a race and is free to move. Reading the height rate without the control beside it cannot tell a propagation regression from a run that simply drew more forks.

7.1. What the nodes paid