10 posts tagged with "throughput"

This week, the Leios team conducted extensive experiments using the previously developed network topologies to study transaction and input block throughput limits under realistic conditions. The team also conducted empirical bandwidth measurements between data centers, advanced formal methods capabilities, and created initial CDDL specifications for core Leios components.

CDDL specification draft​

• Created initial CDDL specifications for core Leios components:
  • Input blocks with VRF lottery and single IB/slot limits
  • Endorser blocks as a new aggregation block type
  • Ranking blocks as Conway extension with optional certificates
  • BLS voting system with persistent/non-persistent voters and key registration
• Followed crypto-benchmarks implementation approach while maintaining Conway CDDL compatibility
• Established foundational structures in the first draft covering common base components
• Future iterations will add detailed specifications for design variants, including full sharding, overcollateralization, and protocol extensions.

Formal methods​

• Added support for Late IB inclusion to the formal specification of Full-Short Leios
• Profiled leios-trace-verifier performance, identifying that approximately 60% of execution time is spent in garbage collection
• Improved performance significantly by switching to --nonmoving-gc garbage collection strategy.

Bandwidth measurements​

• Conducted empirical bandwidth measurements using iperf3 between data centers in North America and Europe
• Measured bidirectional connections across multiple cloud providers (OVH, AWS, CenturyLink)
• Results ranged from 95 Mbps to 973 Mbps, depending on geographic distance and the provider
• Identified 100 Mbps as a conservative lower bound for inter-datacenter connections
• Observed 5-20% reduction in individual link speeds when multiple simultaneous connections are active.

Large-scale network experiments​

• Conducted comprehensive experiments using both the 750-node and 10,000-node network topologies with Haskell and Rust simulations
• Studied transaction and IB throughput limits for realistic scenarios up to 300 TPS and 32 IB/s
• Key findings from the 750-node mini-mainnet experiments are documented in analysis results and summary slides:
  • The 750-node mini-mainnet serves as a suitable replacement for the 10,000-node pseudo mainnet for performance measurements
  • Substantial agreement between Haskell and Rust simulations for mini-mainnet scenarios
  • Block propagation times under one second, consistent with empirical observations from pooltool.io
  • Protocol can support 25 MB/s throughput with 1 Gb/s links before degradation
  • Mean transaction time from mempool to ledger is approximately 150 seconds
  • Achieved 80% disk-space efficiency with ~20% network traffic overhead
  • Six-core VM is sufficient for peak demand at 300 TPS, with average demand under two cores
• Results from 10,000-node pseudo-mainnet experiments are available in analysis documentation and presentation slides:
  • Average transaction lifecycle of 100 seconds from mempool to ledger
  • Approximately 80% efficiency for both disk and network usage
  • Six CPU cores are sufficient for peak load handling even at high TPS rates
  • Block propagation time averaged under one second across the large network.

This week, the Leios team made significant progress in protocol development, focusing on improving simulation capabilities and analyzing protocol behavior under various network conditions. A comparison of Haskell and Rust simulations across 18 scenarios demonstrated that the Leios protocol scales effectively to mainnet-sized networks. However, congestion occurs when the input block rate reaches 30 IB/s.

Simulation comparison​

• Compared 18 scenarios between Haskell and Rust simulations at tag leios-2025w12
• Recent fixes and adjustments enabled meaningful comparison between simulations
• Identified differences when comparing the Haskell and Rust results, which are under active investigation.

Analysis of simulations​

• Completed the first simulation of Short Leios, evaluating IB production rate, IB size, and network topology
• Demonstrated that the Leios protocol scales effectively to mainnet-sized networks
• Identified congestion occurring when the input block rate exceeds 30 IB/s
• Suggested that allowing IBs larger than current Praos RBs may have advantages in TCP efficiency, network usage, and adapting to fluctuating transaction loads.

Peak CPU	Mean CPU

Haskell simulation​

• Implemented expiration of blocks:
  • Blocks are removed from the relay buffer once diffusion stops and cleared from other states as specified
• Developed an initial Full Leios implementation:
  • Currently in early testing
  • Added the praos-chain-quality configuration parameter for the \eta parameter from the specification.

Rust simulation​

• Developed an initial Full Leios implementation using estimated values for some parameters.

Formal methods​

• Short Leios trace verification: modeling local state evolution of a node
• Developed an initial trace verifier for Short Leios simulation traces in leios-trace-verifier.

This week in Leios development, CPS-0018 for transaction throughput was approved, along with improved Docker support for simulations and analysis of cross-simulation results. The team also examined input block (IB) production rates and their impact on network performance.

Protocol development​

• CPS-0018 'Greater transaction throughput' officially approved:
  • Merged into Cardano Foundation's CIP/CPS repository
  • Documents urgency of higher transaction throughput
  • Defines goals for the Leios initiative
  • Identifies key open questions and use cases.

Cross-simulation analysis​

• Conducted a comprehensive analysis of IB production rates ranging from 1 IB/s to 100 IB/s:
  • Developed an ELT workflow for data processing via MongoDB
  • Created an R Jupyter notebook for analysis and visualization
  • Identified and addressed three significant bugs (#207, #208, #209)
• Key findings from the Haskell simulation:
  • Network congestion emerges at high IB production rates
  • Both average propagation time and slow propagation tail increase
  • A critical threshold of ~40 IBs/s was identified, beyond which network congestion severely impacts block reception
• Comparison of PeerNet and Haskell simulations:
  • Both exhibit qualitatively similar block propagation distributions
  • Both demonstrate protocol breakdown under high block production rates
  • Differences in resolution and configuration prevent exact comparison.

Infrastructure improvements​

• Added comprehensive Docker support for both simulations:
  • Optimized multi-stage Docker files for Haskell and Rust
  • Simplified deployment process
  • Enabled easy configuration via volume mounts and parameters
  • Documented usage in README.md.

Rust simulation​

• Enhanced Rust simulation capabilities:
  • Implemented bandwidth usage tracking
  • Added configurable bandwidth limits per connection
  • Fixed issues identified in cross-simulation comparisons
  • Started updating visualizations for improved clarity.

Haskell simulation​

• Enhanced IB sortition handling for IB/slot < 1
• Began integrating block expiration and diffusion-halt proposal
• Implemented ideal timing calculations for diffusion:
  • Added uniform block behavior configuration
  • Identified relay mini-protocol complexities:
    • Variable latency (3-4) for block transfer
    • Latency depends on traffic conditions and request handling.

Formal methods​

• Moved formal specification to a dedicated repository
• Established a conformance testing framework:
  • Enabled testing between Short Leios implementations
  • Documented the test suite execution process
• Initiated a survey of network models across IO consensus projects.

This week, the Leios team worked on cryptography benchmarking and cost calculator improvements. The team completed a reference implementation for Leios cryptography and enhanced the online cost calculator with user-requested features. They also updated both Haskell and Rust simulations to improve visualization and network modeling capabilities.

Haskell simulation​

• Added support for Send and Receive voting stages, providing:
  • A new leios-vote-send-recv-stages configuration option
  • A configurable stage length via leios-stage-active-voting-slots
• Implemented multiple diffusion strategies:
  • Added oldest-first strategy
  • Added configurable strategies for IBs, EBs, and votes via *-diffusion-strategy configurations
• Created a new small scenario for 100 nodes with 2,000 kB links
  • Tuned IB parameters to utilize one-third of link capacity
  • Added configurations for both single-stage and send-recv voting
• Fixed several simulation behaviors:
  • Improved block generation logic
  • Prevented duplicate EB inclusion in the base chain
  • Confirmed proper EB inclusion timing relative to vote diffusion
• The main difference observed between single-stage and send-recv is that the former shows a longer tail in the CPU usage CDF when the simulation is run with unlimited cores.

Cryptography implementation​

The Rust benchmarks for Leios cryptography were redesigned as a reference implementation:

• Implemented the Fait Accompli sortition
• Enhanced sortition to use rational arithmetic instead of quad-precision floats
• Added Quickcheck tests for all capabilities
• Added benchmarks for serialization
• Optimized vote and certificate size.

Cost calculator improvements​

The team enhanced the online Leios cost calculator:

• Added support for both hyperscale and discount cloud providers
• Made discount providers the default option
• Added option to amortize storage costs perpetually
• Updated defaults:
  • Single relay deployment
  • More conservative 50% disk compression
  • Perpetual storage cost amortization.

Throughput simulator​

The team updated the Cardano throughput simulator with:

• The latest cloud-computing cost model
• Synchronized assumptions with an online cost calculator.

Rust simulation​

• Made minor fixes to the new graph generation strategy
• Planned out a roadmap for visualization work focusing on the Leios transaction lifecycle.

GitHub actions​

• Organized continuous integration (CI) configuration with namespace prefixes for better project sorting.

Rust simulation​

• Began visualizing transaction throughput
• Tweaked configuration settings to maximize throughput.

Haskell simulation​

• Merged code for running Praos and Leios visualizations
• Added HLint integration for consistent module imports.

Public meeting on December 28, 2024​

• Meeting recording
• Meeting slides.

Visualizing Node Behavior​

• Added a web server to render throughput of IBs and EBs in the Leios simulation.
• Utilized Tracer interface for JSON-formatted traces and live graph plotting.

Discussion with Nicolas​

• Explored Leios protocol with a focus on IB to EB construction.
• Modeled a single node with input and output queues for IBs and EBs.

Meeting with Research​

• Discussed ongoing research work and networking challenges.
• Aimed to define a ΔQ model for Leios and explore protocol structure.

Public Meeting on November 27, 2024​

• Meeting Recording
• Meeting Slides

Structuring Repository for Open-source​

• Added basic instructions for Code of Conduct, Contribution guidelines, and coding standards.
• Restructured code to merge leios and leios-sim packages.

Designing Test Environment​

• Sketched ideas for exploring Leios behavior with respect to networking.

Leios Meeting​

• Discussed network simulation and ΔQ model of the pipeline.
• Gathered numbers on network latency and throughput.

Visualizing Node Behavior​

• Added a simple web server to render the throughput of IBs and EBs in the Leios simulation.
• Utilized the Tracer interface for JSON-formatted traces and live graph plotting.

Discussion with Nicolas​

• Explored Leios protocol with a focus on IB to EB construction.
• Modeled a single node with input and output queues for IBs and EBs.

Meeting with Research​

• Discussed ongoing research work and networking challenges.
• Aimed to define a ΔQ model for Leios and explore protocol structure.

Public Meeting on October 31, 2024​

• Recording
• Slides

Designing Test Environment​

• Sketched ideas for exploring Leios behavior with respect to networking.
• Developed a high-level design for network simulation.

Leios Meeting​

• Discussed network simulation and ΔQ model of the pipeline.
• Gathered numbers on network latency and throughput.

Structuring Repository for Open-source​

• Added basic instructions for Code of Conduct, Contribution guidelines, and coding standards.
• Restructured code to merge leios and leios-sim packages.

On the 14th of June, 2024, along with Pi Lanningham, I was honoured to be invited to the very first Round-Table organised by the Cardano Community group. The discussion was facilitated by Denicio Bernier and Rick McCracken, and was meant to provide a broad overview of Leios, the protocol and the R&D project we have just started working on.

We first went through Pi's presentation, recalling how Ouroboros Praos works in broad terms and the problem Leios provides a solution for, namely the low throughput of the network. This low throughput is a necessity imposed by diffusion delays across a world spanning network, and constraints of the stake-based sortition algorithm that's used to select block-forging leader. By deconnecting the data diffusion and validation process from the blockchain extension process, trading throughput for latency, Leios aims at dramatically increase the number of transactions that can be processed by Cardano.

We then took questions from the audience, detailing some aspects of the protocol, while answering expectations from the community like the expected throughput and availability of benchmarks, the anticipated timeline for Leios, its possible usage for transient data diffusion, and its relationship with other expected Ouroboros extensions like Peras.

• Original blog post on Cardano.org
• X/Twitter thread
• Discussion recording on YouTube.