6 posts tagged with "analysis"

This week, the team continued refining the protocol and its simulation capabilities, making significant progress in addressing various topics.

Simulation improvements​

Haskell simulation​

• Started specifying a new relay protocol for IB header diffusion without the body
• Improved the shared log format by removing redundancies and harmonizing naming
• Added support for additional events required by conformance testing, including SlotEvent and NoBlockEvent
  • These events can be enabled using the --conformance-events flag with --shared-log-format.

Rust simulation​

• Updated traces to match the new standardized trace format
• Fixed a critical bug in CPU scheduling where nodes were using more cores than allocated.

Analysis of workflow optimization​

The team significantly improved the workflow for analyzing both Haskell and Rust simulations:

• Replaced MongoDB with more efficient jq queries using map-reduce operations
• Created reusable library functions for plotting with R
• Revised and streamlined scripts for creating, executing, and analyzing simulations
• Made the Jupyter notebook for analyses more generic and reusable
• Successfully tested the new workflow on tag leios-2025w15.

These improvements will enable faster setup and execution of future simulation experiments, with quicker turnaround times for analysis. During this optimization work, several discrepancies between the Haskell and Rust simulations were identified and documented as GitHub issues for future investigation.

Edinburgh workshop recaps​

The Edinburgh workshop documentation has been made available, covering key discussions and decisions:

Day 1 highlights​

• Explored ledger design options comparing labeled UTXOs (explicit shards) vs accounts (implicit shards) approaches
• Discussed conformance testing strategies including QuickCheck dynamic and trace verification approaches
• Analyzed critical edge cases for user onboarding and system properties.

Day 2 highlights​

• Conducted a detailed analysis of Leios node costs across different TPS levels
• Key findings on resource usage:
  • At 10 TPS: 1.8x increase in egress and 6x increase in compute compared to Praos
  • At 1K TPS: significant scaling improvements with better resource efficiency
• Provided recommendations for potential integration with Peras, particularly to optimize the voting mechanism
• Discussed performance characteristics at both high and low throughput levels.

Day 3 highlights​

• Held an in-depth discussion on optimistic ledger state references, exploring three main approaches:
  1. RB reference: highest security but highest latency
  2. EB reference: balanced approach with medium security and latency
  3. EB-DAG: advanced approach using directed acyclic graph structure
• Key advantages of the EB-DAG approach:
  • Achieves low latency while maintaining security
  • Provides strong inclusion guarantees for EBs
  • Enables efficient state management and reconstruction
  • Creates a complete, verifiable chain history
• Discussed implementation considerations for state management and block ordering under the EB-DAG model.

For more information, please see the full workshop recaps in the Leios documentation.

This week, the Leios team made significant progress in simulation capabilities, with a successful comparison of Rust and Haskell simulations across 90 scenarios. A mainnet-scale analysis of Leios on a realistic 3,000-node network revealed unexpected performance benefits from network topology. Insights from sharding performance analysis provided important optimization strategies. Finally, the team refined both simulation implementations for greater realism and comparability, while the formal methods team developed initial trace verification tools for Short Leios.

Simulation comparison​

• Compared 90 scenarios between Rust and Haskell simulations at tag leios-2025w11
• Recent fixes and adjustments enabled meaningful comparison between simulations
• Identified issues requiring further investigation.

Analysis of mainnet-scale simulation​

• Completed the first analysis of Leios on a mainnet-scale network simulation using the Rust simulator
• Discovered that a 3,000-node mainnet-scale network transports IBs faster than an artificial 100-node network
• Identified 'shortcut' edges in larger networks as a likely factor in the improved transport speed.

Performance analysis of sharding​

• Created computational models to analyze the relationship between the fraction of shards without an IB and the expected number of extra IBs
• Evaluated performance characteristics of the simplest sharding scheme.

Haskell simulation​

• Fixed a bug in the relay protocol that prevented full diffusion of votes
• Adjusted the priority of certified EBs for inclusion in RBs
• Added support for an output log format that shares a common subset with the Rust simulator
• Analyzed TCP realism in comparison to idealized diffusion:
  • Discovered that higher IB rates and sizes improve diffusion times
  • Identified ledger state access as a significant source of latency.

Rust simulation​

• Expanded logs to include total IB size and parent ID of RBs
• Implemented the same EB selection strategy as in the Haskell simulation
• Added validation of IB headers before propagation to neighbors
• Investigating lower congestion in the Rust simulation compared to Haskell.

Formal methods​

• Developed the initial trace verifier for Short Leios simulation traces in leios-trace-verifier.

Research​

• Progressing on ledger design by exploring options and trade-offs
• Analyzing how concurrent input blocks in Leios create unique ledger-level challenges not present in Praos
• Evaluating approaches that balance multiple properties, including:
  • Conflict avoidance in the blockchain
  • Guaranteed fee payment for block producers
  • Transaction eligibility and inclusion speed
  • User experience regarding fee payment
• Investigating sharding-based solutions with various optimization strategies
• Planning to share more detailed findings at Leios public meeting by the end of March
• Targeting a comprehensive recommendation for implementors by the end of April.

From Short Leios to Full Leios​

• Planning the simulation roadmap for transitioning from Short Leios (currently implemented) to Full Leios
• Developing implementation guidelines for simulators to incorporate the pipeline referencing scheme specified in the papers
• Identifying key components needed to simulate the complete ledger inclusion guarantees of Full Leios.

This week in Leios development, the team focused on simulation analysis, formal methods, and documentation updates. Key accomplishments include in-depth analysis of simulations at tag leios-2025w10, advancements in formal methods through a working trace verifier, and the development of technical reports.

Cross-simulation analysis​

• Completed a comprehensive analysis of simulations at tag leios-2025w10:
  • Analyzed Haskell simulation performance with and without CPU usage considerations
  • Varying key protocol parameters:
    • IB production rate
    • IB size
    • Length of Leios stages
  • Identified the following aspects of Leios:
    • Delay between IB generation and receipt at nodes
    • Peak and mean CPU usage over time
    • Breakdown of CPU usage by task type
    • Sizes of IBs, EBs, and RBs
    • Duplicate IB references in EBs
    • Reference to EBs from RBs
    • Resource utilization in network traffic.

Protocol and formal methods​

• Began developing a trace verifier in Agda:
  • Implemented event trace parsing using the Haskell module leios-trace-hs.

Documentation and research​

• Completed the full draft of the Leios technical report #1
• Created a skeletal draft of the Leios CIP
• Aligned with the latest CIP template
• Developed a detailed simulation analysis for the 100-node Leios network.

Programming and testing​

• Resolved several simulation issues:
  • #235: RB size does not reflect the presence of a certificate
  • #234: Fast transmission of large blocks at moderate IB rate
  • #232: Monotonicity of EB inclusion in RBs
  • #230: EBs are not large enough to include their IBs
  • #229: Rust simulations panics from overflow
• Enabled the visualization of network traffic and logging messages for multiple predefined 'scenarios' instead of a single hard-coded trace
• Updated the visualization to display resource utilization in network traffic.

Rust simulation visualization​

• Improved visualization capabilities:
  • Added support for multiple predefined 'scenarios' instead of single hard-coded trace
  • Moved the visualization logic to the client-side web worker for better performance
  • Added the visualization of per-node network traffic breakdown by message type
• Fixed critical simulation bugs:
  • Resolved issue #229 causing time travel and crashes in high-traffic high-latency scenarios.

This week in Leios development, the team focused on simulation analysis and formal methods. Key accomplishments include detailed analyses of both Haskell and Rust simulations, initial work on a protocol dashboard, and advancements in formal methods through trace verification in Agda.

Cross-simulation analysis​

• Completed a comprehensive analysis of simulations at tag leios-2025w09:
  • Refactored the ELT workflow to improve simulation data processing
  • Modified the Rust simulator to generate fixed-size input blocks (IBs) for comparison with Haskell
  • Partially resolved discrepancies in congestion metrics between simulators
  • Developed detailed analyses of:
    • IB generation to receipt elapsed time
    • Time-in-flight over node-to-node links
  • Identified the dual role of network bandwidth and CPU bottlenecks in high throughput congestion.

Protocol dashboard initiative​

• Initiated the design of an interactive protocol dashboard with planned features:
  • Protocol parameter configuration
  • Stake distribution settings
  • Performance visualization:
    • Block arrival efficiency
    • Transaction duplication
    • Leios operation rewards
    • Resource utilization
  • Security metrics visualization:
    • Quorum failure analysis
    • Certificate forgery detection
    • Adversarial block tracking.

Rust simulation​

• Enhanced parallel message handling capabilities:
  • Implemented parallel mini-protocol message transmission
  • Added even bandwidth distribution between mini-protocols
  • Introduced the simulate-transactions configuration option
  • Updated simulation output for better Haskell compatibility
  • Improved block visualization for scenarios with high IB counts.

Formal methods​

• Commenced trace verifier development in Agda:
  • Added decidability to Short Leios protocol relational specification
  • Implemented a proof-by-computation approach for execution traces
  • Applied the successful methodology from Streamlet formalization.

This week in Leios development, CPS-0018 for transaction throughput was approved, along with improved Docker support for simulations and analysis of cross-simulation results. The team also examined input block (IB) production rates and their impact on network performance.

Protocol development​

• CPS-0018 'Greater transaction throughput' officially approved:
  • Merged into Cardano Foundation's CIP/CPS repository
  • Documents urgency of higher transaction throughput
  • Defines goals for the Leios initiative
  • Identifies key open questions and use cases.

Cross-simulation analysis​

• Conducted a comprehensive analysis of IB production rates ranging from 1 IB/s to 100 IB/s:
  • Developed an ELT workflow for data processing via MongoDB
  • Created an R Jupyter notebook for analysis and visualization
  • Identified and addressed three significant bugs (#207, #208, #209)
• Key findings from the Haskell simulation:
  • Network congestion emerges at high IB production rates
  • Both average propagation time and slow propagation tail increase
  • A critical threshold of ~40 IBs/s was identified, beyond which network congestion severely impacts block reception
• Comparison of PeerNet and Haskell simulations:
  • Both exhibit qualitatively similar block propagation distributions
  • Both demonstrate protocol breakdown under high block production rates
  • Differences in resolution and configuration prevent exact comparison.

Infrastructure improvements​

• Added comprehensive Docker support for both simulations:
  • Optimized multi-stage Docker files for Haskell and Rust
  • Simplified deployment process
  • Enabled easy configuration via volume mounts and parameters
  • Documented usage in README.md.

Rust simulation​

• Enhanced Rust simulation capabilities:
  • Implemented bandwidth usage tracking
  • Added configurable bandwidth limits per connection
  • Fixed issues identified in cross-simulation comparisons
  • Started updating visualizations for improved clarity.

Haskell simulation​

• Enhanced IB sortition handling for IB/slot < 1
• Began integrating block expiration and diffusion-halt proposal
• Implemented ideal timing calculations for diffusion:
  • Added uniform block behavior configuration
  • Identified relay mini-protocol complexities:
    • Variable latency (3-4) for block transfer
    • Latency depends on traffic conditions and request handling.

Formal methods​

• Moved formal specification to a dedicated repository
• Established a conformance testing framework:
  • Enabled testing between Short Leios implementations
  • Documented the test suite execution process
• Initiated a survey of network models across IO consensus projects.

This week, the Leios team made significant progress across multiple areas. Major developments included detailed DeltaQ analysis of network topologies, extensive BLS cryptography benchmarking, and improvements to both simulations. The team also explored succinct schemes for BLS key registration and conducted a detailed certificate performance analysis. Both Haskell and Rust simulations received substantial updates to improve visualization and support more realistic testing conditions.

DeltaQ analysis​

• Enhanced the topology-checker with ΔQSD analysis capabilities:
  • Extracts inter-node latencies from given topologies
  • Classifies latencies into near/far components
  • Builds parameterized ΔQ models
  • Outputs fitted models in delta_q web app syntax
• Key findings from topology analysis:
  • Clear distinction between near/far components in examined topologies
  • Unexpectedly high hop counts in latency-weighted Dijkstra paths:
    • Min 4-5, max 8 for topology - 100
    • Min 8, max 20 for 'realistic' topology
  • Model fitting achieved rough shape matching but showed significant deviations at low latencies
  • Resource usage tracking goals remain unmet due to complexity in understanding load multiplication factors.

BLS cryptography​

• Completed comprehensive benchmarking of certificate operations:
  • Detailed performance analysis across committee sizes (500-1000 seats)
  • Certificate generation: 63.4ms - 92.5ms
  • Certificate verification: 104.8ms - 144.9ms
  • Certificate weighing: ~12ms consistently
• Explored succinct schemes for key registration:
  • Proposed 90-day key evolution with 124-byte KZG commitments
  • Analyzed message sizes for key opening (316 bytes per pool)
  • Investigated SNARK-based alternatives for proof of possession
• Added BLS crypto to the CI pipeline with automated testing
• Documented parallelization strategies for certificate operations.

Formal methods​

• Added a conformance testing client for the executable Short Leios specification
• Successfully merged the executable specification for Simplified Leios into main.

Haskell simulation​

• Updated configuration defaults for block sizes and timings
• Added support for idealized simulation conditions:
  • Single-peer block body requests
  • TCP congestion window modeling
  • Mini-protocol multiplexing
  • Unlimited bandwidth links support
• Enhanced simulation output and analysis:
  • Added raw field for accumulated data
  • Implemented block diffusion CDF extraction
  • Created multi-CDF plotting capabilities.

Rust simulation​

• Enhanced visualization capabilities:
  • Added block size breakdown display
  • Implemented total bytes sent/received tracking
  • Added total TX count and CPU time metrics
• Improved event handling:
  • Updated to standard timestamp format (seconds)
  • Enhanced CPU task event structure
  • Added CBOR output support
• Added support for multiple strategies:
  • Implemented ib-diffusion-strategy (freshest-first, oldest-first, peer-order)
  • Added relay-strategy affecting TXs, IBs, EBs, votes, and RBs
  • Enabled unlimited EB and vote bundle downloads from peers.