3 posts tagged with "simulation-comparison"

This week, the Leios team focused on comparative analysis between simulation implementations and conducted experiments examining the impact of protocol parameters and network topology on Linear Leios performance. The team made progress in cross-validation between Haskell and Rust simulations while investigating protocol behavior under different network conditions.

Cross-simulation analysis​

Haskell vs Rust Linear Leios comparison​

The team completed a comprehensive comparison between the early draft Haskell simulator and the more mature Rust simulator for Linear Leios. Key findings from the analysis documented in analysis/sims/2025w31b/analysis.ipynb revealed several discrepancies requiring investigation:

• CPU usage patterns differed between implementations
• Network usage showed variations across simulators
• Vote diffusion behavior exhibited inconsistencies
• Active investigation underway to resolve implementation differences.

Protocol parameter experiments​

Stage length analysis for "No IBs" Leios​

The team conducted experiments varying the stage-length protocol parameter from 5 to 12 slots per stage in "No IBs" Leios. Results documented in analysis/sims/2025w31/analysis.ipynb showed:

• Settlement time remains relatively stable across the tested parameter range
• Stage length has minimal impact on transaction processing times within 5-12 slot range
• Larger stage lengths result in less frequent voting periods
• Protocol performance appears robust to moderate stage length variations.

Network topology validation​

Mini-mainnet vs pseudo-mainnet comparison​

The team completed a comparative analysis of Linear Leios performance across different network topologies using the Rust simulator. Key findings from analysis/sims/2025w30b/analysis.ipynb include:

• The 750-node mini-mainnet network proved slightly more stressful to the protocol than the 10,000-node pseudo-mainnet
• No substantial differences in simulation results between the two network configurations
• Both networks produce equivalent findings and conclusions for protocol analysis
• Validation confirms the mini-mainnet topology as a suitable proxy for larger-scale analysis.

Rust simulation enhancements​

Linear Leios attack modeling​

• Added equivocation voting delay functionality to Linear Leios implementation
• Implemented attack scenario modeling where endorser block (EB) diffusion is deliberately delayed until the last moment
• Enhanced simulation capabilities for security analysis and adversarial behavior testing.

Next steps​

• Continue investigation of discrepancies between Haskell and Rust Linear Leios implementations
• Expand parameter sensitivity analysis for additional protocol variants
• Refine attack modeling capabilities for comprehensive security assessment
• Apply lessons from network topology comparison to future experimental design.

This week, the Leios team made significant progress in protocol development, focusing on improving simulation capabilities and analyzing protocol behavior under various network conditions. A comparison of Haskell and Rust simulations across 18 scenarios demonstrated that the Leios protocol scales effectively to mainnet-sized networks. However, congestion occurs when the input block rate reaches 30 IB/s.

Simulation comparison​

• Compared 18 scenarios between Haskell and Rust simulations at tag leios-2025w12
• Recent fixes and adjustments enabled meaningful comparison between simulations
• Identified differences when comparing the Haskell and Rust results, which are under active investigation.

Analysis of simulations​

• Completed the first simulation of Short Leios, evaluating IB production rate, IB size, and network topology
• Demonstrated that the Leios protocol scales effectively to mainnet-sized networks
• Identified congestion occurring when the input block rate exceeds 30 IB/s
• Suggested that allowing IBs larger than current Praos RBs may have advantages in TCP efficiency, network usage, and adapting to fluctuating transaction loads.

Peak CPU	Mean CPU

Haskell simulation​

• Implemented expiration of blocks:
  • Blocks are removed from the relay buffer once diffusion stops and cleared from other states as specified
• Developed an initial Full Leios implementation:
  • Currently in early testing
  • Added the praos-chain-quality configuration parameter for the \eta parameter from the specification.

Rust simulation​

• Developed an initial Full Leios implementation using estimated values for some parameters.

Formal methods​

• Short Leios trace verification: modeling local state evolution of a node
• Developed an initial trace verifier for Short Leios simulation traces in leios-trace-verifier.

This week, the Leios team made significant progress in simulation capabilities, with a successful comparison of Rust and Haskell simulations across 90 scenarios. A mainnet-scale analysis of Leios on a realistic 3,000-node network revealed unexpected performance benefits from network topology. Insights from sharding performance analysis provided important optimization strategies. Finally, the team refined both simulation implementations for greater realism and comparability, while the formal methods team developed initial trace verification tools for Short Leios.

Simulation comparison​

• Compared 90 scenarios between Rust and Haskell simulations at tag leios-2025w11
• Recent fixes and adjustments enabled meaningful comparison between simulations
• Identified issues requiring further investigation.

Analysis of mainnet-scale simulation​

• Completed the first analysis of Leios on a mainnet-scale network simulation using the Rust simulator
• Discovered that a 3,000-node mainnet-scale network transports IBs faster than an artificial 100-node network
• Identified 'shortcut' edges in larger networks as a likely factor in the improved transport speed.

Performance analysis of sharding​

• Created computational models to analyze the relationship between the fraction of shards without an IB and the expected number of extra IBs
• Evaluated performance characteristics of the simplest sharding scheme.

Haskell simulation​

• Fixed a bug in the relay protocol that prevented full diffusion of votes
• Adjusted the priority of certified EBs for inclusion in RBs
• Added support for an output log format that shares a common subset with the Rust simulator
• Analyzed TCP realism in comparison to idealized diffusion:
  • Discovered that higher IB rates and sizes improve diffusion times
  • Identified ledger state access as a significant source of latency.

Rust simulation​

• Expanded logs to include total IB size and parent ID of RBs
• Implemented the same EB selection strategy as in the Haskell simulation
• Added validation of IB headers before propagation to neighbors
• Investigating lower congestion in the Rust simulation compared to Haskell.

Formal methods​

• Developed the initial trace verifier for Short Leios simulation traces in leios-trace-verifier.

Research​

• Progressing on ledger design by exploring options and trade-offs
• Analyzing how concurrent input blocks in Leios create unique ledger-level challenges not present in Praos
• Evaluating approaches that balance multiple properties, including:
  • Conflict avoidance in the blockchain
  • Guaranteed fee payment for block producers
  • Transaction eligibility and inclusion speed
  • User experience regarding fee payment
• Investigating sharding-based solutions with various optimization strategies
• Planning to share more detailed findings at Leios public meeting by the end of March
• Targeting a comprehensive recommendation for implementors by the end of April.

From Short Leios to Full Leios​

• Planning the simulation roadmap for transitioning from Short Leios (currently implemented) to Full Leios
• Developing implementation guidelines for simulators to incorporate the pipeline referencing scheme specified in the papers
• Identifying key components needed to simulate the complete ledger inclusion guarantees of Full Leios.