7 posts tagged with "formal-methods"

This week, the Leios team met for an in-person workshop in Edinburgh and continued their efforts in refining the protocol and its simulation capabilities. The team made significant progress in addressing various topics.

On day one, the team discussed topics such as ledger design and trade-offs, as well as two different ways to link the formal specification to the simulations. They explored various ledger design options, including labeled UTXOs and accounts approaches, with detailed consideration of fees, collateral, and conflict prevention mechanisms. The team also discussed conformance testing approaches, including QuickCheck dynamic and trace verification methods.

On day two, the team made significant progress towards estimating the cost of running a Leios node, considering different cost items such as network egress, CPU, and storage. They analyzed resource usage across different TPS levels, from 10 TPS to 1K TPS, and discovered that while there’s significant overhead at low throughput, the protocol becomes more efficient at higher TPS levels. The team hasn’t been able to finish all the cost items yet. The last two, IOPS and memory cost, will be added during this month.

On the last and third day, the team consolidated their options for how optimistic validation of input blocks can be accomplished. They defined three candidates, with one being favored. The main goal was to support the chaining of transactions with Leios, which requires defining a 'point in time' or stage of the protocol at which a subsequent or chained transaction can be built on top of an already submitted transaction. This can be achieved by having the node optimistically compute prospective ledger states using its local knowledge of input blocks referenced in certified endorser blocks or possibly ranking blocks.

• Day 1

• Day 2

• Day 3.

Simulation progress​

• Haskell simulation
  • Added support for dishonest nodes that diffuse an unbounded amount of old IBs, enabling further analysis of freshest-first and oldest-first vote delivery scenarios
  • Identified and fixed a bug in configuration generation for simulation runs, which was causing inconsistencies in vote delivery between default and uniform/extended voting schemes
  • Added an adversarial field to the network topology schema, allowing for the simulation of unbounded IB diffusion by dishonest nodes.

Ongoing investigations​

• Investigating the effects of unbounded IB diffusion on IB delivery reliability and protocol performance under such conditions
• Working on quantifying settlement times and their impact on protocol performance
• Exploring integration possibilities with Ouroboros Peras, mainly focusing on potentially reusing their voting mechanism to reduce resource consumption.

Additional resources​

• GitHub discussions – EB ledger states and 'history rewriting' effects
• The First Full Leios simulation analysis – detailed analysis of the latest simulation results.

This week, the Leios team made significant progress in protocol development, focusing on improving simulation capabilities and analyzing protocol behavior under various network conditions. A comparison of Haskell and Rust simulations across 18 scenarios demonstrated that the Leios protocol scales effectively to mainnet-sized networks. However, congestion occurs when the input block rate reaches 30 IB/s.

Simulation comparison​

• Compared 18 scenarios between Haskell and Rust simulations at tag leios-2025w12
• Recent fixes and adjustments enabled meaningful comparison between simulations
• Identified differences when comparing the Haskell and Rust results, which are under active investigation.

Analysis of simulations​

• Completed the first simulation of Short Leios, evaluating IB production rate, IB size, and network topology
• Demonstrated that the Leios protocol scales effectively to mainnet-sized networks
• Identified congestion occurring when the input block rate exceeds 30 IB/s
• Suggested that allowing IBs larger than current Praos RBs may have advantages in TCP efficiency, network usage, and adapting to fluctuating transaction loads.

Peak CPU	Mean CPU

Haskell simulation​

• Implemented expiration of blocks:
  • Blocks are removed from the relay buffer once diffusion stops and cleared from other states as specified
• Developed an initial Full Leios implementation:
  • Currently in early testing
  • Added the praos-chain-quality configuration parameter for the \eta parameter from the specification.

Rust simulation​

• Developed an initial Full Leios implementation using estimated values for some parameters.

Formal methods​

• Short Leios trace verification: modeling local state evolution of a node
• Developed an initial trace verifier for Short Leios simulation traces in leios-trace-verifier.

This week, the Leios team made significant progress in simulation capabilities, with a successful comparison of Rust and Haskell simulations across 90 scenarios. A mainnet-scale analysis of Leios on a realistic 3,000-node network revealed unexpected performance benefits from network topology. Insights from sharding performance analysis provided important optimization strategies. Finally, the team refined both simulation implementations for greater realism and comparability, while the formal methods team developed initial trace verification tools for Short Leios.

Simulation comparison​

• Compared 90 scenarios between Rust and Haskell simulations at tag leios-2025w11
• Recent fixes and adjustments enabled meaningful comparison between simulations
• Identified issues requiring further investigation.

Analysis of mainnet-scale simulation​

• Completed the first analysis of Leios on a mainnet-scale network simulation using the Rust simulator
• Discovered that a 3,000-node mainnet-scale network transports IBs faster than an artificial 100-node network
• Identified 'shortcut' edges in larger networks as a likely factor in the improved transport speed.

Performance analysis of sharding​

• Created computational models to analyze the relationship between the fraction of shards without an IB and the expected number of extra IBs
• Evaluated performance characteristics of the simplest sharding scheme.

Haskell simulation​

• Fixed a bug in the relay protocol that prevented full diffusion of votes
• Adjusted the priority of certified EBs for inclusion in RBs
• Added support for an output log format that shares a common subset with the Rust simulator
• Analyzed TCP realism in comparison to idealized diffusion:
  • Discovered that higher IB rates and sizes improve diffusion times
  • Identified ledger state access as a significant source of latency.

Rust simulation​

• Expanded logs to include total IB size and parent ID of RBs
• Implemented the same EB selection strategy as in the Haskell simulation
• Added validation of IB headers before propagation to neighbors
• Investigating lower congestion in the Rust simulation compared to Haskell.

Formal methods​

• Developed the initial trace verifier for Short Leios simulation traces in leios-trace-verifier.

Research​

• Progressing on ledger design by exploring options and trade-offs
• Analyzing how concurrent input blocks in Leios create unique ledger-level challenges not present in Praos
• Evaluating approaches that balance multiple properties, including:
  • Conflict avoidance in the blockchain
  • Guaranteed fee payment for block producers
  • Transaction eligibility and inclusion speed
  • User experience regarding fee payment
• Investigating sharding-based solutions with various optimization strategies
• Planning to share more detailed findings at Leios public meeting by the end of March
• Targeting a comprehensive recommendation for implementors by the end of April.

From Short Leios to Full Leios​

• Planning the simulation roadmap for transitioning from Short Leios (currently implemented) to Full Leios
• Developing implementation guidelines for simulators to incorporate the pipeline referencing scheme specified in the papers
• Identifying key components needed to simulate the complete ledger inclusion guarantees of Full Leios.

This week in Leios development, the team focused on simulation analysis, formal methods, and documentation updates. Key accomplishments include in-depth analysis of simulations at tag leios-2025w10, advancements in formal methods through a working trace verifier, and the development of technical reports.

Cross-simulation analysis​

• Completed a comprehensive analysis of simulations at tag leios-2025w10:
  • Analyzed Haskell simulation performance with and without CPU usage considerations
  • Varying key protocol parameters:
    • IB production rate
    • IB size
    • Length of Leios stages
  • Identified the following aspects of Leios:
    • Delay between IB generation and receipt at nodes
    • Peak and mean CPU usage over time
    • Breakdown of CPU usage by task type
    • Sizes of IBs, EBs, and RBs
    • Duplicate IB references in EBs
    • Reference to EBs from RBs
    • Resource utilization in network traffic.

Protocol and formal methods​

• Began developing a trace verifier in Agda:
  • Implemented event trace parsing using the Haskell module leios-trace-hs.

Documentation and research​

• Completed the full draft of the Leios technical report #1
• Created a skeletal draft of the Leios CIP
• Aligned with the latest CIP template
• Developed a detailed simulation analysis for the 100-node Leios network.

Programming and testing​

• Resolved several simulation issues:
  • #235: RB size does not reflect the presence of a certificate
  • #234: Fast transmission of large blocks at moderate IB rate
  • #232: Monotonicity of EB inclusion in RBs
  • #230: EBs are not large enough to include their IBs
  • #229: Rust simulations panics from overflow
• Enabled the visualization of network traffic and logging messages for multiple predefined 'scenarios' instead of a single hard-coded trace
• Updated the visualization to display resource utilization in network traffic.

Rust simulation visualization​

• Improved visualization capabilities:
  • Added support for multiple predefined 'scenarios' instead of single hard-coded trace
  • Moved the visualization logic to the client-side web worker for better performance
  • Added the visualization of per-node network traffic breakdown by message type
• Fixed critical simulation bugs:
  • Resolved issue #229 causing time travel and crashes in high-traffic high-latency scenarios.

This week in Leios development, the team focused on simulation analysis and formal methods. Key accomplishments include detailed analyses of both Haskell and Rust simulations, initial work on a protocol dashboard, and advancements in formal methods through trace verification in Agda.

Cross-simulation analysis​

• Completed a comprehensive analysis of simulations at tag leios-2025w09:
  • Refactored the ELT workflow to improve simulation data processing
  • Modified the Rust simulator to generate fixed-size input blocks (IBs) for comparison with Haskell
  • Partially resolved discrepancies in congestion metrics between simulators
  • Developed detailed analyses of:
    • IB generation to receipt elapsed time
    • Time-in-flight over node-to-node links
  • Identified the dual role of network bandwidth and CPU bottlenecks in high throughput congestion.

Protocol dashboard initiative​

• Initiated the design of an interactive protocol dashboard with planned features:
  • Protocol parameter configuration
  • Stake distribution settings
  • Performance visualization:
    • Block arrival efficiency
    • Transaction duplication
    • Leios operation rewards
    • Resource utilization
  • Security metrics visualization:
    • Quorum failure analysis
    • Certificate forgery detection
    • Adversarial block tracking.

Rust simulation​

• Enhanced parallel message handling capabilities:
  • Implemented parallel mini-protocol message transmission
  • Added even bandwidth distribution between mini-protocols
  • Introduced the simulate-transactions configuration option
  • Updated simulation output for better Haskell compatibility
  • Improved block visualization for scenarios with high IB counts.

Formal methods​

• Commenced trace verifier development in Agda:
  • Added decidability to Short Leios protocol relational specification
  • Implemented a proof-by-computation approach for execution traces
  • Applied the successful methodology from Streamlet formalization.

This week, the Leios team made significant progress across multiple areas. Major developments included detailed DeltaQ analysis of network topologies, extensive BLS cryptography benchmarking, and improvements to both simulations. The team also explored succinct schemes for BLS key registration and conducted a detailed certificate performance analysis. Both Haskell and Rust simulations received substantial updates to improve visualization and support more realistic testing conditions.

DeltaQ analysis​

• Enhanced the topology-checker with ΔQSD analysis capabilities:
  • Extracts inter-node latencies from given topologies
  • Classifies latencies into near/far components
  • Builds parameterized ΔQ models
  • Outputs fitted models in delta_q web app syntax
• Key findings from topology analysis:
  • Clear distinction between near/far components in examined topologies
  • Unexpectedly high hop counts in latency-weighted Dijkstra paths:
    • Min 4-5, max 8 for topology - 100
    • Min 8, max 20 for 'realistic' topology
  • Model fitting achieved rough shape matching but showed significant deviations at low latencies
  • Resource usage tracking goals remain unmet due to complexity in understanding load multiplication factors.

BLS cryptography​

• Completed comprehensive benchmarking of certificate operations:
  • Detailed performance analysis across committee sizes (500-1000 seats)
  • Certificate generation: 63.4ms - 92.5ms
  • Certificate verification: 104.8ms - 144.9ms
  • Certificate weighing: ~12ms consistently
• Explored succinct schemes for key registration:
  • Proposed 90-day key evolution with 124-byte KZG commitments
  • Analyzed message sizes for key opening (316 bytes per pool)
  • Investigated SNARK-based alternatives for proof of possession
• Added BLS crypto to the CI pipeline with automated testing
• Documented parallelization strategies for certificate operations.

Formal methods​

• Added a conformance testing client for the executable Short Leios specification
• Successfully merged the executable specification for Simplified Leios into main.

Haskell simulation​

• Updated configuration defaults for block sizes and timings
• Added support for idealized simulation conditions:
  • Single-peer block body requests
  • TCP congestion window modeling
  • Mini-protocol multiplexing
  • Unlimited bandwidth links support
• Enhanced simulation output and analysis:
  • Added raw field for accumulated data
  • Implemented block diffusion CDF extraction
  • Created multi-CDF plotting capabilities.

Rust simulation​

• Enhanced visualization capabilities:
  • Added block size breakdown display
  • Implemented total bytes sent/received tracking
  • Added total TX count and CPU time metrics
• Improved event handling:
  • Updated to standard timestamp format (seconds)
  • Enhanced CPU task event structure
  • Added CBOR output support
• Added support for multiple strategies:
  • Implemented ib-diffusion-strategy (freshest-first, oldest-first, peer-order)
  • Added relay-strategy affecting TXs, IBs, EBs, votes, and RBs
  • Enabled unlimited EB and vote bundle downloads from peers.

The Leios team continued refining Haskell and Rust simulations, standardizing inputs, outputs, and event logging for better comparability. The team defined standard formats for configuration parameters and network topology for running the Leios protocol. They also worked on logging identical simulation events to compare and feed them into the DeltaQ model and, consequently, the executable specification, ensuring alignment with formal methods.

Haskell simulation updates​

• The short-leios simulation now outputs diffusion latency data
• Added support for different input block (IB) diffusion strategies:
  • freshest-first: higher slot numbers requested first
  • peer-order: requested in order of peer announcement
• Added support for Vote (Send) and Vote (Recv) stages.

Rust simulation progress​

• Added an 'organic' topology generator that better matches mainnet topology
• The generator creates clusters of colocated stake pools and relays
• The simulation uses stake to determine relay connectivity
• Topology insights gathered from stake pool owners:
  • Most pools have multiple relays (2,312 relays across 1,278 pools)
  • Pool operators often run multiple colocated pools sharing relays
  • Relays typically maintain ~25 active outgoing connections
  • Incoming connections scale with stake weight (10-400+ connections).

DeltaQ update​

• Wrote a comprehensive 2025-01 report covering work since September 2024.

Formal methods​

• Finalizing executable specifications for simplified and short Leios
• Extracted short Leios specification to Haskell for conformance testing.