17 posts tagged with "haskell-simulation"

This week, the Leios team advanced the Cardano Improvement Proposal (CIP) documentation, conducted comprehensive attack analysis experiments, and continued cross-validation between simulation implementations. The team successfully demonstrated resilience characteristics of proposed Leios under adversarial conditions and refined protocol specification components for the formal CIP submission.

CIP development progress​

The team made substantial progress on the Ouroboros Leios CIP proposal. The draft specification section is now complete except for the network and incentives components. The motivation and abstract sections have been refined for clarity, enhancing the document's accessibility and technical precision. The first of four main rationale subsections has been fully drafted, providing evidence-based arguments for the necessity and viability of the proposed Leios protocol enhancement.

Attack resistance analysis​

Initial adversarial simulation experiments​

The team conducted the first simulation experiment for attacks, examining late Endorser Block (EB) and transaction diffusion scenarios where adversaries control timing of critical protocol messages. The experiment varied the diffusion parameter L_diff and EB propagation schemes to assess protocol robustness under different adversarial strategies.

Key findings from the initial attack analysis include successful demonstration that late-release attacks can impact proposed Leios throughput under specific conditions. The analysis revealed that transaction loss occurred in some scenarios due to memory pool rule formulations in the simulator, leading to important insights for protocol hardening and implementation requirements.

Cross-simulation validation​

Haskell versus Rust comparison​

The team completed another comprehensive comparison between Haskell and Rust simulation implementations at analysis/sims/202532b. Results indicate successful resolution of previous discrepancies in vote diffusion behavior between the two simulation environments. This validation ensures both implementations produce consistent results for protocol analysis and strengthens confidence in the simulation-based evidence supporting the CIP.

CIP figure preparation​

The experiment for draft figures for CIP progressed with updated simulation runs designed to generate publication-quality performance charts and protocol behavior visualizations for inclusion in the formal CIP documentation.

Next steps​

• Complete the remaining network and incentives sections of the CIP specification
• Conduct follow-up analysis on attack experiment findings to refine protocol parameters
• Continue refinement of memory pool rules based on adversarial scenario insights
• Finalize CIP figures and supporting analysis for submission preparation.

This week, the Leios team made significant progress on the Cardano Improvement Proposal (CIP) documentation, conducted extensive Plutus validation experiments, and resolved outstanding discrepancies between simulation implementations. The team successfully demonstrated Linear Leios performance under various Plutus workloads and completed comprehensive protocol parameter analysis for CIP inclusion.

CIP development progress​

The team completed substantial portions of the Leios CIP draft, bringing it closer to submission readiness. The specification section is now complete except for network and incentives components, with the motivation and abstract refined for clarity. The first of four main rationale subsections has been fully drafted, providing evidence-based arguments for Leios necessity and viability.

Plutus validation experiments​

The team conducted comprehensive experiments examining Linear Leios performance under varying Plutus computation loads using 6-vCPU nodes at 100 TPS. Key findings from the Plutus validation analysis include:

• Linear Leios successfully supported doubling the Plutus per-transaction budget
• Protocol breakdown occurred at sixfold increases in Plutus budgets due to validation bottlenecks
• Endorser blocks could handle approximately 5,000 execution steps of Plutus computation, representing 250 times the current Praos per-block budget
• This capacity could support either a handful of transactions with 20x greater Plutus budgets or increasing every Plutus transaction budget by 50%
• Late diffusion of Plutus-heavy transactions poses potential risks to EB adoption timing.

The analysis revealed significant variability in CPU time requirements for Plutus scripts relative to their execution steps, highlighting the need for careful resource planning in high-throughput scenarios.

Simulation improvements and comparisons​

Cross-simulation validation​

The team completed another comprehensive comparison between Haskell and Rust simulations at analysis/sims/202532b, successfully resolving previous discrepancies in vote diffusion behavior. This validation ensures both simulation implementations produce consistent results for protocol analysis.

Protocol parameter optimization​

The 2025w32 experiment established a comprehensive set of protocol parameters and throughput scenarios for inclusion in the CIP Evidence section:

• Protocol variant: Linear Leios with conservative resource allocation
• Resource requirements: 4 vCPU per node, 10 Mb/s bandwidth
• Stage configuration: 7 slots each for voting and diffusion stages
• Block limits: Maximum 12 MB transaction references per EB
• Transaction size: 1,500 bytes per transaction with normal Plutus frequency

Key performance findings demonstrate that modest computational resources adequately support throughput up to 0.3 TxMB/s, with 7-slot stages providing sufficient diffusion time while minimizing EB discard probability. The 12 MB EB limit allows occasional peak utilization to compensate for unlucky sortition periods.

Implementation enhancements​

Haskell simulation​

The team addressed head-of-line blocking issues in Linear Leios by implementing message slicing capabilities in the mini-protocol multiplexer, eliminating unexpected delays in vote diffusion. Additional work focused on developing new mini-protocols for enhanced Linear Leios simulation fidelity, with ongoing refinements to balance protocol granularity and sophistication.

Rust simulation​

Implementation of transaction withholding attack scenarios for Linear Leios, where EB producers delay transaction publication until EB release. The simulation also received updates to improve handling of late transactions, EBs, and RBs in Linear Leios scenarios.

Next steps​

• Complete remaining network and incentives sections of the CIP specification
• Finalize mini-protocol designs for enhanced simulation accuracy
• Continue investigation of mempool rule adequacy for high-throughput scenarios
• Expand Plutus validation analysis to cover additional execution budget scenarios.

This week, the Leios team focused on comparative analysis between simulation implementations and conducted experiments examining the impact of protocol parameters and network topology on Linear Leios performance. The team made progress in cross-validation between Haskell and Rust simulations while investigating protocol behavior under different network conditions.

Cross-simulation analysis​

Haskell vs Rust Linear Leios comparison​

The team completed a comprehensive comparison between the early draft Haskell simulator and the more mature Rust simulator for Linear Leios. Key findings from the analysis documented in analysis/sims/2025w31b/analysis.ipynb revealed several discrepancies requiring investigation:

• CPU usage patterns differed between implementations
• Network usage showed variations across simulators
• Vote diffusion behavior exhibited inconsistencies
• Active investigation underway to resolve implementation differences.

Protocol parameter experiments​

Stage length analysis for "No IBs" Leios​

The team conducted experiments varying the stage-length protocol parameter from 5 to 12 slots per stage in "No IBs" Leios. Results documented in analysis/sims/2025w31/analysis.ipynb showed:

• Settlement time remains relatively stable across the tested parameter range
• Stage length has minimal impact on transaction processing times within 5-12 slot range
• Larger stage lengths result in less frequent voting periods
• Protocol performance appears robust to moderate stage length variations.

Network topology validation​

Mini-mainnet vs pseudo-mainnet comparison​

The team completed a comparative analysis of Linear Leios performance across different network topologies using the Rust simulator. Key findings from analysis/sims/2025w30b/analysis.ipynb include:

• The 750-node mini-mainnet network proved slightly more stressful to the protocol than the 10,000-node pseudo-mainnet
• No substantial differences in simulation results between the two network configurations
• Both networks produce equivalent findings and conclusions for protocol analysis
• Validation confirms the mini-mainnet topology as a suitable proxy for larger-scale analysis.

Rust simulation enhancements​

Linear Leios attack modeling​

• Added equivocation voting delay functionality to Linear Leios implementation
• Implemented attack scenario modeling where endorser block (EB) diffusion is deliberately delayed until the last moment
• Enhanced simulation capabilities for security analysis and adversarial behavior testing.

Next steps​

• Continue investigation of discrepancies between Haskell and Rust Linear Leios implementations
• Expand parameter sensitivity analysis for additional protocol variants
• Refine attack modeling capabilities for comprehensive security assessment
• Apply lessons from network topology comparison to future experimental design.

This week, the team made significant progress on simulation improvements, trace verification, and a comprehensive analysis of Leios' transaction processing capacity.

Trace verification​

• Improved the trace verifier with better error handling and reporting
• Added support for starting verification from non-initial states
• Created manually curated test cases for the Leios trace verifier
  • Valid traces
  • Invalid traces
• Integrated the trace verifier into Nix infrastructure and CI builds
• Removed deterministic conformance testing in favor of a trace-based approach.

Simulation improvements​

Haskell simulation​

• Conducted an informal review assessing code quality, design, and implementation
• Analyzed the simulation organization and identified areas for future improvement
• Found that most prospective changes to the Leios protocol would only involve a small fraction of the codebase
• Determined that adding memory pool and transactions would take approximately 100-200 hours of labor.

The review of the Haskell simulator was documented in detail in PR#353, covering its statistics, organization, code quality, design, implementation, and documentation aspects.

Rust simulation​

• Added tx-start-time and tx-stop-time parameters to avoid effects of slow starts or sudden terminations on transaction analysis
• Created a new Leios variant full-without-ibs where endorser blocks directly reference transactions.

Documentation and analysis​

• Relocated the original Leios report to avoid confusion, while preserving valuable background information
• Added partially-drafted technical reports on Haskell simulations to Nix and CI builds:
  • Ouroboros Leios network specification
  • Ouroboros Leios simulation: building confidence in the performance results.

The team ran higher excess-capacity simulations to test hypotheses about transaction inclusion. The transaction lifecycle simulations raised the question of whether duplicated transactions in IBs were preventing other transactions from ever being included. The team ran simulations with IBs produced at three times the normal rate to test this, providing ample space for transaction duplication.

Detailed analysis showed that transaction loss persisted despite increased capacity, indicating that other factors are preventing transactions from reaching the ledger. The results are documented in:

• Analysis overview
• Results at 1x IB capacity
• Results at 3x IB capacity.

This week, the team made substantial progress in both the Haskell and Rust simulations, refined cost estimates, and carried out detailed analyses of the transaction lifecycle and Full Leios simulations.

Simulation improvements​

Haskell simulation​

• Completed the first draft of new mini protocols for Leios diffusion
  • Modeled protocols after block-fetch and node-to-node transaction submission from ouroboros-network
  • Included IB relay, EB relay, and vote relay for header diffusion and body announcements
  • Included IB fetch and EB fetch for body diffusion
  • Worked on the CatchUp protocol for older blocks
  • See simulation/docs/network-spec for full protocol details
• Renamed short-leios command to leios as it now covers the full variant as well
  • short-leios is kept as alias for compatibility.

Rust simulation​

• Fixed conformance issues with the shared trace format
• Fixed a bug in the voting logic that prevented EBs from receiving enough votes to be included on-chain
• Updated visualizations to use smaller trace files in preparation for hosting on the documentation site.

Revisions to the cost dashboard​

The cost dashboard was updated with lower and more realistic IO estimates.

Transaction lifecycle analysis​

The Jupyter notebook Analysis of transaction lifecycle estimates the delay introduced at each of the seven stages of Full Leios as a transaction progresses from the memory pool to being referenced by a Praos block.

Key findings from the analysis:

1. Reducing stage lengths below 10 slots offers little benefit
2. The number of shards should remain low enough to maintain a high IB rate per shard relative to the stage length
3. Low EB rates result in many orphaned IBs
4. With realistic parameters, the delay from transaction submission to its inclusion in an RB is approximately two minutes.

Potential next steps:

• Translate the model into Delta QSD to capture network effects
• Compare the model's output with results from the Rust simulator
• Extend the model to account for different memory pool and ledger variants under evaluation.

Simulation and analysis of Full Leios​

The team conducted comprehensive simulations using both Haskell and Rust simulators at tag leios-2025w16. The simulations covered 648 scenarios of Full and Short Leios with varied parameters:

• IB production rate
• IB size
• EB production rate
• Stage length
• CPU constraints.

Two new output files were generated:

1. A summary of network, disk, and CPU resource usage over the course of the simulation
2. The vertices and edges of the Leios graph, showing linkages between transactions, IBs, EBs, RBs, and votes (can be visualized as an interactive web page).

Key findings:

• The Rust and Haskell simulations show generally close agreement
• The Haskell simulation encounters network congestion at 16 IB/s, while the Rust simulation does not
• The Rust simulation consumes more CPU at high IB rates than the Haskell simulation
• In some cases, the Rust simulation does not produce enough votes to certify an EB.

Detailed results are available in the Jupyter notebook analysis/sims/2025w16/analysis.ipynb.

This week, the team continued refining the protocol and its simulation capabilities, making significant progress in addressing various topics.

Simulation improvements​

Haskell simulation​

• Started specifying a new relay protocol for IB header diffusion without the body
• Improved the shared log format by removing redundancies and harmonizing naming
• Added support for additional events required by conformance testing, including SlotEvent and NoBlockEvent
  • These events can be enabled using the --conformance-events flag with --shared-log-format.

Rust simulation​

• Updated traces to match the new standardized trace format
• Fixed a critical bug in CPU scheduling where nodes were using more cores than allocated.

Analysis of workflow optimization​

The team significantly improved the workflow for analyzing both Haskell and Rust simulations:

• Replaced MongoDB with more efficient jq queries using map-reduce operations
• Created reusable library functions for plotting with R
• Revised and streamlined scripts for creating, executing, and analyzing simulations
• Made the Jupyter notebook for analyses more generic and reusable
• Successfully tested the new workflow on tag leios-2025w15.

These improvements will enable faster setup and execution of future simulation experiments, with quicker turnaround times for analysis. During this optimization work, several discrepancies between the Haskell and Rust simulations were identified and documented as GitHub issues for future investigation.

Edinburgh workshop recaps​

The Edinburgh workshop documentation has been made available, covering key discussions and decisions:

Day 1 highlights​

• Explored ledger design options comparing labeled UTXOs (explicit shards) vs accounts (implicit shards) approaches
• Discussed conformance testing strategies including QuickCheck dynamic and trace verification approaches
• Analyzed critical edge cases for user onboarding and system properties.

Day 2 highlights​

• Conducted a detailed analysis of Leios node costs across different TPS levels
• Key findings on resource usage:
  • At 10 TPS: 1.8x increase in egress and 6x increase in compute compared to Praos
  • At 1K TPS: significant scaling improvements with better resource efficiency
• Provided recommendations for potential integration with Peras, particularly to optimize the voting mechanism
• Discussed performance characteristics at both high and low throughput levels.

Day 3 highlights​

• Held an in-depth discussion on optimistic ledger state references, exploring three main approaches:
  1. RB reference: highest security but highest latency
  2. EB reference: balanced approach with medium security and latency
  3. EB-DAG: advanced approach using directed acyclic graph structure
• Key advantages of the EB-DAG approach:
  • Achieves low latency while maintaining security
  • Provides strong inclusion guarantees for EBs
  • Enables efficient state management and reconstruction
  • Creates a complete, verifiable chain history
• Discussed implementation considerations for state management and block ordering under the EB-DAG model.

For more information, please see the full workshop recaps in the Leios documentation.

This week, the Leios team met for an in-person workshop in Edinburgh and continued their efforts in refining the protocol and its simulation capabilities. The team made significant progress in addressing various topics.

On day one, the team discussed topics such as ledger design and trade-offs, as well as two different ways to link the formal specification to the simulations. They explored various ledger design options, including labeled UTXOs and accounts approaches, with detailed consideration of fees, collateral, and conflict prevention mechanisms. The team also discussed conformance testing approaches, including QuickCheck dynamic and trace verification methods.

On day two, the team made significant progress towards estimating the cost of running a Leios node, considering different cost items such as network egress, CPU, and storage. They analyzed resource usage across different TPS levels, from 10 TPS to 1K TPS, and discovered that while there’s significant overhead at low throughput, the protocol becomes more efficient at higher TPS levels. The team hasn’t been able to finish all the cost items yet. The last two, IOPS and memory cost, will be added during this month.

On the last and third day, the team consolidated their options for how optimistic validation of input blocks can be accomplished. They defined three candidates, with one being favored. The main goal was to support the chaining of transactions with Leios, which requires defining a 'point in time' or stage of the protocol at which a subsequent or chained transaction can be built on top of an already submitted transaction. This can be achieved by having the node optimistically compute prospective ledger states using its local knowledge of input blocks referenced in certified endorser blocks or possibly ranking blocks.

• Day 1

• Day 2

• Day 3.

Simulation progress​

• Haskell simulation
  • Added support for dishonest nodes that diffuse an unbounded amount of old IBs, enabling further analysis of freshest-first and oldest-first vote delivery scenarios
  • Identified and fixed a bug in configuration generation for simulation runs, which was causing inconsistencies in vote delivery between default and uniform/extended voting schemes
  • Added an adversarial field to the network topology schema, allowing for the simulation of unbounded IB diffusion by dishonest nodes.

Ongoing investigations​

• Investigating the effects of unbounded IB diffusion on IB delivery reliability and protocol performance under such conditions
• Working on quantifying settlement times and their impact on protocol performance
• Exploring integration possibilities with Ouroboros Peras, mainly focusing on potentially reusing their voting mechanism to reduce resource consumption.

Additional resources​

• GitHub discussions – EB ledger states and 'history rewriting' effects
• The First Full Leios simulation analysis – detailed analysis of the latest simulation results.

This week, the Leios team continued working on various aspects of the protocol and its simulation capabilities. They made progress in implementing and testing the Haskell and Rust simulators, focusing on protocol behavior under different network conditions.

Simulation progress​

• Haskell simulation

  • Moved configuration and topology parsers to the leios-trace-hs package for reuse in formal methods
  • Investigated differences in IBs referenced with Rust simulation: identified that inconsistencies were caused by the same sequence of random samples being used across different runs
  • Simplified sortition code by using an external statistics package
  • Tested Full Leios, resolving tension between r_EB/eb-max-age-slots and praos-chain-quality/η
  • Fixed cabal run ols -- generate-topology close-and-random, listing producers properly and decreasing variance in upstream peers.

• Rust simulation

  • Investigated anomalies in simulation results: identified that earlier IB production failures were caused by low connectivity and lower CPU usage compared to the Haskell simulation
  • Refined Full Leios implementation
  • Added Full Leios support to the visualizer
  • Migrated the visualizer from Next.js to Vite.

Analysis of simulations​

• Tag leios-2025w13: simulated 198 Short Leios scenarios, varying IB production rate, IB size, network topology, CPU limits, and protocol flags
• CPU limits: analyzed the impact of CPU constraints on IB propagation, finding that diffusion can be affected under stress conditions
• Vote propagation: compared freshest-first and oldest-first vote propagation, with freshest-first potentially improving IB delivery reliability
• Extended voting period: compared an extended voting period to a limited one in the Haskell simulation, observing minimal differences except for occasional improvements in reliable vote delivery.

Ongoing investigations​

• Investigating qualitative discrepancies between Haskell and Rust simulation results to determine whether they stem from differences in simulator resolution or simulation infidelities.

Additional resources​

• Monthly review meeting – March 2025.

This week, the Leios team made significant progress across multiple areas. Major developments included detailed DeltaQ analysis of network topologies, extensive BLS cryptography benchmarking, and improvements to both simulations. The team also explored succinct schemes for BLS key registration and conducted a detailed certificate performance analysis. Both Haskell and Rust simulations received substantial updates to improve visualization and support more realistic testing conditions.

DeltaQ analysis​

• Enhanced the topology-checker with ΔQSD analysis capabilities:
  • Extracts inter-node latencies from given topologies
  • Classifies latencies into near/far components
  • Builds parameterized ΔQ models
  • Outputs fitted models in delta_q web app syntax
• Key findings from topology analysis:
  • Clear distinction between near/far components in examined topologies
  • Unexpectedly high hop counts in latency-weighted Dijkstra paths:
    • Min 4-5, max 8 for topology - 100
    • Min 8, max 20 for 'realistic' topology
  • Model fitting achieved rough shape matching but showed significant deviations at low latencies
  • Resource usage tracking goals remain unmet due to complexity in understanding load multiplication factors.

BLS cryptography​

• Completed comprehensive benchmarking of certificate operations:
  • Detailed performance analysis across committee sizes (500-1000 seats)
  • Certificate generation: 63.4ms - 92.5ms
  • Certificate verification: 104.8ms - 144.9ms
  • Certificate weighing: ~12ms consistently
• Explored succinct schemes for key registration:
  • Proposed 90-day key evolution with 124-byte KZG commitments
  • Analyzed message sizes for key opening (316 bytes per pool)
  • Investigated SNARK-based alternatives for proof of possession
• Added BLS crypto to the CI pipeline with automated testing
• Documented parallelization strategies for certificate operations.

Formal methods​

• Added a conformance testing client for the executable Short Leios specification
• Successfully merged the executable specification for Simplified Leios into main.

Haskell simulation​

• Updated configuration defaults for block sizes and timings
• Added support for idealized simulation conditions:
  • Single-peer block body requests
  • TCP congestion window modeling
  • Mini-protocol multiplexing
  • Unlimited bandwidth links support
• Enhanced simulation output and analysis:
  • Added raw field for accumulated data
  • Implemented block diffusion CDF extraction
  • Created multi-CDF plotting capabilities.

Rust simulation​

• Enhanced visualization capabilities:
  • Added block size breakdown display
  • Implemented total bytes sent/received tracking
  • Added total TX count and CPU time metrics
• Improved event handling:
  • Updated to standard timestamp format (seconds)
  • Enhanced CPU task event structure
  • Added CBOR output support
• Added support for multiple strategies:
  • Implemented ib-diffusion-strategy (freshest-first, oldest-first, peer-order)
  • Added relay-strategy affecting TXs, IBs, EBs, votes, and RBs
  • Enabled unlimited EB and vote bundle downloads from peers.

This week, the Leios team worked on cryptography benchmarking and cost calculator improvements. The team completed a reference implementation for Leios cryptography and enhanced the online cost calculator with user-requested features. They also updated both Haskell and Rust simulations to improve visualization and network modeling capabilities.

Haskell simulation​

• Added support for Send and Receive voting stages, providing:
  • A new leios-vote-send-recv-stages configuration option
  • A configurable stage length via leios-stage-active-voting-slots
• Implemented multiple diffusion strategies:
  • Added oldest-first strategy
  • Added configurable strategies for IBs, EBs, and votes via *-diffusion-strategy configurations
• Created a new small scenario for 100 nodes with 2,000 kB links
  • Tuned IB parameters to utilize one-third of link capacity
  • Added configurations for both single-stage and send-recv voting
• Fixed several simulation behaviors:
  • Improved block generation logic
  • Prevented duplicate EB inclusion in the base chain
  • Confirmed proper EB inclusion timing relative to vote diffusion
• The main difference observed between single-stage and send-recv is that the former shows a longer tail in the CPU usage CDF when the simulation is run with unlimited cores.

Cryptography implementation​

The Rust benchmarks for Leios cryptography were redesigned as a reference implementation:

• Implemented the Fait Accompli sortition
• Enhanced sortition to use rational arithmetic instead of quad-precision floats
• Added Quickcheck tests for all capabilities
• Added benchmarks for serialization
• Optimized vote and certificate size.

Cost calculator improvements​

The team enhanced the online Leios cost calculator:

• Added support for both hyperscale and discount cloud providers
• Made discount providers the default option
• Added option to amortize storage costs perpetually
• Updated defaults:
  • Single relay deployment
  • More conservative 50% disk compression
  • Perpetual storage cost amortization.

Throughput simulator​

The team updated the Cardano throughput simulator with:

• The latest cloud-computing cost model
• Synchronized assumptions with an online cost calculator.

Rust simulation​

• Made minor fixes to the new graph generation strategy
• Planned out a roadmap for visualization work focusing on the Leios transaction lifecycle.