20 posts tagged with "rust-simulation"

This week, the Leios team worked on cryptography benchmarking and cost calculator improvements. The team completed a reference implementation for Leios cryptography and enhanced the online cost calculator with user-requested features. They also updated both Haskell and Rust simulations to improve visualization and network modeling capabilities.

Haskell simulation​

• Added support for Send and Receive voting stages, providing:
  • A new leios-vote-send-recv-stages configuration option
  • A configurable stage length via leios-stage-active-voting-slots
• Implemented multiple diffusion strategies:
  • Added oldest-first strategy
  • Added configurable strategies for IBs, EBs, and votes via *-diffusion-strategy configurations
• Created a new small scenario for 100 nodes with 2,000 kB links
  • Tuned IB parameters to utilize one-third of link capacity
  • Added configurations for both single-stage and send-recv voting
• Fixed several simulation behaviors:
  • Improved block generation logic
  • Prevented duplicate EB inclusion in the base chain
  • Confirmed proper EB inclusion timing relative to vote diffusion
• The main difference observed between single-stage and send-recv is that the former shows a longer tail in the CPU usage CDF when the simulation is run with unlimited cores.

Cryptography implementation​

The Rust benchmarks for Leios cryptography were redesigned as a reference implementation:

• Implemented the Fait Accompli sortition
• Enhanced sortition to use rational arithmetic instead of quad-precision floats
• Added Quickcheck tests for all capabilities
• Added benchmarks for serialization
• Optimized vote and certificate size.

Cost calculator improvements​

The team enhanced the online Leios cost calculator:

• Added support for both hyperscale and discount cloud providers
• Made discount providers the default option
• Added option to amortize storage costs perpetually
• Updated defaults:
  • Single relay deployment
  • More conservative 50% disk compression
  • Perpetual storage cost amortization.

Throughput simulator​

The team updated the Cardano throughput simulator with:

• The latest cloud-computing cost model
• Synchronized assumptions with an online cost calculator.

Rust simulation​

• Made minor fixes to the new graph generation strategy
• Planned out a roadmap for visualization work focusing on the Leios transaction lifecycle.

The Leios team continued refining Haskell and Rust simulations, standardizing inputs, outputs, and event logging for better comparability. The team defined standard formats for configuration parameters and network topology for running the Leios protocol. They also worked on logging identical simulation events to compare and feed them into the DeltaQ model and, consequently, the executable specification, ensuring alignment with formal methods.

Haskell simulation updates​

• The short-leios simulation now outputs diffusion latency data
• Added support for different input block (IB) diffusion strategies:
  • freshest-first: higher slot numbers requested first
  • peer-order: requested in order of peer announcement
• Added support for Vote (Send) and Vote (Recv) stages.

Rust simulation progress​

• Added an 'organic' topology generator that better matches mainnet topology
• The generator creates clusters of colocated stake pools and relays
• The simulation uses stake to determine relay connectivity
• Topology insights gathered from stake pool owners:
  • Most pools have multiple relays (2,312 relays across 1,278 pools)
  • Pool operators often run multiple colocated pools sharing relays
  • Relays typically maintain ~25 active outgoing connections
  • Incoming connections scale with stake weight (10-400+ connections).

DeltaQ update​

• Wrote a comprehensive 2025-01 report covering work since September 2024.

Formal methods​

• Finalizing executable specifications for simplified and short Leios
• Extracted short Leios specification to Haskell for conformance testing.

Simulation progress​

Haskell implementation​

• Enhanced parameter handling with support for reading configurations and topologies from disk
• Added a new generate-topology command for random topology generation
• Aligned Leios sortition with algorithms from sortition benchmarks and the technical report
• Completed analysis comparing the Praos simulation with the benchmark cluster
  • Adoption times within 10% of measured values
  • Review of simulation parameters pending
• Next steps identified:
  • Generate topologies with block producers behind relays
  • Begin comparison with the idealized diffusion model
  • Configure and run simulations for higher throughput.

Rust implementation​

• Completed the first pass of block-level visualization
• Updated topology files to include baked-in latencies
• Improved output with human-readable names from the shared topology format
• Enhanced simulation output comparability across different simulations.

Analysis and research​

Sortition analysis​

• Completed a detailed analysis of the 'Fiat Accompli' sortition scheme using mainnet stake distribution (Epoch 535)
• Key findings for 500-vote committees:
  • 406 largest stake block-producers would be deterministic voters
  • ~88 voters would be randomly selected
  • Significant certificate size reduction achieved through deterministic voter selection.

Downstream impact assessment​

Started comprehensive analysis of Leios's impact on the ecosystem:

• Identified impacts on indexers, explorers, SDKs, and APIs resulting from ledger and node changes
• Transaction construction and memory-pool sharding effects on DApps and wallets
• Physical layer visibility considerations for sophisticated use cases
• High throughput implications for event filtering efficiency
• Transaction journey time considerations from memory pool to Praos block reference.

DeltaQ analysis​

• Successfully matched ΔQ model for IB diffusion across both simulations and implementations
• Identified key differences in simulation approaches:
  • Haskell simulation includes bandwidth effects (328ms network delay per hop at 1MB/s)
  • Rust simulation currently excludes bandwidth effects
• Enabled cross-simulation topology sharing for consistent testing.

Cryptography benchmarks​

• Implemented and benchmarked the complete Leios cryptography suite in the leios_crypto_benchmarks Rust crate
• Key VRF performance metrics:
  • Proving: 240 µs
  • Verifying: 390 µs
• Sortition performance (excluding VRF):
  • Leadership checks (RB/IB/EB): 0.17 µs per slot/pipeline
  • Vote number calculation: 3.8 µs per pipeline
• BLS operations benchmarked:
  • Key possession proof verification: 1.5 ms per key
  • Vote generation/verification: 280 µs / 1.4 ms per vote
  • Certificate operations (300-vote quorum): 50 ms generation, 90 ms verification.

Cryptography design progress​

• Optimized vote signature size to potentially as small as 192 bytes
• Determined that 500-vote committee certificates (60% quorum) would fit within Praos blocks at ~58 kB
• Explored potential synergies with KES rotation and Praos VRF BLS keys
• Completed cryptography sections for the first technical report
• Decision made to freeze current report content and move new findings to future documents.

Simulation development​

Haskell simulation​

• Achieved diffusion latency comparable to benchmark cluster data for Praos blocks
• Integrated agreed-upon simulation parameters with the Rust team
• Added event log output functionality with JSON support
• Implemented 'short-leios' simulation variant matching mainnet ranking block interval
• Fixed coordination issues in Relay mini-protocol consumers
• Completed the PI goal by adding total data transmitted per node visualization.

Rust simulation​

• Implemented more granular CPU simulation times
• Fixed race condition in the simulated clock
• Started consuming a new shared configuration file format
• Established a shared configuration format with default parameters in data/simulation/default.yaml.

Rust simulation​

• Added a basic simulation of central processing unit (CPU) usage/latency
• Implemented 'lottery won' events to identify the start of CPU processing
• Configured each node with four simulated cores, adjustable per node
• Transaction validation and ranking block/input block/endorser block generation/validation each take one CPU task
• All virtual CPU costs were copied from the cost estimator.

DeltaQ summary update​

• Added MIN/MAX combinators for best- and worst-case simulation results
• The Rust simulation best case does not match the analytically best behavior
• The Haskell simulation best case is too fast; the ΔQ expression must assume more than 200 peers per node.

Cost dashboard updates​

• Improved input parameters and computations
• Lengthened phases and reduced endorser block rate
• Updated CPU costs for votes and certificates
• Revised input/output operations per second (IOPS) values based on empirical data from Cardano nodes.

Benchmarking BLS signatures​

• Benchmarked BLS votes using the Rust bls-signatures package
• Aggregate verification significantly speeds up the process
• Provided CPU time estimates for various operations.

Votes and certificates​

• Updated size estimates for votes
• Added CPU time estimates for BLS votes and certificates
• Drafted technical report sections on BLS and MUSEN certificates.

Sortition analysis​

• Analyzed sortition for input and endorser blocks and votes
• Added findings to the draft of the first technical report.

Rust simulation​

• Abandoned Waxman graph generation favoring a more straightforward distance-weighted approach to better control graph connectivity.

Haskell simulation​

• Added support for bounded and unbounded parallelism to the Leios node
• Fixed relay protocol messages to ensure ordered delivery
• Next steps include loading protocol configuration from disk and investigating endorser block (EB) inclusion rates.

Revised analysis of votes and certificates​

• Continued research on cryptographic options for Leios votes and certificates
• BLS was identified as the most viable option.

Jupyter support for DeltaQ​

• Introduced new high-performance Haskell packages for DeltaQ with comprehensive test suites.

GitHub actions​

• Organized continuous integration (CI) configuration with namespace prefixes for better project sorting.

Rust simulation​

• Began visualizing transaction throughput
• Tweaked configuration settings to maximize throughput.

Haskell simulation​

• Merged code for running Praos and Leios visualizations
• Added HLint integration for consistent module imports.

Public meeting on December 28, 2024​

• Meeting recording
• Meeting slides.

Rust simulation​

• Optimized virtual clock to be lock-free, removing contention from the previous implementation.

Haskell simulation​

• Merged Leios visualizations on main
• Improved peer-to-peer (P2P) visualization with block type differentiation and latency charting.

Analysis of vote size and ALBA certificates​

• Estimated minimum possible size for votes using ephemeral keys or key evolving signatures (KES)
• Benchmarked central processing unit (CPU) time for ALBA certificates.

Node discussions with IOG site reliability engineering​

• Discussed the cloud engineering aspects of Cardano node deployments relevant to Leios
• Received recommendations for updating the cost calculator with predefined cost defaults.

Threat model analysis​

• Drafted a section on the Leios threat model analysis, detailing 25 hypothetical threats
• Discussed potential impacts and mitigations.

Rust simulation​

• Generated new test data set to match the geographical distribution of mainnet nodes.

Nix Development Environment​

• Created a separate repository for a nix develop environment with the correct Haskell compiler and dependencies.

Rust Simulation​

• Developed a fully virtual clock to decouple simulation performance from wall clock time.

ΔQ Model​

• Created ΔQ models for transaction diffusion in Rust and Praos block diffusion in Haskell simulations.
• Identified discrepancies between simulation behavior and ΔQ expressions.