10 posts tagged with "deltaq"

This week, the Leios team made significant progress across multiple areas. Major developments included detailed DeltaQ analysis of network topologies, extensive BLS cryptography benchmarking, and improvements to both simulations. The team also explored succinct schemes for BLS key registration and conducted a detailed certificate performance analysis. Both Haskell and Rust simulations received substantial updates to improve visualization and support more realistic testing conditions.

DeltaQ analysis​

• Enhanced the topology-checker with ΔQSD analysis capabilities:
  • Extracts inter-node latencies from given topologies
  • Classifies latencies into near/far components
  • Builds parameterized ΔQ models
  • Outputs fitted models in delta_q web app syntax
• Key findings from topology analysis:
  • Clear distinction between near/far components in examined topologies
  • Unexpectedly high hop counts in latency-weighted Dijkstra paths:
    • Min 4-5, max 8 for topology - 100
    • Min 8, max 20 for 'realistic' topology
  • Model fitting achieved rough shape matching but showed significant deviations at low latencies
  • Resource usage tracking goals remain unmet due to complexity in understanding load multiplication factors.

BLS cryptography​

• Completed comprehensive benchmarking of certificate operations:
  • Detailed performance analysis across committee sizes (500-1000 seats)
  • Certificate generation: 63.4ms - 92.5ms
  • Certificate verification: 104.8ms - 144.9ms
  • Certificate weighing: ~12ms consistently
• Explored succinct schemes for key registration:
  • Proposed 90-day key evolution with 124-byte KZG commitments
  • Analyzed message sizes for key opening (316 bytes per pool)
  • Investigated SNARK-based alternatives for proof of possession
• Added BLS crypto to the CI pipeline with automated testing
• Documented parallelization strategies for certificate operations.

Formal methods​

• Added a conformance testing client for the executable Short Leios specification
• Successfully merged the executable specification for Simplified Leios into main.

Haskell simulation​

• Updated configuration defaults for block sizes and timings
• Added support for idealized simulation conditions:
  • Single-peer block body requests
  • TCP congestion window modeling
  • Mini-protocol multiplexing
  • Unlimited bandwidth links support
• Enhanced simulation output and analysis:
  • Added raw field for accumulated data
  • Implemented block diffusion CDF extraction
  • Created multi-CDF plotting capabilities.

Rust simulation​

• Enhanced visualization capabilities:
  • Added block size breakdown display
  • Implemented total bytes sent/received tracking
  • Added total TX count and CPU time metrics
• Improved event handling:
  • Updated to standard timestamp format (seconds)
  • Enhanced CPU task event structure
  • Added CBOR output support
• Added support for multiple strategies:
  • Implemented ib-diffusion-strategy (freshest-first, oldest-first, peer-order)
  • Added relay-strategy affecting TXs, IBs, EBs, votes, and RBs
  • Enabled unlimited EB and vote bundle downloads from peers.

The Leios team continued refining Haskell and Rust simulations, standardizing inputs, outputs, and event logging for better comparability. The team defined standard formats for configuration parameters and network topology for running the Leios protocol. They also worked on logging identical simulation events to compare and feed them into the DeltaQ model and, consequently, the executable specification, ensuring alignment with formal methods.

Haskell simulation updates​

• The short-leios simulation now outputs diffusion latency data
• Added support for different input block (IB) diffusion strategies:
  • freshest-first: higher slot numbers requested first
  • peer-order: requested in order of peer announcement
• Added support for Vote (Send) and Vote (Recv) stages.

Rust simulation progress​

• Added an 'organic' topology generator that better matches mainnet topology
• The generator creates clusters of colocated stake pools and relays
• The simulation uses stake to determine relay connectivity
• Topology insights gathered from stake pool owners:
  • Most pools have multiple relays (2,312 relays across 1,278 pools)
  • Pool operators often run multiple colocated pools sharing relays
  • Relays typically maintain ~25 active outgoing connections
  • Incoming connections scale with stake weight (10-400+ connections).

DeltaQ update​

• Wrote a comprehensive 2025-01 report covering work since September 2024.

Formal methods​

• Finalizing executable specifications for simplified and short Leios
• Extracted short Leios specification to Haskell for conformance testing.

Simulation progress​

Haskell implementation​

• Enhanced parameter handling with support for reading configurations and topologies from disk
• Added a new generate-topology command for random topology generation
• Aligned Leios sortition with algorithms from sortition benchmarks and the technical report
• Completed analysis comparing the Praos simulation with the benchmark cluster
  • Adoption times within 10% of measured values
  • Review of simulation parameters pending
• Next steps identified:
  • Generate topologies with block producers behind relays
  • Begin comparison with the idealized diffusion model
  • Configure and run simulations for higher throughput.

Rust implementation​

• Completed the first pass of block-level visualization
• Updated topology files to include baked-in latencies
• Improved output with human-readable names from the shared topology format
• Enhanced simulation output comparability across different simulations.

Analysis and research​

Sortition analysis​

• Completed a detailed analysis of the 'Fiat Accompli' sortition scheme using mainnet stake distribution (Epoch 535)
• Key findings for 500-vote committees:
  • 406 largest stake block-producers would be deterministic voters
  • ~88 voters would be randomly selected
  • Significant certificate size reduction achieved through deterministic voter selection.

Downstream impact assessment​

Started comprehensive analysis of Leios's impact on the ecosystem:

• Identified impacts on indexers, explorers, SDKs, and APIs resulting from ledger and node changes
• Transaction construction and memory-pool sharding effects on DApps and wallets
• Physical layer visibility considerations for sophisticated use cases
• High throughput implications for event filtering efficiency
• Transaction journey time considerations from memory pool to Praos block reference.

DeltaQ analysis​

• Successfully matched ΔQ model for IB diffusion across both simulations and implementations
• Identified key differences in simulation approaches:
  • Haskell simulation includes bandwidth effects (328ms network delay per hop at 1MB/s)
  • Rust simulation currently excludes bandwidth effects
• Enabled cross-simulation topology sharing for consistent testing.

Rust simulation​

• Added a basic simulation of central processing unit (CPU) usage/latency
• Implemented 'lottery won' events to identify the start of CPU processing
• Configured each node with four simulated cores, adjustable per node
• Transaction validation and ranking block/input block/endorser block generation/validation each take one CPU task
• All virtual CPU costs were copied from the cost estimator.

DeltaQ summary update​

• Added MIN/MAX combinators for best- and worst-case simulation results
• The Rust simulation best case does not match the analytically best behavior
• The Haskell simulation best case is too fast; the ΔQ expression must assume more than 200 peers per node.

Cost dashboard updates​

• Improved input parameters and computations
• Lengthened phases and reduced endorser block rate
• Updated CPU costs for votes and certificates
• Revised input/output operations per second (IOPS) values based on empirical data from Cardano nodes.

Benchmarking BLS signatures​

• Benchmarked BLS votes using the Rust bls-signatures package
• Aggregate verification significantly speeds up the process
• Provided CPU time estimates for various operations.

Votes and certificates​

• Updated size estimates for votes
• Added CPU time estimates for BLS votes and certificates
• Drafted technical report sections on BLS and MUSEN certificates.

Sortition analysis​

• Analyzed sortition for input and endorser blocks and votes
• Added findings to the draft of the first technical report.

Rust simulation​

• Abandoned Waxman graph generation favoring a more straightforward distance-weighted approach to better control graph connectivity.

Haskell simulation​

• Added support for bounded and unbounded parallelism to the Leios node
• Fixed relay protocol messages to ensure ordered delivery
• Next steps include loading protocol configuration from disk and investigating endorser block (EB) inclusion rates.

Revised analysis of votes and certificates​

• Continued research on cryptographic options for Leios votes and certificates
• BLS was identified as the most viable option.

Jupyter support for DeltaQ​

• Introduced new high-performance Haskell packages for DeltaQ with comprehensive test suites.

Nix Development Environment​

• Created a separate repository for a nix develop environment with the correct Haskell compiler and dependencies.

Rust Simulation​

• Developed a fully virtual clock to decouple simulation performance from wall clock time.

ΔQ Model​

• Created ΔQ models for transaction diffusion in Rust and Praos block diffusion in Haskell simulations.
• Identified discrepancies between simulation behavior and ΔQ expressions.

Visualizing Node Behavior​

• Added a web server to render throughput of IBs and EBs in the Leios simulation.
• Utilized Tracer interface for JSON-formatted traces and live graph plotting.

Discussion with Nicolas​

• Explored Leios protocol with a focus on IB to EB construction.
• Modeled a single node with input and output queues for IBs and EBs.

Meeting with Research​

• Discussed ongoing research work and networking challenges.
• Aimed to define a ΔQ model for Leios and explore protocol structure.

Public Meeting on November 27, 2024​

• Meeting Recording
• Meeting Slides

Structuring Repository for Open-source​

• Added basic instructions for Code of Conduct, Contribution guidelines, and coding standards.
• Restructured code to merge leios and leios-sim packages.

Designing Test Environment​

• Sketched ideas for exploring Leios behavior with respect to networking.

Leios Meeting​

• Discussed network simulation and ΔQ model of the pipeline.
• Gathered numbers on network latency and throughput.

Visualizing Node Behavior​

• Added a simple web server to render the throughput of IBs and EBs in the Leios simulation.
• Utilized the Tracer interface for JSON-formatted traces and live graph plotting.

Discussion with Nicolas​

• Explored Leios protocol with a focus on IB to EB construction.
• Modeled a single node with input and output queues for IBs and EBs.

Meeting with Research​

• Discussed ongoing research work and networking challenges.
• Aimed to define a ΔQ model for Leios and explore protocol structure.

Public Meeting on October 31, 2024​

• Recording
• Slides

Designing Test Environment​

• Sketched ideas for exploring Leios behavior with respect to networking.
• Developed a high-level design for network simulation.

Leios Meeting​

• Discussed network simulation and ΔQ model of the pipeline.
• Gathered numbers on network latency and throughput.

Structuring Repository for Open-source​

• Added basic instructions for Code of Conduct, Contribution guidelines, and coding standards.
• Restructured code to merge leios and leios-sim packages.