Dev Update 20th May, 2025

[neptune-team]

This week by the numbers

• 1 issue closed
• 6 pull-requests raised
• 3 pull-requests merged

Mempool

A decision was made to keep the existing mempool implementation for now and patch it to address specific problem areas rather than spend effort on a complete re-write. The priority is to improve speed of transaction confirmation.

Merkle Tree and Removal Records

@aszepieniec reports that the Merkle tree work is ready for merging. Progress was made on the dense representation and packing strategies for removal records. This work necessitates modifying one of the consensus programs, specifically the merge path.

Chunk Length

It has been theoretically determined that using 12 bits to represent the length of a chunk is sound and has a proof. A unit test verifies the theoretical analysis.

Transaction Confirmation Issues

There has been discussion around issues related to transactions not being confirmed in a timely manner. It is a multifaceted problem involving the mempool, composer resources, and incentives for composers and proof operators. An initial PR has been merged that avoids zero-valued outputs when the composer gives the entire block subsidy to the guesser.

Block Propagation and Network Efficiency

@sword-smith had an insight that in many cases, it is not necessary to send out entire blocks, as nodes already have block proposals and would only need the nonce and guesser digest. This could potentially reduce network traffic spikes for new blocks, though (smaller) spikes for block proposals could still occur. This idea is currently hampered by the lack of tolerance for new message types and the inability to update handshake data.

Network Enumeration and Backwards Compatibility

@danda discussed the Network enum being exhaustive and suggested setting it to #[non-exhaustive] in the next release to start the path towards backwards compatibility. New variants cannot be added until we are certain all nodes have upgraded to a version with the #[non_exhaustive] marker.

Testnet Mock Network

@danda has been working on a testnet mock network which would be a public testnet that utilizes mock proofs and incorporates a difficulty reset mechanism similar to that used by bitcoin-core’s testnet. Such a network is much less likely to stall because blocks can be generated by any commodity device or vps. The intent is to test release-candidates on this network with a test-harness, as well as provide community members a window of time to test release candidates and report issues before each release.

Contributors updates

• @aszepieniec

  • merkle tree refactor
  • removal record compressed representation / packing
  • removal record statistics
  • presentation

• @sword-smith

  • Bootstrap from raw block data #585 and #517
  • Benchmark and improve speed of wallet state update, 587
  • Never add zero-valued composed outputs, #537
  • Make compiler compatible with latest Triton VM/tasm-lib

• @jfs spent most of the week on the mempool redesign by opening a draft pull-request that started using the suggested new API and discussing further design improvements with @danda. A re-evaluation of priorities revealed that the redesign might not be the most time efficient path to resolve the most pressing issues, and a series of patches was suggested instead. After passing the torch on to @sword-smith, @jfs started investigating a performance drop-off of Triton VM that was reported by a user.

• @danda

  • fixed #586 with merged PR #590 that avoids a panic in TransactionDetails::new() and improves error handling.
  • addressed review comments for #584 (job-queue ergonomics)
  • reviewed mempool design
  • investigate #596 – dashboard rpc shutdown when calling spendable_inputs with 2000+ spendable utxos.
  • make a testnet-mock
    • renamed Testnet::Alpha to TestnetMock
    • documented how to add new variants to Network in backwards-compatible way.
    • investigated if NeptuneProof could encode a ProofStream that would enable calls to nondeterminism() to succeed. Determined infeasible due to verification in StarkVerify::extract_nondeterministic_digests().
    • make ProofBuilder::nondeterminism() accept a closure so nondeterminism() is not called for mock proofs. fixes panic(s) in mining loop, which now runs without error. (also fixes for regtest)
    • add difficulty-reset (to genesis difficulty) when block interval >= 20 mins.
    • move some params into Network:
      • Network::perform_peer_discovery() → bool
      • difficulty_reset_interval(&self) → Option
      • genesis_difficulty(&self) → Difficulty
      • minimum_block_time(&self) → Timestamp
      • target_block_interval(&self) → Timestamp
    • some refactors to use network param(s) instead of constants.