This is the first in a hopefully long-running series of weekly development updates.
The Neptune team is beginning a new initiative to inform the wider community of ongoing progress and improve transparency with regards to the development process. These updates are created as a collaborative document by individual contributors.
We plan to publish an update every Tuesday, so mark your calendars. We invite comments, q’s, and discussion.
In this first post, we will summarize some of the bigger changes going into neptune-core since release v0.2.2.
refactor of transaction-creation code and a new public API for initiating transactions.
This exposes a set of public APIs for fine-grained control over all aspects of creating a neptune transaction. Both for rust and RPC usage. It also marks the beginning of a new well-documented “api” module that is specifically intended for third-party usage of neptune-core as a rust library. This module aims to ease the task of wallet developers or anyone building out the neptune ecosystem.
regtest mode with mock proofs for controlled testing and simulations
neptune-core has always had a --network regtest option, but it behaved just like the other networks. The new regtest mode makes it work more like bitcoin-core’s regtest network. In particular any number of blocks can be instantly generated on demand by calling the new mine_blocks_to_wallet RPC method. This makes it easier and faster to test software that uses the neptune-core RPC interface.
neptune-core transactions and blocks require proofs that presently take minutes to generate for even the fastest machines. The regtest mode side-steps this by utilizing mock proofs that are accepted as valid by peer nodes.
wallet database schema migration framework
As neptune-core changes over time, it is important to maintain compatibility with databases created by older versions of the software. The chosen approach is to version the schema of databases and automatically migrate older schema versions to the latest version. This has so far been implemented for the wallet database due to the addition of a new field in the TxOutput type which is stored in the wallet-db.
integration tests for testing neptune-core nodes, alone or networked.
neptune-core already has hundreds of unit tests covering individual areas of the code.
We have now created an integration test framework that facilitates writing tests that launch simulated nodes in regtest mode and perform transactions, etc. These tests exercise areas of the code and interactions between nodes that are not covered by unit tests.
use shared job-queue and tokio runtime in async unit tests
This is kind of a housekeeping item.
neptune-core uses a JobQueue to ensure that only one proof can be generated at a time by the triton virtual machine. This is necessary to prevent overloading the physical hardware and running out of resources, especially RAM. Previously unit tests were each using their own JobQueue instance which meant the tests had to be run sequentially when generating new proofs (proofs are cached for tests). These tests have now been improved to use a shared tokio runtime which enables them to share the JobQueue as intended.
Individual contributor updates:
- Work on compressed representation for lists of
RemovalRecords - Refactor Merkle tree in [
twenty-first] - Migrate Neptune Cash website to new server
- PR reviews
- User tech support
@jfs:
- Work towards compiling
twenty-firston 32-bit architectures, specifically, WASM. - Host Triton VM’s website through GitHub pages, not custom server. Shoutout & thanks to @sshine for hosting the site up to now!
- Move continuous integration towards passing state: update the runner images used in distribution tool
dist. - Start work on a new mempool design.
- finished up mock-proofs for regtest mode PR and got it merged.
- use single tokio runtime in tests, for single jobqueue
- fixed broken tests to get CI (continuous integration) green again, including a stubborn windows firewall issue.