Lighthouse targets its first release, introduces node discovery using discv5 and adds a new, experimental fork-choice algorithm
Quite a few exiting developments have occurred these past weeks. As a simple summary, we have:
- Lighthouse is targeting it's first release
v0.0.1with a tentative release date of 15/7.
- An initial discovery v5 libp2p module has been built and integrated into lighthouse. Nodes now discover each other using this protocol.
- Implemented a new reduced-tree fork choice algorithm, which (with current testing) has shown ~5x speed improvements (without additional overheads) to previous implementations.
- Passing all
v0.6.3tests. Intend to skip
v0.7.xand go straight to the frozen spec, once released.
- Update Lighthouse to the frozen specification.
- Improve and stabilise the syncing mechanism.
- Update the libp2p ETH2-RPC to handle substreams for request/response.
- Add extra network testing/tooling for testing network protocols at larger scales.
- Start pushing the changes in our core BLS library to the Apache Milagro upstream library.
With the upcoming spec-freeze, we're pleased to announce that Lighthouse will
now be producing releases. The first release,
v0.0.1 (Genesis) has a
tentative date of the 15/7, and will be primarily targeted for researchers and
developers. Expected items that will be built in the release can be found
here. Continuing on from
v0.0.1will be more structured development through releases as Lighthouse
begins its interoperability phase with other clients.
Reduced Tree Fork Choice
During the annual IC3 event at Cornell University, a team lead by Vlad Zamfir developed a LMD GHOST fork-choice mechanism dubbed "reduced-tree". This algorithm is directly applicable to Eth2 and we have implemented it in Lighthouse with great results. For those unfamiliar with fork choice algorithms, we discuss them in Update #08. The slides from the presentation can be found here and the full membership of the group is: Vlad Zamfir, Aditya Asgaonkar, Nate Rush, Carl Beekhuizen, Hsiao-Wei Wang & Paul Hauner.
Each Eth2 full-node naturally has some form of database that stores all the blocks in the Beacon Chain so they can be provided to users or used to validate other blocks. Several implementations, Lighthouse included, keep another fork-choice-dedicated block-tree that acts as a cache to reduce the time taken to run the fork choice algorithm.
The key observation from the IC3 group was that the fork-choice algorithm does not need to store information about every single block in the block-tree. Instead, it only needs to be concerned about blocks that are the target of a validators latest vote (message) or blocks that are a decision point between the two. The idea that fork-choice can safely bypass some blocks is the basis of the name "reduced-tree".
The reduced-tree has the advantages that it is smaller to store and requires less operations to iterate through. However, compared to the full-tree, we must define more complex add/remove/prune operations to maintain the reduced-tree. We have not performed detailed analysis of reduced-tree versus full-tree, however we have observed significant performance improvements in our testnets. Notably, there are several optimisations proposed for full-tree fork-choice that are also applicable to reduced-tree, so we can expect to see further efficiency improvements.
As was mentioned in previous updates, we have been working on a discovery v5
implementation. This has been built in a branch of our rust-libp2p library and
can be found here. This
protocol implements the high level RPC messages
These are sufficient to use the protocol as a discovery mechanism to find other
nodes on the network and determine external IP's for NAT traversal. The libp2p
integration allows for libp2p
PeerId's to be discovered and connected to once
peers are found on the UDP discovery mechanism.
Currently topic advertisement and registration have not been implemented and
are postponed until further testing of the fundamental implementation is
Fuzzing continues on the new and existing protocols being built on lighthouse. In particular, we have begun testing some of the higher-level functionality within lighthouse, such as state transitions.
We're now targeting block processing by loading a minimal state file and
fuzzing attestation, attestation slashing and block header processing
functions. Code coverage is optimised by providing
libFuzzer a valid corpus
(i.e. examples of valid blocks, headers, attestations, etc.) and fuzzing speed
is greatly enhanced by loading the beacon state from file.
Our naive per block processing fuzzer has also been updated to use blocks with valid signatures as input.