KAGOME – the C++ implementation of Polkadot Host milestone 3
KAGOME treasury proposal 3
KAGOME is a C++ implementation of the Polkadot Host that is protocol compatible with the original implementation in Rust and is already participating in Kusama relay chain validation. With KAGOME and other client implementations we bring client diversity to Polkadot, mitigating risks of fatal bugs, bringing innovations and broadening the development community. During previous work as part of Polkadot treasury proposal 2, the following results were achieved:
- Asynchronous backing
- BEEFY integration
- New Wasm engine (WasmEdge)
- Initial security audit by SRLabs (link)
- General maintenance
In addition to features that were planned during the previous proposal our team achieved the following:
- Implemented Grid and Cluster network topologies for statements distribution
- Partially Integrated disabled validators feature
- Partially implemented elastic scaling
- Implemented multiple security features
This proposal asks for funding for the next 4 months, in addition to the previous 6 months of work on features that have been retroactively implemented (marked with Done status in Section 4).
Problem statement
Client diversity is essential for the security and resilience of Polkadot as it helps to mitigate the risk of bugs and exploits. If there is only one implementation (which is currently the case Polkadot), then any bug or exploit that is found in that implementation could potentially bring down the entire network. However, with multiple implementations, the risk of a bug or exploit affecting all of the nodes in the network is greatly reduced.
Importance of having multiple client implementations was highlighted multiple times by Polkadot twitter (source), and Web3 foundation RFP (source). Moreover, W3F recognizes clients diversity importance and allocates 10 million DOT prise for the future multiple implementations of JAM protocol.
Gavin Wood mentioned KAGOME during 2023 yearly roundup article (link) and highlighted our achievement of executing KAGOME node in Kusama during his recent interview (link).
To learn more about multi-client philosophy and KAGOME Polkadot Host implementation watch our previous presentations:
- Building alternative clients in Polkadot | Polkadot Decoded 2023
- Polkadot Host architecture in 2024 | Sub0 Asia 2024
Alignment with JAM
Quadrivium is planning to develop a JAM client and is already working on some features for this in parallel with KAGOME. We also started work on SASSAFRAS many months ago, but stopped because the reference implementation in Rust had not yet been merged into the Polkadot SDK. Given that SAFROLE is the simplified version of SASSAFRAS, we are well positioned to implement a new leader election mechanism in the JAM implementation.
However, we recognize the importance of multiple Polkadot Host implementations today, especially since most features are already available and KAGOME is compatible with the Polkadot SDK. Moreover, it's not anticipated that JAM will be launched within the next one or two years. Luckily, we can repurpose much of the KAGOME codebase for the future JAM client implementation. For instance:
- Grandpa – fully implemented and audited.
- SASSAFRAS – partially implemented.
- SCALE – fully implemented.
But since we didn't initially intend to reuse these components, some refactoring of KAGOME is required. This will enable these components to be reusable and easily integrated into the future JAM client implementation.
The scope of proposal
- Security improvements (retroactive)
- Grid and cluster topologies (retroactive)
- Elastic scaling (partially implemented)
- Disabled validators mechanism (partially implemented)
- Validation protocol upgrade
- Systematic chunks
- Minor features and improvements (partially implemented)
- DevOps and QA maintenance
- Security assurance by SRLabs
Requested DOT
- Quadrivium: 86507
- SRLabs: 54888
- Total: 141395
Please review the full proposal with detailed tasks descriptions and price breakdown: https://docs.google.com/document/d/1NOkXMSAOYYgm_NCEzuTCKE-rl5MnlaZD3FN8v0otYbc
About Quadrivium
Quadrivium (https://www.qdrvm.io ) is a blockchain infrastructure development company founded in 2023.
The company specializes in the development of blockchain clients, peer-to-peer networking tools, and zk-cryptography. Quadrivium develops KAGOME Polkadot Host implementation, in partnership with the Web3 Foundation. The company also maintains the C++ libp2p library.
About SRLabs
SRLabs (https://www.srlabs.de/) is home to knowledge leaders securing critical infrastructures in finance, blockchain, energy, and telecommunications.
The company focuses on hands-on hacking resilience, not compliance. Their approach is shaped by combining their hacking research with impactful consulting work for innovation leaders who naturally thrive on cutting-edge technologies.
SRLabs is one of the leading blockchain audit companies with experience in many Substrate-based blockchains, including the Polkadot layer-0 relay chain and parachains built on top.
Comments (5)
Requested
Proposal Passed
Summary
0%
Aye
0%
Nay
Aye (42)0.0 DOT
Support0.0 DOT
Nay (33)0.0 DOT
Hello, thank you for submitting your proposal.
Regarding JAM: are you planning to fully move to JAM impl eventually? You mention in the document that some components for KAGOME can be repurposed: I am wondering if you have a timeline to start with the JAM client impl and whats the roadmap for this, since you mentioned this on the document.
SRLabs security assurance team is for a total of 54888 DOT: 4 people, 5 months. Is the scope of the audit that big to be valued at this rate? I assume then this covers features not included on this proposal and already developed? And do you plan to publish the audit reports?
Thank you guys!
@rtti5220
Thank you for the comment, Raul. With the news of JAM protocol we took a serious consideration of how to proceed with implementation of KAGOME. We carefully reviewed graypaper and consulted many Parity members including Gavin Wood and Pierre Aubert.
For the near future (at very least one year), we envision KAGOME as our core project. We will be able to switch KAGOME to the maintenance mode as soon as reference Rust implementation (Polkadot-SDK) will do the same as we always want to have the full compatibility.
Nonetheless, we have already started working on JAM-related tasks and allocated two employees to focus on JAM-specific features like the binary Merkle trie and PVM implementation. We’ve got the impression that this is similar framework to what is going on in the Parity, where the majority of the team is still working on Polkadot-SDK.
The first iterations of JAM's binary trie are planned to be developed by the end of Q3 this year, and PVM closer to the end of the year.
I want to emphasize that we are not double-charging for intersecting features between JAM and KAGOME. For example, we implemented a significant portion of the SASSAFRAS (predecessor of SAFROLE) algorithm into KAGOME (PR, PR). However, since SAFROLE is within the scope of JAM prizes, we won’t request reimbursement for that from the treasury as part of KAGOME project.
Regarding moving completely from KAGOME, we will consider it later when the JAM implementation matures. Given that JAM's introduction doesn't eliminate parachains, and considering that Parachain and Relay chain nodes share many common components, one possibility could be repurposing KAGOME's source code to implement the Parachain Host.
Audit scope will be fixed with the release v0.9.5 which we plan for August when the latest features like systematic chunks and elastic scaling are implemented.
Since the initial security audit was conducted on version 0.9.3, you may already estimate the diff (which will expand by August) that SRLabs team will take for review (link). The detailed scope is also described in more details in section 4.IX and 5.
The audit report will be published, just like we did it last time (link).
Regarding, the price of the audit I would like to share the SRLabs team's response:
"The audit scope is comprehensive, covering all new features and retroactive improvements since the last audit by SRLabs. Additionally, SRLabs will be monitoring pull requests and providing additional security consultations as needed
The rate of 54,888 DOT for a 4-person team over 5 months is competitive and reflects the high level of expertise and thoroughness required for this kind of work. As with the first security audit, we also plan to publish all audit reports for transparency."
I will add that very few teams understand the Polkadot protocol at the level SRLabs does. They have been working in the Polkadot ecosystem for many years, and their expertise has positively impacted the quality of KAGOME after the previous audit.
Hope this clarifies things!
Because JAM implementations are just getting started but aiming directly for multiple client implementations, OpenGov should support Kagome and Gossamer and "keep the old tech going" (Polkadot) while the "new tech" (JAM) is being put together. We believe this investment into MULTIPLE client implementations for the "old tech" will have some payoff for when the "cutover" from Polkadot => JAM will happen and expect Gossamer + KAGOME to be active participants in the cutover. We are very glad this team (of two) is implementing JAM and assuming Gossamer does the same, we urge any voters to look past the "oh, so they may win both JAM prize AND OpenGov" and recognize that the more teams we have doing both "old" and "new" in multiple languages the better for both Polkadot present and JAM future. This is a competent team at the heart of Polkadot SDK audits. Any "retroactive" payment is a test of OpenGov/Polkadot Community honoring their commitment and so long as it is reasonable should be honored.
@sourabhniyogi
This is actually a horrible argument, IMO. There should be a complete disinvestment in old projects whilst the new is being built. That's what a Steve Jobs would do, if he were in charge.