Treasury Proposal: QMDB Core Integration for Substrate Storage

Enhancement Proponent: 12K97oseeBxoWnyYRU3DJkxKTiDGFBh1pK4jLyuUJzYHkqdD

Date: [29.01.2025]

Requested DOT: 5 000

Short description: Initial integration of QMDB (https://github.com/LayerZero-Labs/qmdb) core functionality into Substrate (https://github.com/paritytech/polkadot-sdk). Project Category/Type: Storage and Performance Optimization

Previous Treasury Proposals:

**Context of the Proposal

**This proposal focuses on the first phase of QMDB integration, specifically implementing core storage functionality without iterators and prefixes with limited support or non-optimal support. This foundational work will enable future enhancements while maintaining compatibility with existing Substrate systems.

Project Details

Technology stack:

• RUST language for QMDB integration with Substrate
• DevOps tools for deployment and monitoring

Documentation:

• Core components and architecture documentation to be developed
• Integration specifications and API documentation
• Performance benchmarking methodology

Research and PoC:

• https://github.com/QuantumFusion-network/spec/issues/16
• https://qfnetwork.xyz/litepaper

What this project will not provide:

• Full prefix search functionality
• Production-ready indexing solution
• Advanced iterator implementation

Team members

• Team Lead: Vas Soshnikov
• Team Member: Andrew Orlov

Team's experience

Previous Polkadot grants:

• https://github.com/kaonone/polkahub-monorepo
• https://substrate.io/ecosystem/projects/polkahub/

Team Code Repos

• https://github.com/QuantumFusion-network
• https://github.com/andor0
• https://github.com/dedok

Problem Statement
Current Substrate storage implementation faces several fundamental challenges that this initial integration phase aims to address: Current crate ‘sp-trie’ implementation requires optimisation Inefficient key-based operations affecting basic transaction processing Suboptimal transaction verification process

Key Limitations of QMDB

• Does not support iterators and prefix searches
• Non-limitation, mostly its design of QMDN, requires additional memory (RAM, Disk) for indexing (indexer)

Proposal Objectives & Solutions Objectives

• Integrate QMDB as the proof storage engine within Substrate
• Enable Merkle Proof-based transaction verification
• Benchmark QMDB inside our DevNet (https://dev.qfnetwork.xyz)

Milestones, Timelines, and Budgets Milestone

1. Implement QMDB storage traits Working prototype integrated with Substrate
3 weeks
2 000 DOT

2. Develop a storage indexing mechanism In-memory B-Tree or sorted sparse list
2 weeks
1 500 DOT

3. Conduct full system testing All Substrate tests pass
1 week
750 DOT

4. Deploy, benchmark and optimise Live on DevNet with monitoring
1 week
750 DOT

Risks and Mitigation Risk Mitigation Strategy

• QMDB may not be production-ready
• Extensive DevNet testing before MainNet deployment
• Substrate’s evolving architecture may cause compatibility issues
• Freeze the Substrate version until integration is stable
• Increased resource consumption due to indexers
• Optimise RAM/disk usage and implement caching mechanisms

Resources Required

1. Senior Rust Developer: Expertise in Substrate and low-level systems
2. Senior Rust Developer: Specializing in database architecture
3. DevNet Environment: Dedicated nodes for rigorous testing

Success Criteria

• Substrate nodes use QMDB for transaction storage
• All Substrate and pallet-specific tests pass successfully
• Performance benchmarks demonstrate improved transaction processing times
• DevNet deployment runs stably with real-time monitoring and alerting

Payment Conditions

Payments will be made in instalments based on milestone completion. Each instalment will be subject to the successful delivery of agreed milestones. A detailed progress report will be submitted on Polkassembly/Subsquare.

Why Polkadot Network?

Polkadot’s focus on modular and scalable blockchain infrastructure makes it an ideal candidate for advanced storage optimisations like QMDB integration. By enhancing storage efficiency, this proposal aligns with Polkadot’s long-term vision of high-performance, decentralized systems.

Additional References
QMDB Technical Paper
QMDB GitHub Repository