Proposal summary

@SIM-DOT

All of these will need to be tested on a real data. However, rough estimates could be:

1. Size of archive storage ~-50% (removing Merkle trie for the old states + shrinking or even removing block bodies)
2. Historical state queries should be much faster (important for indexers such as SubQuery). Instead of going through Merkle trie (log(N) reads, where N – size of a state), there will be a constant number of reads (ideally just one)
3. New erasure coding performance estimates could be found here . However, we will introduce some more low level optimizations (reducing number of redundant memory allocations) to bring these numbers even lower. In the past our Erasure coding implementation was the fastest in Polkadot (link)
4. State sync improvements are actually fixing some existing issues like OOM crash for large state syncs, which will become a more important problem as we have larger Parachain states. We will be also introducing a new state sync, which should have less duplicate information sent over the network and should theoretically be faster, but not clear yet how much. I would say something around 20% maybe. With large storage states this however may be a significant gain in terms of user experience
5. For Multi-slot collation it is hard to define particular metric, however, what we will get is that malicious collators won't be able to censor blocks they don't want. With minimal relay and moving a lot of Polkadot Relay Chain logic to Parachains this will become more important
6. For Discrete Event Simulator research it is also hard to define a metric. What we will get is reproducible tests. Which could be useful for thorough bug reproduction and debugging. E.g. if we had 1000 test runs, and only one failed, we will be easily reproduce this particular test run.

@polkaworld

1

Some of this functionality might not be visible to end users, however it contributes to the overall robustness of the system and improves development experience:

• Shadow research: this research would be useful for creating a more robust testing framework. In simple words, current zombienet tests (which is the framework used for testing different edge case scenarios) have the following limitations:
  • Each test scenario depends on many random factors, which makes it hard to reproduce the same test scenario. E.g. if only one test out of 1000 runs fails, it is impossible to reproduce the exact scenario that caused the failure.
  • It is impossible to test behavior of the network in the presence of nodes with bad networking connectivity
    With Shadow integration, it would be possible to have deterministic test scenarios, which would allow to reproduce the same test scenario and debug it. It would also allow to test behavior of the network in the presence of nodes with bad networking connectivity. Parity devs got very excited, when we explained them what we can do with Shadow integration: potentially we would be able to run scenarios that were previously impossible to test, and potentially this could help prevent issues that network experienced in the past
• State archiving: with async backing and elastic scaling transactions throughput and load on the Polkadot will be increased significantly, which would in turn increase the size of the state and make it harder to maintain, leading to higher costs for archive node RPC validators (who would then in turn come to treasury for more compensation). For example today wallet developers already cannot afford to run their own archive nodes, relying on third parties, which introduces a single point of failure and more trust assumptions. We need cheaper archive nodes and this requires some deep engineering work:
  • Getting rid of the Merkle trie for the obsolete states that currently might consume up to 50% of the state size
  • Implementing a new state format that would allow to store the state in a more compact way (proposal has more details on that)
  • Compressing or getting rid of block bodies that consume ~17% of the state size in some of our nodes
  • Cool side effect of these improvements is that querying historical data becomes multiple times faster, which is important for indexing tools such as Subquery that wallets heavily rely on
• Project management: I am as project manager constantly monitoring progress of all the tasks, testing them, in many cases contributing to the codebase, keeping track of the latest developments in the ecosystem, coordinating with Parity and W3F. That's a lot of work that is done in addition to CEO responsibilities, and engineering tasks. So I believe dedicating 100 hours a month for that is a fair (optimistic) estimate

2

• Compliance expenses: we have to pay for the legal services to ensure that we are compliant with the local laws and regulations. We have entities in Singapore and Kazakhstan, and we have to ensure that we are compliant with the local laws in both countries.
• Tooling fees: we use various tools such as Google Cloud Platform (most of the costs), Netcup, Notion, Cursor, Zoom, and others to ensure that we can work efficiently. Running Polkadot nodes that consume a lot of traffic and maintaining development infrastructure such as CI/CD pipelines, logging systems, monitoring systems is expensive

3

• We kept our hourly rate at $100/hour for 3 years (starting from this old proposal).
• We have 1 Engineering manager ($10k/month), 4 Protocol engineers ($4-9k/month), 1 DevOps ($8k/month), 1 Administrator ($4k/month). Salaries are more than fair according to https://web3.career/web3-salaries
• We choose quality over quantity. We could hire more people at a lower rate, but we would not be able to deliver the same quality of work. The people that work in our team are working in the ecosystem for many years (between 3 to 8 years) and capable of delivering high quality work

Happy to elaborate more if needed