We are excited to introduce our proposal for the Substrate Subsquid Migration to ArrowSquid & Indexing Upgrades. This project targets enhancing the performance and functionality of Subsquid, a vital component providing access to on-chain data within the Substrate ecosystem. Through this proposal, we aim to improve the speed, efficiency, and indexing capabilities of Subsquid, ultimately benefiting developers and users across various chains such as Polkadot, Kusama, Moonbeam, and Acala.

Subsquid currently provides the community with access to on-chain data via GraphQL gateways, serving as data sources for APIs and on-chain data exploration tools. The main objective of this project is to migrate the existing Substrate Subsquid ecosystem to the ArrowSquid processor as well as enhance its indexing capabilities. This migration will enhance the speed and efficiency of all 50+ public squid endpoints.

Currently, all Giant Squid Main squids utilize the previous Substrate processor, Firesquid. The recent upgrade to the ArrowSquid processor introduces significant improvements, such as the ability to ingest unfinalized blocks directly from a network node, resulting in near real-time APIs with subsecond latency. However, this upgrade to the processor resulted in breaking changes and all squids that want to use it have to be adjusted to conform to the breaking changes.

Additionally, Substrate squids currently lack support for indexing Staking and Crowdloans pallets, which contain critical information for various chains in the Substrate ecosystem, including Polkadot, Kusama, Moonbeam, and Acala. The migration will also improve the performance of Polkascan as it depends directly on the data and performance from GiantSquid

The project aims to address these challenges by:

- Migrating the existing giant-squid-main codebase to utilize the ArrowSquid processor, improving speed and efficiency across all supported chains.
- Enhancing indexing capabilities by implementing indexing of calls and events from the Staking and Crowdloans pallets, enabling access to vital on-chain information.

The project will be executed in two milestones. The first one is focused on migrating giant-squid-main to ArrowSquid, which includes upgrading the dependencies, configuration, and endpoints adaptations, as well as testing and deployment. The second milestone will be focused on indexing Staking and Crowdloan pallets calls and events.

For a detailed description of our proposal, please visit: [Polkadot Treasury - Subsquid Upgrades Proposal](https://docs.google.com/document/d/148ZnlnG7nrDRVhS-VQQ-Ld21u78qaXWqcVsV366EPn4/edit)

**Companies Background:**

Subsquid is providing the community with access to on-chain data in the form of GraphQL gateways and it is doing it completely for free for the time being. Such endpoints are being used both as data sources for complex APIs and as an on-chain data exploration tool. The purpose of the proposal is to ask for funds to cover the cost of running the infrastructure, given the utility they provide to the community.

LimeChain is a software development company founded in 2017. Positioned to handle end-to-end product development with the capability to manage projects from conceptualization and design to the complete implementation of a certain product. LimeChain specializes in blockchain technology, with an emphasis on developing blockchain-related and infrastructure solutions. In the context of the Polkadot ecosystem, the expertise is focused on developing infrastructure implementations, including [Gosemble, a framework for building Substrate compatible Runtimes in Go](https://github.com/LimeChain/gosemble), [Fruzhin, a Host implementation in Java](https://github.com/limechain/fruzhin), a framework for runtimes in AssemblyScript, a framework for runtimes in AssemblyScript. On top of that, Limechain has also implemented a [Parachain Validation Conformance Testing suite](https://github.com/LimeChain/polkadot-conformance).

Subsquid Migration to ArrowSquid & Indexing Upgrades

- **Problem Statement:** Enhancing the performance and functionality of Subsquid by migrating to ArrowSquid and improving indexing capabilities to benefit developers and users in the Substrate ecosystem.
- **Significance:** Current limitations in speed, efficiency, and indexing capabilities of Subsquid impact data access for chains like Polkadot, Kusama, Moonbeam, and Acala.
- **Proposed Solution:** Migrate giant-squid-main to ArrowSquid for speed and efficiency improvements, and enhance indexing by including Staking and Crowdloans pallets data.
- **Expected Outcomes:** Improved speed, efficiency, and indexing capabilities across supported chains, benefiting developers and users accessing on-chain data.

#2219 Subsquid Migration to ArrowSquid & Indexing Upgrades

No Active Proposals