Subscrape: Releasing v1.0 & Roadmap for v2.0

After announcing Subscrape to the world for the first time a little over a month ago, the library has grown quite a bit and today I am proud to announce v1.0 of Subscrape and present the roadmap for Subscrape 2.0

Releasing Subscrape v1.0

Subscrape is a Python scraper for Substrate chains that uses Subscan.

It is configured by a config file through which you use to tell the scraper which data to get. It is stored locally on disk and can then be queried and transformed to your needs.

Subscrape v1.0 has just been released on the ChaosDAO Github. It allows for the scraping of extrinsics on any Subscan supported chain. The Subscrape README provides the basic documentation needed to understand the mechanics. If you have additional questions, feel free to reach out via DM on Twitter: @alice_und_bob

The Roadmap to Subscrape 2.0

Subscrape is a personal project that evolves with my needs. As I see that it might serve the needs of others too, I am open sourcing it and give you a chance to influence what I do with it or for you to pick it up and continue development. Version 1.0 served the purpose of extracting crowdloan related data and further process it in a Spreadsheet.

Version 2.0 will focus on helping me do my taxes. Taxes in Substrate are a topic that is not yet deeply explored and could prove to become very difficult in the future. My country recently introduced a rather degen friendly legislation for crypto taxes, but it is still a huge challenge.

To do my taxes, I plan to introduce utilities to understand the capital flows of the following mechanics:

• Transfers in, out and between several owned accounts
• Staking rewards
• Crowdloan participation
• Airdrops/Paradrops
• Treasury tips
• XCM transfers between parachains
• DeFi
  • Swaps
  • Liquidity Provision
  • liquidity mining
• NFTs buys and sells
  • RMRK buys and sells

This does not mean that this will be full-fledged application that supports all those cases. It means I will implement just as much as needed to fulfull my tax obligations. Corners might be cut sharply.

I welcome any collaboration on this repository. I will especially share any knowledge you are interested in to get up to speed quickly. But please do not expect that feature requests might be fulfilled if they do not fit my agenda.

I post updates on the project to Twitter from time to time. You will find the latest info by searching for the hashtag #Subscrape

Feel free to retweet the announcement tweet.