Indexers
Throughout this section we have built applications that don’t depend on an index of the blockchain. That doesn’t mean indexers are useless. There are things only an indexer can tell us, and in this section we will look at when to reach for one, and which one to reach for.
Before you reach for an indexer
Section titled “Before you reach for an indexer”It is worth asking a few questions first, because most of the time the Transaction we are looking for can come from somewhere else:
| If the Transaction is… | …then get it from |
|---|---|
| one our own wallet built | our wallet, which already holds it |
| a payment to us | the payer, who hands it to us as a BEEF (Paying peer to peer) |
| part of a domain’s shared state (a token, an advertisement) | the domain’s overlay network (Overlay networks) |
| one we broadcast, and we need its merkle proof | the broadcaster we submitted it through (Broadcasting and merkle proofs) |
What remains are the cases where nobody we deal with holds the Transactions we need:
- Importing a wallet that was built elsewhere, whose history exists only on the blockchain.
- Following activity we are not party to, e.g. analytics, a block explorer, or monitoring a protocol across the whole chain.
These are the jobs for an indexer.
Do note that even an indexer’s answer can be checked. When we fetch a Transaction from an indexer, we fetch its merkle proof too, and check it against our own header chain before we rely on it.
JungleBus
Section titled “JungleBus”JungleBus, from GorillaPool, is the better general-purpose indexer. Rather than answering questions one at a time, it lets an application subscribe to the Transactions it is interested in, and streams them to the application as they arrive, starting from a block height of the application’s choosing. This suits the jobs indexers are still good for, such as following a protocol’s Transactions across the whole chain.
WhatsOnChain
Section titled “WhatsOnChain”WhatsOnChain has served the community well as a block explorer and a REST API for addresses, Transactions and merkle proofs. We now consider it deprecated as a foundation for applications, for the reasons given in the Overview. It is still useful:
- as a block explorer, for people,
- for importing a wallet: libspiffy uses it at the time of writing to discover a wallet’s used Addresses and fetch their Transactions with proofs, as described in Wallets that keep their own history.
WhatsOnChain limits the rate of requests, and libspiffy keeps to three requests per second by default. Importing a wallet with a long history can therefore take a few minutes.
Running your own node
Section titled “Running your own node”In 2020 we spent a good deal of this section on running your own Bitcoin Node and talking to it over JSON-RPC and ZeroMQ. An index-free application doesn’t need a node of its own. The network’s node software, Teranode, is built to process Transactions at scale, not to serve an application’s queries.
A node is still very useful as a development tool. A local network in regtest mode, with a local Arcade in front of it, lets us mine blocks on demand and test every step of a payment without spending real coins.
- libspiffy: Local network shows how to run one for libspiffy applications.
- Our RegTest Teranode tutorial walks through running a regtest Teranode, with Arcade, Merkle Service and a faucet, on your own machine.