Event Sourcing

Let’s take a quick detour to talk about an architectural pattern that is generally useful for working with immutable databases like the blockchain.

In a traditional system, to mutate the state of an Entity/Domain Object one would affect that change directly, overwriting the previous state. Within an Event Sourcing model, the state of an Entity/Domain Object’s mutations are recorded as a series of state-change events. To know with absolute certainty what the current state of the system is, one would simply replay all the individual state-change events for a particular Entity to arrive at the current “canonical state”.

This architectural pattern was described in a 2005 blog post by Martin Fowler entitled Event Sourcing - Capture all changes to an application state as a sequence of events.

One of the consequences of persisting application state in this manner, is the creation of an Append-Only Database.

Bitcoin is an Event Sourcing system. Each Transaction records a state-change event for a specific UTXO, essentially appending a new record to the “Bitcoin Database”. One can check the validity of any UTXO that one is given by retracing and replaying all the state-change events (spending history) against that UTXO back to the originating “coinbase transaction” that released the bitcoins into the system.

Figure 1: Event Sourcing within Bitcoin

There exists a very particular architectural pattern known as CQRS (Command/Query Responsibility Separation). This refers to a method of structuring an application in such a way that there exists clear boundaries between the parts of the application that mutates the application’s persistent state (database) and the part that retrieves the current view of persistent state(database).

Figure 1 shows how we first applied CQRS to Bitcoin, in 2020. The Bitcoin Node held the event store, i.e. the blockchain, and our application sent it commands over JSON-RPC, subscribed to its events over ZeroMQ, and sent it queries too. This was not quite ideal. Our application was tied to a node it had to run itself, and it asked that node about Transactions that had nothing to do with it.

Do note that it is possible to have an event-sourced system without CQRS, but in the wild the two are usually found in each other’s company.

Event Sourcing in the wallet

A full-history wallet, as described in Wallets that keep their own history, applies the same pattern to its own Transactions. Instead of asking a node for the state of its coins, the wallet keeps its own event store, holding only the events that concern it (Figure 2).

Figure 2: Event Sourcing and CQRS inside a full-history wallet

Referring to Figure 2, we notice a few interesting things:

1. Every change to the wallet starts as a command: our application asks it to create an invoice or pay one, a counterparty hands it a BEEF, or Arcade reports that a Transaction was mined.
2. The wallet aggregate checks each command against the wallet’s current state, and records what happened as an event in its event store, e.g. “payment received” or “Transaction confirmed”. The event store is append-only, just like the blockchain.
3. Projections replay those events into read models: the balance, the list of coins, the history.
4. Our application only ever queries the read models. Queries never touch the event store, and they never touch the network.

The wallet can rebuild its read models at any time by replaying its events, just as one can check any UTXO by replaying its spending history back to the coinbase. And because the event store holds the wallet’s whole history, it is also what a backup has to carry.

This is exactly how libspiffy is built: each wallet is an event-sourced aggregate, with projections that keep its read models up to date.