Running Your Own Node
When building an application, the primary way of interacting with the Bitcoin network will be through a node that acts as an “edge router”. This “edge router” speaks the wire-level protocol of the Bitcoin network on one end, and exposes application-friendly APIs on the other. While it might be the case that for most uses the running of one’s own “edge router” or “listening node” is not desirable or required, there do exist integration scenarios where this would be the preferred way of interacting with the network. The decision will be left up to the individual developer.
The Bitcoin Client Node can act as an “edge router” for your application.
Architecture
Figure 1: Connecting to a Node using Dart
Our Dart application architecture for directly connecting to a Bitcoin Node has hit a snag.
There are unfortunately, at the time of writing, no Dart bindings for ZeroMQ available. The Dart FFI (Foreign Function Interface) is in Beta as of Dart V2.7.x. It is hoped that in future a FFI C-language binding to libzmq will emerge for us to use. In the event that one would like to listen to and respond to ZeroMQ notifications, and alternative solution to a Dart server-side application should be considered. We will make up for this shortcoming with different architectures in subsequent chapters.
While we will discuss the proper configuration and setup of a Bitcoin Node for zeroMQ, we will be unable to build a Dart code example in this chapter.
Setup / Installation
You should be familiar by now with operating a local running instance of the Bitcoin Client Node, including how to configure it. Refer to the section on Getting Started for a refresher.
Connect
RPC Interface
Configuration
In general, you should already have the JSON-RPC configuration setup, since that is the interface that the bitcoin-cli program uses to interact with the Bitcoin Client Node. The specific sections of the configuration file that deal with the RPC interface are:
# Server setting to turn on RPC (required for bitcoin-cli to work)
server=1
rpcbind=127.0.0.1
rpcport=18332
whitelist=127.0.0.1
rpcworkqueue=128
rpcthreads=128
rpctimeout=220
JSON-RPC Config Settings
This is the list of settings that control the RPC server. These settings should be added/modified inside the bitcoind.conf configuration file.
server
Accept command line and JSON-RPC commands
rest
Accept public REST requests (default: 0)
rpcbind=<addr>
Bind to given address to listen for JSON-RPC connections. Use
[host]:port notation for IPv6. This option can be specified
multiple times (default: bind to all interfaces)
rpccookiefile=<loc>
Location of the auth cookie (default: data dir)
rpcuser=<user>
Username for JSON-RPC connections
rpcpassword=<pw>
Password for JSON-RPC connections
rpcauth=<userpw>
Username and hashed password for JSON-RPC connections. The field
<userpw> comes in the format: <USERNAME>:<SALT>$<HASH>. A
canonical python script is included in share/rpcuser. The client
then connects normally using the
rpcuser=<USERNAME>/rpcpassword=<PASSWORD> pair of arguments. This
option can be specified multiple times
rpcport=<port>
Listen for JSON-RPC connections on <port> (default: 8332 or testnet:
18332)
rpcallowip=<ip>
Allow JSON-RPC connections from specified source. Valid for <ip> are a
single IP (e.g. 1.2.3.4), a network/netmask (e.g.
1.2.3.4/255.255.255.0) or a network/CIDR (e.g. 1.2.3.4/24). This
option can be specified multiple times
JSON-RPC via the CLI
The RPC cookie file contains a username and password combination that is formatted as : “[username]:[password]”. In REGTEST mode you can find the cookie file in $BITCOIN_HOME/data/regtest/.cookie.
cat $BITCOIN_HOME/data/regtest/.cookie
__cookie__:9612263a3c3e8dbe0fedc143baa7717ca56f2888cea9c8c24bc1dfa956bfbf63
There are quite a lot of RPC commands. The best current way of getting to know these would be to interrogate the bitcoin-cli program directly. We can start by asking for the list of RPC commands that are supported. Please substitute user and password as appropriate for your own installation.
bitcoin-cli -conf=data/bitcoind.conf -rpcuser=<user> -rpcpassword=<rpcpassword> -rpcport=18332 help
To get information on a specific command, including a helpful curl example
bitcoin-cli -conf=data/bitcoind.conf -rpcuser=<user> -rpcpassword=<rpcpassword> -rpcport=18332 getwalletinfo help
.
.
.
Examples:
> bitcoin-cli getwalletinfo
> curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getwalletinfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/
Substituting for our actual credentials from the .cookie file, and piping through the jq command to render our json nicely we get
curl --user __cookie__:9612263a3c3e8dbe0fedc143baa7717ca56f2888cea9c8c24bc1dfa956bfbf63 --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getwalletinfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 422 100 345 100 77 26538 5923 --:--:-- --:--:-- --:--:-- 30142
{
"result": {
"walletname": "wallet.dat",
"walletversion": 160300,
"balance": 0,
"unconfirmed_balance": 0,
"immature_balance": 0,
"txcount": 0,
"keypoololdest": 1581753212,
"keypoolsize": 1000,
"keypoolsize_hd_internal": 1000,
"paytxfee": 0,
"hdmasterkeyid": "c6c336722ac7101991837673cd94068a40cee46c"
},
"error": null,
"id": "curltest"
}
Notifications
ZeroMQ is not enabled within the node by default. It has to be explicitly enabled through configuration parameters in data/bitcoind.conf. The following configuration settings apply to zeroMQ:
# The address bitcoind will listen for new ZeroMQ connection requests.
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubhashtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
There is unfortunately no commandline utility that allows us to receive ZeroMQ notifications. Consider this information a placeholder for now. We will make use of it in a later chapter.
Code Example
JSON-RPC
Making a JSON-RPC call is rather simple. We can simply translate the curl example we are given by the bitcoin-cli command’s help system into the appropriate HTTP client code within Dart. Note that you will need to adapt the following code to account for your own node’s username and password as found in the .cookie file.
/*
Use JSON-RPC to retrieve local test node's wallet information
*/
import 'dart:convert';
import 'package:dartsv/dartsv.dart';
import 'package:http/http.dart' as http;
Future<String> getWalletInfo() async{
var url = 'http://localhost:18332';
var client = http.Client();
var username = '__cookie__';
var password = '9612263a3c3e8dbe0fedc143baa7717ca56f2888cea9c8c24bc1dfa956bfbf63';
var authHeader = base64.encode(utf8.encode('${username}:${password}'));
var bodyParams = jsonEncode({
"jsonrpc": "1.0",
"id":"twostack",
"method": "getwalletinfo",
"params": []
});
var response = await client.post(url, headers: {
'Authorization' : 'Basic ${authHeader}'
}, body: bodyParams);
print(response.statusCode);
print(response.body.toString());
return response.body.toString();
}
Let’s do something a little more ambitious. Let’s reuse our code from when we created a P2PKH transaction earlier. If you recall, we had to copy-paste the raw transaction into our code. Let’s update our code, so that instead of taking a raw transaction, it will instead use a Transaction ID.
- Generate a coinbase output with some fake “mining”, and send some of those coins to our local node’s bitcoin wallet. When we run the sendtoaddress command a Transaction ID will be printed on the commandline. We’ll grab that.
bitcoin-cli -conf=data/bitcoind.conf -rpcuser=__cookie__ -rpcpassword=9612263a3c3e8dbe0fedc143baa7717ca56f2888cea9c8c24bc1dfa956bfbf63 -rpcport=18332 generate 101
[
.
.
"7ed1eedfbc33c4defcd3a62646510b037b83de529a57be96083f968b4ad34058"
]
bitcoin-cli -datadir=bin/bitcoin-sv-1.0.0/data sendtoaddress mz7ZQ1XWBc65GrgpsG3djwd3cJFyaksKdz 100
564499f42ba76bb2b19f12c1e4292bc0eebd1e024ab5653c9889c59b07fa2286 <--- Transaction ID to Grab
- Now we want to get the appropriate curl command that corresponds to the “getrawtransaction” JSON-RPC call.
----
Let's get the appropriate curl command to
bitcoin-cli -conf=data/bitcoind.conf -rpcuser=__cookie__ -rpcpassword=9612263a3c3e8dbe0fedc143baa7717ca56f2888cea9c8c24bc1dfa956bfbf63 -rpcport=18332 getrawtransaction
.
.
.
Examples:
> bitcoin-cli getrawtransaction "mytxid"
> bitcoin-cli getrawtransaction "mytxid" true
> curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getrawtransaction", "params": ["mytxid", true] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/
And as a last step we combine the two above bits of information, and write the code to create a spending transaction just like we did before.
/*
Use JSON-RPC to retrieve local test node's wallet information
*/
import 'dart:convert';
import 'package:dartsv/dartsv.dart';
import 'package:http/http.dart' as http;
Future<String> getRawTransaction(String transactionId) async{
var url = 'http://localhost:18332';
var client = http.Client();
var username = '__cookie__';
var password = '9612263a3c3e8dbe0fedc143baa7717ca56f2888cea9c8c24bc1dfa956bfbf63';
var authHeader = base64.encode(utf8.encode('${username}:${password}'));
var bodyParams = jsonEncode({
"jsonrpc": "1.0",
"id":"twostack",
"method": "getrawtransaction",
"params": [transactionId, true]
});
var response = await client.post(url, headers: {
'Authorization' : 'Basic ${authHeader}'
}, body: bodyParams);
print(response.statusCode);
print(response.body.toString());
var jsonResponse = response.body.toString();
var resultMap = jsonDecode(jsonResponse) as Map<String, dynamic>;
//Error handling is left as an exercise to the reader ( *evil grin* )
return resultMap["result"]["hex"];
}
Future<String> jsonRPCWithP2PKH( ) async {
//Let's reconstruct our Private Key
var privateKey = SVPrivateKey.fromWIF("cVVvUsNHhbrgd7aW3gnuGo2qJM45LhHhTCVXrDSJDDcNGE6qmyCs");
//Creates Address : "mu4DpTaD75nheE4z5CQazqm1ivej1vzL4L"
var recipientAddress = privateKey.toAddress(); // my address
var changeAddress = Address("n3aZKucfWmXeXhX13MREQQnqNfbrWiYKtg"); //bitcoin-cli wallet address
//Create a Transaction instance from the RAW transaction data created by bitcoin-cli.
//Remember that this transaction contains the UTXO we are interested in
var rawTx = await getRawTransaction('564499f42ba76bb2b19f12c1e4292bc0eebd1e024ab5653c9889c59b07fa2286');
var txWithUTXO = Transaction.fromHex(rawTx);
//Let's create the set of Spending Transaction Inputs. These Transaction Inputs need to refer to the Outputs in
//the Transaction we are spending from.
var utxo = txWithUTXO.outputs[0]; //looking at the decoded JSON we can see that our UTXO is at vout[0]
//1. where is the money coming from - (spendFromOutput())
//2. where is the money going to - (spendTo())
//3. where should we send what's left over as difference between (1) and (2) - (sendChangeTo())
//4. how much are we willing to pay the miners - (withFeePerKb())
var unlockBuilder = P2PKHUnlockBuilder(privateKey.publicKey);
var transaction = new Transaction()
.spendFromOutput(utxo, Transaction.NLOCKTIME_MAX_VALUE, scriptBuilder: unlockBuilder) //implicitly creates correct Inputs in spending transaction
.spendTo(recipientAddress, BigInt.from(50000000), scriptBuilder: P2PKHLockBuilder(recipientAddress)) //spend half a bitcoin == 50 million satoshis (creates Outputs in spending transaction)
.sendChangeTo(changeAddress, scriptBuilder: P2PKHLockBuilder(changeAddress)) // spend change to myself
.withFeePerKb(1000); //how much we are willing to pay the miners
//Sign the Transaction Input
transaction.signInput(0, privateKey, sighashType: SighashType.SIGHASH_ALL | SighashType.SIGHASH_FORKID);
print(transaction.serialize());
}
Send it !
The JSON-RPC interface also contains a sendrawtransaction method. As an exercise, use the methods learnt above to send the transaction directly to the local node, rather than using bitcoin-cli.