Using Delv To Debug TDD Tesing In Golang - BTC ledger crawler program
Today let's introducing the delve usage during Gopher's daily development. I will show you a sample to demonstrate how I persist BTC block data to DB.
Context
Blockchain industry from backend technical point of view, scanner or crawler of the block is necessary --- Structuring to display things like blockchain explorer or analysis things like cryptocurrency fraud or any others Defi products.
Many blockchain developers are struggling to find an effective solution to monitor on-chain data.
Many firms of wallets/exchanges would host/develop an in-house program to track blockchain latest ledger definitely, nevertheless I wanna said there is not a completely open-source, crawling ledger quickly within a day.
Investigating an event-sourcing method to mirror BTC ledger to Postgres database. Once it keeps up with the best block, dump it to a graph database like neo4j to do analysis and query stuff.
Sample & delv debug usage
Added block projector and test case code snippet on gist, please check it out first. Then getting start from here to demostrate how to debug via delv. It's heavily behavior when I am writing golang code.
When you obtain more detail from the snippet. let's debug TestBlockGenerated.
Quickly overview of dlv help:
➜ utxo-cqrs git:(develop) cd app/service/block_parser/pkg/repository
➜ repository git:(develop) dlv test ./
Type 'help' for list of commands.
(dlv) help
The following commands are available:
Running the program:
call ------------------------ Resumes process, injecting a function call (EXPERIMENTAL!!!)
continue (alias: c) --------- Run until breakpoint or program termination.
next (alias: n) ------------- Step over to next source line.
rebuild --------------------- Rebuild the target executable and restarts it. It does not work if the executable was not built by delve.
restart (alias: r) ---------- Restart process.
step (alias: s) ------------- Single step through program.
step-instruction (alias: si) Single step a single cpu instruction.
stepout (alias: so) --------- Step out of the current function.
Manipulating breakpoints:
break (alias: b) ------- Sets a breakpoint.
breakpoints (alias: bp) Print out info for active breakpoints.
clear ------------------ Deletes breakpoint.
clearall --------------- Deletes multiple breakpoints.
condition (alias: cond) Set breakpoint condition.
on --------------------- Executes a command when a breakpoint is hit.
trace (alias: t) ------- Set tracepoint.
Viewing program variables and memory:
args ----------------- Print function arguments.
display -------------- Print value of an expression every time the program stops.
examinemem (alias: x) Examine memory:
locals --------------- Print local variables.
print (alias: p) ----- Evaluate an expression.
regs ----------------- Print contents of CPU registers.
set ------------------ Changes the value of a variable.
vars ----------------- Print package variables.
whatis --------------- Prints type of an expression.
Listing and switching between threads and goroutines:
goroutine (alias: gr) -- Shows or changes current goroutine
goroutines (alias: grs) List program goroutines.
thread (alias: tr) ----- Switch to the specified thread.
threads ---------------- Print out info for every traced thread.
Viewing the call stack and selecting frames:
deferred --------- Executes command in the context of a deferred call.
down ------------- Move the current frame down.
frame ------------ Set the current frame, or execute command on a different frame.
stack (alias: bt) Print stack trace.
up --------------- Move the current frame up.
Other commands:
config --------------------- Changes configuration parameters.
disassemble (alias: disass) Disassembler.
edit (alias: ed) ----------- Open where you are in $DELVE_EDITOR or $EDITOR
exit (alias: quit | q) ----- Exit the debugger.
funcs ---------------------- Print list of functions.
help (alias: h) ------------ Prints the help message.
libraries ------------------ List loaded dynamic libraries
list (alias: ls | l) ------- Show source code.
source --------------------- Executes a file containing a list of delve commands
sources -------------------- Print list of source files.
types ---------------------- Print list of types
Type help followed by a command for full documentation.
Notes: It's not baby tutorial, the post would not get involved in how to install dlv.
Unlike script program language, compilation language debug during development isn't a friendly thing. When I writing Ruby code, byebug is a very comfortable thing to interact on console during TDD process: Running unit test, byebug as breakpoint interruping and verifying program logic.
➜ repository git:(develop) dlv test ./
Type 'help' for list of commands.
(dlv) b TestBlockGenerated
Breakpoint 1 set at 0x19672db for github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*BlockProjectorTestSuite).TestBlockGenerated() ./block_projector_suite_test.go:17
(dlv) c
2021/03/12 16:56:30 WARNING: proto: file "block.proto" is already registered
A future release will panic on registration conflicts. See:
https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict
2021/03/12 16:56:32 Established connection to RPC server localhost:8431
2021/03/12 16:56:32 Starting RPC client localhost:8431
> github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*BlockProjectorTestSuite).TestBlockGenerated() ./block_projector_suite_test.go:17 (hits goroutine(118):1 total:1) (PC: 0x19672db)
12:
13: type BlockProjectorTestSuite struct {
14: RepoSuite
15: }
16:
=> 17: func (s *BlockProjectorTestSuite) TestBlockGenerated() {
18: cases := []struct {
19: BlockCmd *rpcproto.GetBlockVerboseTxResult
20: constraintEvent bool
21: err error
22: }{
(dlv) b 59
Breakpoint 2 set at 0x196799c for github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*BlockProjectorTestSuite).TestBlockGenerated() ./block_projector_suite_test.go:59
(dlv) c
> github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*BlockProjectorTestSuite).TestBlockGenerated() ./block_projector_suite_test.go:59 (hits goroutine(118):1 total:1) (PC: 0x196799c)
54: false,
55: BlockExistErr{Msg: "block exist: 0000000000000000000aafc247e4f795e1de32f20356bac70ce39e7eb9d53ae1"},
56: },
57: }
58:
=> 59: for _, tc := range cases {
60: err := s.repository.BlockGenerated(context.Background(), tc.BlockCmd, tc.constraintEvent)
61: if tc.err == nil {
62: s.Assert().NoError(err)
63: } else {
64: s.Assert().EqualError(tc.err, err.Error())
(dlv) n
> github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*BlockProjectorTestSuite).TestBlockGenerated() ./block_projector_suite_test.go:60 (PC: 0x1967a40)
55: BlockExistErr{Msg: "block exist: 0000000000000000000aafc247e4f795e1de32f20356bac70ce39e7eb9d53ae1"},
56: },
57: }
58:
59: for _, tc := range cases {
=> 60: err := s.repository.BlockGenerated(context.Background(), tc.BlockCmd, tc.constraintEvent)
61: if tc.err == nil {
62: s.Assert().NoError(err)
63: } else {
64: s.Assert().EqualError(tc.err, err.Error())
65: }
(dlv) s
> context.Background() /usr/local/Cellar/go/1.16/libexec/src/context/context.go:208 (PC: 0x11641c0)
203:
204: // Background returns a non-nil, empty Context. It is never canceled, has no
205: // values, and has no deadline. It is typically used by the main function,
206: // initialization, and tests, and as the top-level Context for incoming
207: // requests.
=> 208: func Background() Context {
209: return background
210: }
211:
212: // TODO returns a non-nil, empty Context. Code should use context.TODO when
213: // it's unclear which Context to use or it is not yet available (because the
(dlv) stepout
> github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*BlockProjectorTestSuite).TestBlockGenerated() ./block_projector_suite_test.go:60 (PC: 0x1967a45)
Values returned:
~r0: context.Context(*context.emptyCtx) *0
55: BlockExistErr{Msg: "block exist: 0000000000000000000aafc247e4f795e1de32f20356bac70ce39e7eb9d53ae1"},
56: },
57: }
58:
59: for _, tc := range cases {
=> 60: err := s.repository.BlockGenerated(context.Background(), tc.BlockCmd, tc.constraintEvent)
61: if tc.err == nil {
62: s.Assert().NoError(err)
63: } else {
64: s.Assert().EqualError(tc.err, err.Error())
65: }
(dlv) s
> github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*repo).BlockGenerated() ./block_projector.go:26 (PC: 0x19628bb)
21:
22: func (err BlockExistErr) Error() string {
23: return err.Msg
24: }
25:
=> 26: func (repo *repo) BlockGenerated(ctx context.Context, event *rpcproto.GetBlockVerboseTxResult, constraintEvent bool) error {
27: conn := repo.db.Conn()
28: defer conn.Close()
29:
30: // query with block hash and heigh
31: // block-parser=# explain (analyze,verbose,timing,costs,buffers)select count(*) from events where event_json @> '{"hash":"0000000033a27c42a6d51c428de207c78007492be46d5488e636121e84161a18", "height":"5247"}';
(dlv) b 67
Breakpoint 3 set at 0x19632f5 for github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*repo).BlockGenerated() ./block_projector.go:67
(dlv) c
> github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*repo).BlockGenerated() ./block_projector.go:67 (hits goroutine(118):1 total:1) (PC: 0x19632f5)
62: eventMeta, err := pgjson.Marshal(&txsWithHex)
63: if err != nil {
64: return errors.New("fail to marshal block tx: " + err.Error())
65: }
66:
=> 67: event.Tx = nil
68: eventJSON, err := pgjson.Marshal(&event)
69: if err != nil {
70: return errors.New("fail to marshal block payload: " + err.Error())
71: }
72:
(dlv) p txsWithHex
[]github.com/wenweih/bitcoin-rpc-golang/proto.Transaction len: 2, cap: 2, [
{
state: (*"google.golang.org/protobuf/internal/impl.MessageState")(0xc0001cc140),
sizeCache: 0,
unknownFields: []uint8 len: 0, cap: 0, nil,
Txid: "",
Hash: "",
Version: 0,
Size: 0,
Vsize: 0,
Weight: 0,
Locktime: 0,
Vin: []*github.com/wenweih/bitcoin-rpc-golang/proto.Vin len: 0, cap: 0, nil,
Vout: []*github.com/wenweih/bitcoin-rpc-golang/proto.Vout len: 0, cap: 0, nil,
Hex: "0100000000010100000000000000000000000000000000000000000000000000...+690 more",},
{
state: (*"google.golang.org/protobuf/internal/impl.MessageState")(0xc0001cc1e0),
sizeCache: 0,
unknownFields: []uint8 len: 0, cap: 0, nil,
Txid: "",
Hash: "",
Version: 0,
Size: 0,
Vsize: 0,
Weight: 0,
Locktime: 0,
Vin: []*github.com/wenweih/bitcoin-rpc-golang/proto.Vin len: 0, cap: 0, nil,
Vout: []*github.com/wenweih/bitcoin-rpc-golang/proto.Vout len: 0, cap: 0, nil,
Hex: "01000000025867051e9262a2191c6627cca7c65a44a1b9080156af96b1f6eaa3...+680 more",},
]
(dlv) c
> github.com/wenweih/utxo-cqrs/app/service/block_parser/pkg/repository.(*repo).BlockGenerated() ./block_projector.go:67 (hits goroutine(118):2 total:2) (PC: 0x19632f5)
62: eventMeta, err := pgjson.Marshal(&txsWithHex)
63: if err != nil {
64: return errors.New("fail to marshal block tx: " + err.Error())
65: }
66:
=> 67: event.Tx = nil
68: eventJSON, err := pgjson.Marshal(&event)
69: if err != nil {
70: return errors.New("fail to marshal block payload: " + err.Error())
71: }
So far it ends for the sharing. Hope you guys have a good weekend, happy Friday!