Support parsing Go 1.17 stack traces Fixes #61. In Go 1.17, the format of function arguments printed in stack traces was improved. This is mentioned in the release notes: https://tip.golang.org/doc/go1.17#compiler > The format of stack traces from the runtime (printed when an uncaught > panic occurs, or when runtime.Stack is called) is improved. Previously, > the function arguments were printed as hexadecimal words based on the > memory layout. Now each argument in the source code is printed > separately, separated by commas. Aggregate-typed (struct, array, string, > slice, interface, and complex) arguments are delimited by curly braces. > A caveat is that the value of an argument that only lives in a register > and is not stored to memory may be inaccurate. Function return values > (which were usually inaccurate) are no longer printed. This runtime change was primarily performed in https://go-review.googlesource.com/c/go/+/304470. The description on that CL presents the change in greater detail. This commit adds support to panicparse for parsing these new aggregate-typed arguments. It does so while retaining the type structure from the stack traces, which allows for the structure to be faithfully recreated and for aggregate-type information to be accurately augmented, when available. The aggregate-type structure is stored in a tree of mutually recursive `Args` and `Arg` structs. The commit also adds support for handling the new "offset too large" operator that was added in the same runtime CL. This operator comes in the form of an "_" scalar argument in cases where the argument's frame offset was too large (>= 0xf0), preventing the argument from being printed in the stack trace. This change raises questions about backwards compatability in a few different areas. - this change does not break backwards compatability with pre-1.17 stack traces. However, due to the nature of how most of the unit tests are set up (i.e. shelling out to local go binary), we are losing some test coverage of pre-1.17 stack trace parsing. Is that ok? - similarly, this change breaks the library's tests for pre-1.17 installations of Go. As a result, it bumps the Go version run in CI. Is that ok? - this change will not cause users of the `github.com/maruel/panicparse/v2/stack` library to fail to compile. However, due to the additions to the `Arg` struct, users of the library will fail to observe all arguments when parsing post-1.17 stack traces (but not pre-1.17 stack traces) unless they are aware of the new `IsAggregate` field. Is that ok? The change has the following effect on microbenchmarks: ``` name old time/op new time/op delta ScanSnapshot_Passthru-16 2.45ns ± 3% 2.42ns ± 2% ~ (p=0.094 n=9+9) Aggregated_ToHTML-16 5.81ms ± 1% 5.87ms ± 3% ~ (p=0.079 n=10+9) ScanSnapshot_NoGuess-16 2.36ms ± 3% 2.52ms ± 1% +7.10% (p=0.000 n=10+9) ScanSnapshot_Guess-16 2.35ms ± 4% 2.53ms ± 1% +7.52% (p=0.000 n=9+10) Aggregate-16 139µs ± 2% 195µs ± 3% +40.14% (p=0.000 n=9+10) name old alloc/op new alloc/op delta Aggregated_ToHTML-16 1.17MB ± 0% 1.17MB ± 0% -0.18% (p=0.010 n=9+10) ScanSnapshot_Passthru-16 0.00B 0.00B ~ (all equal) ScanSnapshot_NoGuess-16 730kB ± 0% 1076kB ± 0% +47.39% (p=0.000 n=10+10) ScanSnapshot_Guess-16 730kB ± 0% 1076kB ± 0% +47.39% (p=0.000 n=10+10) Aggregate-16 143kB ± 0% 254kB ± 0% +77.81% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Aggregate-16 565 ± 0% 565 ± 0% ~ (all equal) ScanSnapshot_Passthru-16 0.00 0.00 ~ (all equal) Aggregated_ToHTML-16 41.0k ± 0% 41.1k ± 0% +0.16% (p=0.000 n=10+8) ScanSnapshot_Guess-16 8.73k ± 0% 9.39k ± 0% +7.46% (p=0.000 n=10+9) ScanSnapshot_NoGuess-16 8.73k ± 0% 9.39k ± 0% +7.46% (p=0.000 n=9+10) ```
Parses panic stack traces, densifies and deduplicates goroutines with similar stack traces. Helps debugging crashes and deadlocks in heavily parallelized process.
panicparse helps make sense of Go crash dumps:

See v2.0.1 blog post.

panicparse was created with ❤️️ and passion by Marc-Antoine Ruel and friends.
go get github.com/maruel/panicparse/v2/cmd/pp
|&|&2>&1 |^|pp streams its stdin to stdout as long as it doesn‘t detect any panic. panic() and Go’s native deadlock detector print to stderr via the native print() function.
Bash v4 or zsh: |& tells the shell to redirect stderr to stdout, it's an alias for 2>&1 | (bash v4, zsh):
go test -v |&pp
Windows or macOS native bash (which is 3.2.57): They don't have this shortcut, so use the long form:
go test -v 2>&1 | pp
Fish: &| redirects stderr and stdout. It's an alias for 2>&1 | (fish piping):
go test -v &| pp
PowerShell: It has broken 2>&1 redirection. The workaround is to shell out to cmd.exe. :(
On POSIX, use Ctrl-\ to send SIGQUIT to your process, pp will ignore the signal and will parse the stack trace.
To dump to a file then parse, pass the file path of a stack trace
go test 2> stack.txt pp stack.txt
Starting with go1.11, the toolchain starts to inline more often. This causes traces to be less informative. Starting with go1.17, optimization also interfere with traces. You can use the following to help diagnosing issues:
go install -gcflags '-N -l' path/to/foo foo |& pp
or
go test -gcflags '-N -l' ./... |& pp
Run go tool compile -help to get the full list of valid values for -gcflags.
Starting with Go 1.6, GOTRACEBACK defaults to single instead of all / 1 that was used in 1.5 and before. To get all goroutines trace and not just the crashing one, set the environment variable:
export GOTRACEBACK=all
or set GOTRACEBACK=all on Windows. Probably worth to put it in your .bashrc.
Install bash v4+ on macOS via homebrew or macports. Your future self will appreciate having done that.
/usr/bin/pp installedIf you try pp for the first time and you get:
Creating tables and indexes... Done.
and/or
/usr/bin/pp5.18: No input files specified
you may be running the Perl PAR Packager instead of panicparse.
You have two choices, either you put $GOPATH/bin at the beginning of $PATH or use long name panicparse with:
go get github.com/maruel/panicparse/v2
then using panicparse instead of pp:
go test 2> panicparse