Allow Sum64String and (*Digest).WriteString to be inlined Benchmarks: name old time/op new time/op delta Sum64String/4B-12 4.78ns ± 1% 3.57ns ± 4% -25.27% (p=0.000 n=8+10) Sum64String/100B-12 14.5ns ± 1% 12.9ns ± 0% -10.76% (p=0.000 n=9+10) Sum64String/4KB-12 229ns ± 0% 229ns ± 1% ~ (p=0.395 n=7+10) Sum64String/10MB-12 628µs ± 1% 630µs ± 2% ~ (p=1.000 n=9+10) DigestString/4B-12 11.4ns ± 1% 9.7ns ± 1% -14.95% (p=0.000 n=10+10) DigestString/100B-12 23.6ns ± 1% 21.3ns ± 2% -9.65% (p=0.000 n=10+10) DigestString/4KB-12 241ns ± 1% 239ns ± 0% -0.67% (p=0.001 n=10+7) DigestString/10MB-12 627µs ± 1% 628µs ± 1% ~ (p=0.631 n=10+10) name old speed new speed delta Sum64String/4B-12 837MB/s ± 1% 1124MB/s ± 2% +34.42% (p=0.000 n=10+9) Sum64String/100B-12 6.88GB/s ± 2% 7.72GB/s ± 1% +12.16% (p=0.000 n=10+10) Sum64String/4KB-12 17.5GB/s ± 0% 17.5GB/s ± 1% ~ (p=0.408 n=8+10) Sum64String/10MB-12 15.9GB/s ± 1% 15.9GB/s ± 2% ~ (p=1.000 n=9+10) DigestString/4B-12 350MB/s ± 1% 411MB/s ± 1% +17.55% (p=0.000 n=10+10) DigestString/100B-12 4.23GB/s ± 1% 4.69GB/s ± 1% +10.84% (p=0.000 n=10+9) DigestString/4KB-12 16.6GB/s ± 1% 16.7GB/s ± 0% +0.67% (p=0.001 n=10+8) DigestString/10MB-12 16.0GB/s ± 1% 15.9GB/s ± 1% ~ (p=0.631 n=10+10) And with -tags purego: name old time/op new time/op delta Sum64String/4B-12 5.57ns ± 1% 4.22ns ± 1% -24.14% (p=0.000 n=10+9) Sum64String/100B-12 16.0ns ± 1% 14.8ns ± 0% -7.27% (p=0.000 n=10+6) Sum64String/4KB-12 327ns ± 2% 325ns ± 1% ~ (p=0.050 n=10+10) Sum64String/10MB-12 866µs ± 3% 856µs ± 0% -1.05% (p=0.002 n=9+8) DigestString/4B-12 11.2ns ± 1% 10.0ns ± 1% -10.90% (p=0.000 n=10+9) DigestString/100B-12 25.5ns ± 1% 22.8ns ± 0% -10.62% (p=0.000 n=10+9) DigestString/4KB-12 342ns ± 1% 340ns ± 1% -0.56% (p=0.018 n=9+10) DigestString/10MB-12 877µs ± 1% 878µs ± 2% ~ (p=0.400 n=10+9) name old speed new speed delta Sum64String/4B-12 718MB/s ± 1% 947MB/s ± 1% +31.82% (p=0.000 n=10+9) Sum64String/100B-12 6.26GB/s ± 1% 6.75GB/s ± 1% +7.81% (p=0.000 n=10+10) Sum64String/4KB-12 12.2GB/s ± 2% 12.3GB/s ± 1% +0.70% (p=0.022 n=10+9) Sum64String/10MB-12 11.6GB/s ± 3% 11.7GB/s ± 0% +1.05% (p=0.002 n=9+8) DigestString/4B-12 357MB/s ± 1% 401MB/s ± 1% +12.32% (p=0.000 n=10+9) DigestString/100B-12 3.93GB/s ± 1% 4.40GB/s ± 0% +11.95% (p=0.000 n=10+9) DigestString/4KB-12 11.7GB/s ± 1% 11.8GB/s ± 1% +0.68% (p=0.011 n=10+10) DigestString/10MB-12 11.4GB/s ± 1% 11.4GB/s ± 2% ~ (p=0.400 n=10+9)
xxhash is a Go implementation of the 64-bit xxHash algorithm, XXH64. This is a high-quality hashing algorithm that is much faster than anything in the Go standard library.
This package provides a straightforward API:
func Sum64(b []byte) uint64
func Sum64String(s string) uint64
type Digest struct{ ... }
func New() *Digest
The Digest type implements hash.Hash64. Its key methods are:
func (*Digest) Write([]byte) (int, error) func (*Digest) WriteString(string) (int, error) func (*Digest) Sum64() uint64
This implementation provides a fast pure-Go implementation and an even faster assembly implementation for amd64.
This package is in a module and the latest code is in version 2 of the module. You need a version of Go with at least “minimal module compatibility” to use github.com/cespare/xxhash/v2:
I recommend using the latest release of Go.
Here are some quick benchmarks comparing the pure-Go and assembly implementations of Sum64.
| input size | purego | asm |
|---|---|---|
| 5 B | 979.66 MB/s | 1291.17 MB/s |
| 100 B | 7475.26 MB/s | 7973.40 MB/s |
| 4 KB | 17573.46 MB/s | 17602.65 MB/s |
| 10 MB | 17131.46 MB/s | 17142.16 MB/s |
These numbers were generated on Ubuntu 18.04 with an Intel i7-8700K CPU using the following commands under Go 1.11.2:
$ go test -tags purego -benchtime 10s -bench '/xxhash,direct,bytes' $ go test -benchtime 10s -bench '/xxhash,direct,bytes'