Ok, I lied, there's no weird trick. However, you can easily reduce a Go binary size by more than 6 times with some flags and common tools.
Note: I don't actually believe a 30MB static binary is a problem in this day and age, and I would not trade (build time | complexity | performance | debug-ability) for it, but people care about it apparently, so here we are. For the record, the dev team cares about it, too.
It all started with an observation Jessie Frazelle made: go binaries compress to almost 1/5th of their size.
$ ls -l go go.xz
-rwxr-xr-x 1 filippo staff 12493536 Apr 16 16:58 go
-rwxr-xr-x 1 filippo staff 2647596 Apr 16 16:58 go.xz
Not a surprise if you look at the entropy graph (generated with binwalk). A lot of sections there should compress extremely well.
When I mentioned this is the office John Graham-Cumming suggested that we should then just make binaries self-decompress at runtime. At first I though he was joking, then I realized that no, this almost made sense!
So this weekend was going to be about hard-core engineering, building a binary that decompresses a payload and then JMPs to it. But soon enough I found out that such a thing exists already, it's called UPX and is quite nice.
So this post will be about graphs instead!
But first, let me strip the binary
Before we go all in on compression, there's something we can do to make binaries smaller: strip them.
We can use the -s and -w linker flags to strip the debugging information like this:
$ GOOS=linux go build cmd/go
$ ls -l go
-rwxr-xr-x 1 filippo staff 12493536 Apr 16 16:58 go
$ GOOS=linux go build -ldflags="-s -w" cmd/go
$ ls -l go
-rwxr-xr-x 1 filippo staff 8941952 Apr 16 17:08 go
We already lost 28% of our weight! Here's the new entropy graph; you can see that the last pretty-low entropy section is gone.
Interestingly, what gets stripped is only the DWARF tables needed for debuggers, not the annotations needed for stack traces, so our panics are still readable!
$ cat ../hello.go
package main
func TheHitchhikersGuideToTheGalaxy() {
panic("DO NOT PANIC")
}
func main() {
TheHitchhikersGuideToTheGalaxy()
}
$ go build -ldflags="-s -w" hello.go
$ ./hello
panic: DO NOT PANIC
goroutine 1 [running]:
panic(0x569c0, 0xc82000a140)
/usr/local/Cellar/go/1.6.1/libexec/src/runtime/panic.go:464 +0x3e6
main.TheHitchhikersGuideToTheGalaxy()
/Users/filippo/code/gopack/hello.go:4 +0x65
main.main()
/Users/filippo/code/gopack/hello.go:8 +0x14
Enter UPX
Next step, let's run upx. It works out of the box on linux binaries built by 1.6, but for earlier versions you'll need goupx.
$ GOOS=linux go build cmd/go
$ ls -l go
-rwxr-xr-x 1 filippo staff 12493536 Apr 16 16:58 go
$ upx --brute go
Ultimate Packer for eXecutables
Copyright (C) 1996 - 2013
UPX 3.91 Markus Oberhumer, Laszlo Molnar & John Reiser Sep 30th 2013
File size Ratio Format Name
-------------------- ------ ----------- -----------
12493536 -> 2554140 20.44% linux/ElfAMD go
Packed 1 file.
$ ls -l go
-rwxr-xr-x 1 filippo staff 2554140 Apr 16 16:58 go.upx
We went from 12MB to 2.5MB! Combined with -s and -w, we can reduce the binary size to just 15% of the default build. Almost 7 times smaller!
Here's a graph of the sizes obtained by each technique, applied to the Go compiler, to Gogs and to hello.go.
Obviously decompression is not free, but the overhead should only be at process start. In my benchmarks the UPX version of go was taking 160ms more to start, and hello 15ms.
Here's all the data and here is the simple build script I used.
It's getting better
One final note: a lot of work went into the 1.7 cycle to shrink the binaries, including new conditional tree pruning and generic SSA savings.
Here's the size of the same binaries built with Go tip, where 100% is the size of the default 1.6.1 build.
If you like reading interesting facts about Go in a totally-not-Buzzfeed style, you might want to follow me on Twitter.