Improve append allocations (in loops)

Is append causing more allocs/op then you expected in your benchmarks?

There is an optimization you can use when you know the (approximate) number of elements that you’re going to append to a slice: Create the initial slice with enough capacity.

You can do this using make with 3 parameters, the last parameter will be the capacity of the slice.

s := make([]string, 3, 12)

Will create a slice of strings called s with length of 3 and a capacity of 12.

So why can this improve performance when calling append?

Every time append is called, it checks if there is enough capacity in the backing array of the original slice.

If the array of the original slice does not have enough capacity, append will allocate a new backing array and copy the relevant elements.

However, when the original slice has enough capacity, append will use its existing backing array. No new backing array is allocated.

A common situation where this can make a big difference is when append is called in a loop. This can be seen in the following snippet, which prints out a message every time a new backing array is allocated.

Run the snippet with both versions of s to see the difference.

package main

import (
	"fmt"
)

func main() {
	// Use the following line to create s with enough capacity.
	//s := make([]int, 0, 100)
	s := make([]int, 0)

	prevCap := cap(s)
	count := 0
	for i := 0; i < 100; i++ {
		s = append(s, i)

		// Check if a new backing array was allocated by checking
		// if the capacity changed.
		c := cap(s)
		if prevCap != c {
			fmt.Printf("%d: new array with cap %d\n", i, c)
			prevCap = c
			count++
		}
	}

	fmt.Printf("%d new arrays allocated", count)
}