Improve append allocations (in loops)

Avatar of the author Willem Schots
28 Jul, 2023
~2 min.
RSS

Is append causing more allocs/op then you expected in your benchmarks?

There is an optimization you can use when you know the (approximate) number of elements that you’re going to append to a slice: Create the initial slice with enough capacity.

You can do this using make with 3 parameters, the last parameter will be the capacity of the slice.

s := make([]string, 3, 12)

Will create a slice of strings called s with length of 3 and a capacity of 12.

So why can this improve performance when calling append?

Every time append is called, it checks if there is enough capacity in the backing array of the original slice.

If the array of the original slice does not have enough capacity, append will allocate a new backing array and copy the relevant elements.

However, when the original slice has enough capacity, append will use its existing backing array. No new backing array is allocated.

A common situation where this can make a big difference is when append is called in a loop. This can be seen in the following snippet, which prints out a message every time a new backing array is allocated.

Run the snippet with both versions of s to see the difference.

main.go
package main

import (
	"fmt"
)

func main() {
	// Use the following line to create s with enough capacity.
	//s := make([]int, 0, 100)
	s := make([]int, 0)

	prevCap := cap(s)
	count := 0
	for i := 0; i < 100; i++ {
		s = append(s, i)

		// Check if a new backing array was allocated by checking
		// if the capacity changed.
		c := cap(s)
		if prevCap != c {
			fmt.Printf("%d: new array with cap %d\n", i, c)
			prevCap = c
			count++
		}
	}

	fmt.Printf("%d new arrays allocated", count)
}

Get my free newsletter every second week

Used by 500+ developers to boost their Go skills.

"I'll share tips, interesting links and new content. You'll also get a brief guide to time for developers for free."

Avatar of the author
Willem Schots

Hello! I'm the Willem behind willem.dev

I created this website to help new Go developers, I hope it brings you some value! :)

You can follow me on Twitter/X, LinkedIn or Mastodon.

Thanks for reading!