Build your own slice: Make, Literals and Re-slicing

In this series we’ve thus far seen only one way of creating slices: by slicing an existing array.

food:= [4]string{"🍔", "🍕", "🍏", "🍊"}
fruits := food[2:4]
fmt.Println(fruits) // prints [🍏 🍊]

Creating slices like this was useful to introduce and demonstrate the relationship between arrays and slices.

However, it would be a bit annoying if this was the only way to create a slice. We’d be forced to use two variables (one for the array, one for the slice) every time we need one.

Luckily, Go gives us more concise ways to create slices:

• The built-in make function.
• Slice literals.
• Re-slicing an existing slice.

In the first section of this article we’ll discuss each “creation method” in detail.

In the second section we will get our hands dirty and emulate them for the custom Slice type we have been building.

Are you or your team struggling with aspects of Go?

Book a 30 minute check-in with me.

I can help with questions like:

• How should we structure our Go web app?
• Is this package a good choice for us?
• How do I get my team up to speed quickly?
• Can I pick your brain on {subject}?

Learn more

Looking forward to meeting you.

- Willem

Make function

The built-in make function is rather versatile, it can be used to create several types: slices, maps and channels.

In this article we will limit ourselves to just the slices.

make is the primary way to create slices of specific lengths and capacities. If, for example, you need to create a slice where the length is based on some variable, make is your friend.

make will always create a new backing array and a new slice that references it. It can be called with two or three parameters.

With two parameters

When calling make with two parameters:

• The first parameter is the type of the slice you want to create.
• The second parameter is the length and capacity of the slice (and hence, also the length of the backing array).

For example, if you want to create a slice of strings with a length and capacity of 4. That would look like make([]string, 4).

Try it yourself below. What do you think the values of the slice elements will be?

package main

import "fmt"

func main() {
	// create a slice using make with two parameters.
	s := make([]string, 4)
	fmt.Println(s)
	fmt.Println("len", len(s))
	fmt.Println("cap", cap(s))
}

One way to think about calling make with two parameters is to visualize the created slice as a “window” into its entire backing array:

In the diagram you can also see that all elements default to the zero-value of the element type. In this case, the empty string, which is the zero type of a string.

With three parameters

When calling make with three parameters:

• The first parameter is the type of the slice you want to create.
• The second parameter is the length of the slice.
• The third parameter is the capacity of the slice (and the length of the backing array).

For example, calling make([]string, 4, 6) will result in a slice of strings of length 4 and capacity 6.

Again, all elements default to their zero values. Try it yourself below.

package main

import "fmt"

func main() {
	// create a slice using make with three parameters.
	s := make([]string, 4, 6)
	fmt.Println(s)
	fmt.Println("len", len(s))
	fmt.Println("cap", cap(s))
}

If we visualize the above example, it shows us that calling make like this allows you to control the size of the backing array and the dimensions of the “window” into it.

Slice literal

A slice literal allows you to define a slice with a number of elements in one go. No need to define a variable and append the values separately.

The syntax for slice literals looks like: []T{e1, e2, e3, ...eN}. You can specify 0 or more elements.

In the example below we define a variable s and assign a slice of strings using a literal with 5 elements. We then print its length and capacity.

package main

import "fmt"

func main() {
	// create a slice using a literal
	s := []string{"🍏", "🍊", "🍋", "🍐", "🍉"}
	fmt.Println(s)
	fmt.Println("len", len(s))
	fmt.Println("cap", cap(s))
}

When using a literal, Go will always create a backing array that exactly fits the provided elements. In the above example, s will have:

• An offset of 0.
• A length and capacity of 5.

If we visualize s, we can see that a slice literal will always have a “window” into its entire backing array.

Re-slicing an existing slice

In the first article we saw that it’s possible to “slice” an array to create a new slice. Similarly, you can also create a new slice by “re-slicing” an existing slice.

You can re-slice an existing slice by using an index expression with two indices: s[low:high].

All elements between low (inclusive) and high (exclusive) in the existing slice s will be part of the resulting slice. low must be 0 or greater and high can be at most the capacity of s.

The example below shows how this works in code, it creates a new slice newS by re-slicing oldS. We then modify one of the elements of newS, what do you think happens to the elements in oldS?

package main

import "fmt"

func main() {
	// create a slice by re-slicing oldS
	oldS := []string{"🍏", "🍊", "🍋", "🍐", "🍉"}
	newS := oldS[1:4]
	fmt.Println(newS)
	fmt.Println("len", len(newS))
	fmt.Println("cap", cap(newS))

	fmt.Println("oldS before", oldS)
	// update first element of newS
	newS[2] = "🍔"
	fmt.Println("oldS after ", oldS)
}

If you ran the code, you should have seen that oldS now contains a "🍔" as well. This is because both oldS and newS share the same backing array.

Re-slicing only creates a new slice, ands this new slice always refers to the same backing array as the original slice.

If we visualize the above example it looks like this:

There is one important limit we need to discuss. Due to the way slices are implemented in Go (and described in the Go language spec), you can’t re-slice using a negative low index.

In other words, it’s not possible to use re-slicing to create a “window” that starts before the “window” of a particular slice.

If we try to re-slice newS with a negative index like this:

s := newS[-1, 3]

It will either not compile:

invalid argument: index -1 (constant of type int) must not be negative

Or it will cause a panic during runtime:

slice bounds out of range [-1:]

Get the backing array

In the previous articles we always started out with an array from which we “sliced” a slice. Since the backing array was just another variable in our code, we could always get a reference or copy of it.

When using make, literals or re-slicing, Go will create the backing arrays for you behind the scenes. How do we now get a reference or copy of the backing array?

In most cases you can convert your slice to (a pointer to) an array. If you want to know more, check out this code snippet.

Build your own

Now that we have seen how to use make, literals and re-slicing. Let’s implement them ourselves for our Slice type.

Again, we build on the work we did in the previous articles.

Make functions

As we saw earlier, the make function comes in two variants that vary slightly in the parameters that they accept.

We will build two functions to emulate these two variants:

• Make(length) that emulates a make([]string, length) call.
• MakeCap(length, capacity) which will emulate a make([]string, length, capacity) call.

We will ignore the type parameter for our implementation since we only work with slices of strings.

Since MakeCap is essentially a more versatile version of Make we will implement it first and then use it to implement Make.

MakeCap has the following signature:

func MakeCap(length, capacity int) Slice {
	// ...
}

Our first step is validating the provided values: length should not be negative or greater than the capacity.

func MakeCap(length, capacity int) Slice {
	// 1. validate the length and capacity.
	if length < 0 || length > capacity {
		panic("len out of range")
	}

	// ...
}

We then need to create a backing array. We already know how to do that, because we implemented the “creation of a new backing array” when we wrote the Append function in the previous article.

func MakeCap(length, capacity int) Slice {
	// 1. validate the length and capacity.
	if length < 0 || length > capacity {
		panic("len out of range")
	}

	// 2. create a new array of type [capacity]string.
	arrType := reflect.ArrayOf(length, reflect.TypeOf(""))
	arr := reflect.New(arrType).Elem()

	// ...
}

Now for the last step, we return a new Slice with:

• A reference to the newly created backing array.
• An offset of 0, slices created via make always have a “window” starting at the beginning of the backing array.
• The length and capacity should be as provided by the caller.

In code this looks as follows:

func MakeCap(length, capacity int) Slice {
	// 1. validate the length and capacity.
	if length < 0 || length > capacity {
		panic("len out of range")
	}

	// 2. create a new array of type [capacity]string.
	arrType := reflect.ArrayOf(length, reflect.TypeOf(""))
	arr := reflect.New(arrType).Elem()

	// 3. return a slice
	return Slice{
		array:    arr,
		offset:   0,
		length:   length,
		capacity: capacity,
	}
}

Now that we have a MakeCap function, we can use this to build the Make function.

Make only accepts a length parameter which doubles as a capacity. Implementing it is only a matter of calling MakeCap:

func Make(length int) Slice {
	return MakeCap(length, length)
}

That’s it for Make and MakeCap. Let’s take a look at emulating literals.

Emulating literals

A string slice literal can have 0 or more elements. You know what kind of function can be called with 0 or more arguments?

A variadic function.

If we use this signature for our Literal function:

func Literal(vals ...string) Slice {
	// ...
}

It can then be called like this:

// returns an empty slice
Literal()
// returns a slice containing 3 elements.
Literal("a", "b", "c")

So what does our Literal function need to do? It needs to:

1. Create a slice that has exactly enough capacity and length to contain all the provided values.
2. Set all the values in the slice.
3. Return the slice.

We can use our earlier Make function to create the slice, and then loop over the values and set them.

In code this looks like this:

func Literal(vals ...string) Slice {
	// 1. create a slice that exactly fits the provided values.
	s := Make(len(vals))

	// 2. set all the values.
	for i, v := range vals {
		s.Set(i, v)
	}

	// 3. return the slice.
	return s
}

Re-slicing

Now for the last function (or method, since we define it on the Slice struct) of this article: Reslice.

Reslice should emulate the s[low:high] index expression, it’s signature will look like this:

func (s Slice) Reslice(low, high int) Slice {
	// ...
}

Reslice will need to undertake two steps:

1. Validate the provided bounds.
2. Return a new slice with the appropriate window.

Let’s start by validating those bounds.

• low should not be negative.
• low should not be greater than high. The smallest the “window” can be is 0.
• high should not be greater than the capacity of the original slice. You can’t reslice outside of the backing array.

In code, our bound checks look like this:

func (s Slice) Reslice(low, high int) Slice {
  // 1. validate the provided bounds.
	if low < 0 || low > high || high > s.capacity {
		panic(fmt.Sprintf("slice bounds out of range [%d:%d]", low, high))
	}

	// ...
}

We now need to return a new slice, but what values should we use for the fields? Let’s assume that s is the original slice like in the above signature.

Re-slicing a slice should return a slice that references the same backing array, so for the array field we should use s.array.

For the other fields, we will need to do some calculations. These calculations are similar to what we did for SliceArray in the first article, but now they need to be relative to s.offset.

Let’s go over the fields:

• offset we need to shift the “window” of the original slice by low elements. This comes down to s.offset + low.
• length is the “size of the window” and doesn’t change because of the offset. We can again use high - low.
• capacity, is the number of elements from an offset of a slice to the end of the backing array. So, if we do s.offset + low to get a new offset, we need to subtract low from the s.capacity to get the corresponding capacity.

If we add these calculations to the Reslice method it looks like this:

func (s Slice) Reslice(low, high int) Slice {
	// 1. validate the provided bounds.
	if low < 0 || low > high || high > s.capacity {
		panic(fmt.Sprintf("slice bounds out of range [%d:%d]", low, high))
	}

	// 2. return a new slice using the bounds.
	return Slice{
		array:    s.array,
		offset:   s.offset + low,
		length:   high - low,
		capacity: s.capacity - low,
	}
}

That wraps up Reslice. Let’s try out all our code in the demo below!

Demo

package main

import "fmt"

func main() {
	// assign using make
	s1 := Make(3)
	fmt.Println("Make(3)")
	printSlice(s1)

	// assign using make with capacity
	s2 := MakeCap(3, 6)
	fmt.Println("MakeCap(3, 6)")
	printSlice(s2)

	// assign using a literal
	s3 := Literal("🍔", "🍕", "🍏")
	fmt.Println("Literal(\"🍔\", \"🍕\", \"🍏\")")
	printSlice(s3)

	// reslice the literal to only select the last two elements
	s4 := s3.Reslice(1, 3)
	fmt.Println("s3.Reslice(1, 3)")
	printSlice(s4)
}

func printSlice(s Slice) {
	fmt.Printf("cap: %d, len: %d, %v\n\n", s.Cap(), s.Len(), s)
}

package main

import (
	"reflect"
)

func MakeCap(length, capacity int) Slice {
	// 1. validate the length and capacity.
	if length < 0 || length > capacity {
		panic("len out of range")
	}

	// 2. create a new array of type [capacity]string.
	arrType := reflect.ArrayOf(length, reflect.TypeOf(""))
	arr := reflect.New(arrType).Elem()

	// 3. return a slice
	return Slice{
		array:    arr,
		offset:   0,
		length:   length,
		capacity: capacity,
	}
}

func Make(length int) Slice {
	return MakeCap(length, length)
}

package main

func Literal(vals ...string) Slice {
	// 1. create a slice that exactly fits the provided values.
	s := Make(len(vals))

	// 2. set all the values.
	for i, v := range vals {
		s.Set(i, v)
	}

	// 3. return the slice.
	return s
}

package main

import "fmt"

func (s Slice) Reslice(low, high int) Slice {
	// 1. validate the provided bounds.
	if low < 0 || low > high || high > s.capacity {
		panic(fmt.Sprintf("slice bounds out of range [%d:%d]", low, high))
	}

	// 2. return a new slice using the bounds.
	return Slice{
		array:    s.array,
		offset:   s.offset + low,
		length:   high - low,
		capacity: s.capacity - low,
	}
}

package main

import (
	"fmt"
	"reflect"
)

type Slice struct {
	array    reflect.Value
	offset   int
	length   int
	capacity int
}

func (s Slice) Len() int {
	return s.length
}

func (s Slice) Cap() int {
	return s.capacity
}

func SliceArray(a any, low, high int) Slice {
	// 1. check that a is a non-nil pointer.
	ptr := reflect.ValueOf(a)
	if ptr.Kind() != reflect.Pointer || ptr.IsNil() {
		panic("can only slice a non-nil pointer")
	}

	// 2. check if a points to an array of strings.
	v := ptr.Elem()
	if v.Kind() != reflect.Array || v.Type().Elem().Kind() != reflect.String {
		panic("can only slice arrays of strings")
	}

	// 3. validate the bounds.
	if low < 0 || high > v.Len() || low > high {
		panic(fmt.Sprintf("slice bounds out of range [%d:%d]", low, high))
	}

	// 4. calculate offset, length and capacity and return the slice
	return Slice{
		array:    v,
		offset:   low,
		length:   high - low,
		capacity: v.Len() - low,
	}
}

func (s Slice) Get(x int) string {
	// 1. Check if x is in range.
	if x < 0 || x >= s.length {
		panic(fmt.Sprintf("index out of range [%d] with length %d", x, s.length))
	}

	// 2. Retrieve the element.
	return s.array.Index(s.offset + x).String()
}

func (s Slice) Set(x int, value string) {
	// 1. Check if x is in range.
	if x < 0 || x >= s.length {
		panic(fmt.Sprintf("index out of range [%d] with length %d", x, s.length))
	}

	// 2. Set the element value.
	s.array.Index(s.offset + x).SetString(value)
}

func (s Slice) String() string {
	out := "["
	for i := 0; i < s.length; i++ {
		if i > 0 {
			// add a space between elements
			out += " "
		}
		out += s.Get(i)
	}
	return out + "]"
}

func Append(s Slice, vals ...string) Slice {
	newS := s
	valsLen := len(vals)

	// Grow newS to fit the vals.
	newS.length += valsLen

	if valsLen <= s.capacity-s.length {
		// Case 1: Append using the array of the original slice.
		setValsAfter(newS, s.length, vals...)

		return newS
	}

	// Case 2: Append using a new backing array.

	// 1. The new slice begins at the start of the new array.
	newS.offset = 0
	newS.capacity = calcNewCap(s.capacity, newS.length)

	// 2. Create a new backing array with that capacity.
	arrType := reflect.ArrayOf(newS.capacity, reflect.TypeOf(""))
	newS.array = reflect.New(arrType).Elem()
	newS.offset = 0 // the new slice begins at the start of this array.

	// 3. Copy the values of the original slice.
	Copy(newS, s)

	// 4. Copy the vals in order like we do for the first case.
	setValsAfter(newS, s.length, vals...)

	return newS
}

func setValsAfter(s Slice, offset int, vals ...string) {
	for i, v := range vals {
		s.Set(offset+i, v)
	}
}

func calcNewCap(oldCap int, newLen int) int {
	newCap := oldCap
	doubleCap := newCap + newCap
	if newLen > doubleCap {
		newCap = newLen
	} else {
		const threshold = 256
		if oldCap < threshold {
			newCap = doubleCap
		} else {
			// Check 0 < newcap to detect overflow
			// and prevent an infinite loop.
			for 0 < newCap && newCap < newLen {
				// Transition from growing 2x for small slices
				// to growing 1.25x for large slices. This formula
				// gives a smooth-ish transition between the two.
				newCap += (newCap + 3*threshold) / 4
			}

			// Set newcap to the requested cap when
			// the newcap calculation overflowed.
			if newCap <= 0 {
				newCap = newLen
			}
		}
	}

	return newCap
}

func Copy(dst, src Slice) int {
	return copyRecursive(dst, src, 0)
}

func copyRecursive(dst, src Slice, i int) int {
	if i >= src.Len() || i >= dst.Len() {
		return i
	}

	v := src.Get(i)
	copied := copyRecursive(dst, src, i+1)
	dst.Set(i, v)

	return copied
}

That looks like it works as expected :)

Summary

That wraps up this article. I hope it gave you an overview of the different ways you can create slices.

We discussed and implemented our own version of:

• The make function, the main way to create slices of specific lengths and capacities.
• Literals, the way to create slices with specific elements.
• Re-slicing, a way to create a new “window” into an existing slice and backing array.

If you have any questions or comments, feel free to reach out.

Also, sign up for my newsletter if you want to be notified when the next (and last!) article in this series is released.