Build your own slice: Arrays and slices

When it comes to slices you can get pretty far without knowing how the sausage is made. However, things can get weird once you start slicing and reslicing.

If you consider slices to be “some kind of dynamic array”, the following output might be surprising.

package main

import "fmt"

func main() {
    // create an array with 4 fruits.
    fruits := [4]string{"🍊", "🍏", "🍎", "🍐"}

    // citrus is a slice with only the orange.
    citrus := fruits[0:1]

    // replace the orange with a lemon in citrus.
    citrus[0] = "🍋"

    // now print the original fruits array.
    fmt.Println(fruits)
}

If citrus and fruits were two independent collections, this would have output [🍊 🍏 🍎 🍐].

But as you can see when you ran the above code, they aren’t. So what are slices?

This article will explain.

We will:

• Give some background on arrays.
• Explain how to think about slices.
• And finally, build our own slice.

Are you or your team struggling with aspects of Go?

Book a 30 minute check-in with me.

I can help with questions like:

• How should we structure our Go web app?
• Is this package a good choice for us?
• How do I get my team up to speed quickly?
• Can I pick your brain on {subject}?

Learn more

Looking forward to meeting you.

- Willem

Arrays in Go

Slices are based on arrays, so let’s take a look at them first.

In Go, an array is a numbered sequence of elements. This sequence has a fixed length. Each element must be of the same type.

When you declare an array, all elements will default to their zero value.

// declare an array with 4 elements.
// each element must be a string.
// the value of each element will be the empty string.
var empties [4]string

You can also initialize an array by providing an array literal.

// declares and initializes an array with 6 elements.
// each element must be a string.
// the value of each element will be one of the provided values.
animals := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

Elements missing from the array literal will default to their zero value.

// declares and initializes an array with 3 elements.
// each element must be a string.
// the value of the first two elements will be one of the provided values.
// the last element will be the empty string.
mixed := [3]string{"🦥", "🐘"}

In my mind I picture arrays as “blocks of data” of a fixed size. If you were to diagram them they would look like this:

Play

In the above diagrams you can see I labeled each element with an index. These indices range from 0 to array length - 1.

Element values can accessed by using indices in index expressions.

Index expressions look like a[x].

Where a is the name of the array and x is the index of the element. They can be used to “get” and “set” values in the array.

Play around with indices in the following example. Check what happens if you provide indices that are out of range.

package main

import "fmt"

func main() {
    // an array with 6 elements.
    animals := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

    // "set" the first element value.
    animals[0] = "🐉"

    // "get" and print the first element value.
    fmt.Println(animals[0])

    // "get" and print the last element value.
    fmt.Println(animals[5])
}

There are two notable restrictions to arrays.

Restriction 1: Fixed length

The length of an array is fixed. This means that you can’t grow or shrink an array.

For example, extending the array animals array and adding a 7th element is not possible.

Similarly, you can’t remove an element from animals and end up with an array containing 5 elements.

The array length is also part of the data type. Trying to assign a [5]string to a variable of type [6]string will result in a compilation error like this:

...cannot use [5]string{…} (value of type [5]string) as [6]string value in assignment

You can use [...] instead of a number when creating an array:

animals := [...]{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

This does nothing to change the nature of arrays. It just infers the length from the numbers of elements in the source code. In the example above animals is still of type [6]string.

Restriction 2: Constant length

The length of an array must be known at compile time. This means that it must be a constant.

For example, you can’t use variables as the size of an array:

x := someNumber()
var arr [x]string

This will result in a compilation errors like:

...invalid array length

or

array length ... (value of type ...) must be constant

This restriction can be worked around by using reflection. Reflection is not recommended for general code: it is slow and you are working around Go’s type system.

Arrays are inflexible

These restrictions make arrays an impractical choice in a lot of situations. When dealing with “collections” of elements, it’s common to:

• Not know in advance how many elements you will need.
• Want to change the length of the collection dynamically.

This is where slices come in.

Slices: Flexible windows into arrays

A slice describes a sequence of elements inside an array. This array is sometimes called the “backing array”.

You can see a slice as a “window” into an array:

Play

One array can have many slices referencing it.

To form a slice we need:

• A reference to an array Which array does this slice apply to?
• Offset The index of the first element of the “window”.
• Length The number of elements in the “window”.
• Capacity The number of elements from the offset to the end of the backing array.

Play

Slices are lightweight. When passing a slice to a function only the data above is copied. The array is not copied with it (only a reference to it).

Now that we have an idea of the theory, let’s get practical.

Slicing an existing array

We will begin with the situation from earlier: You have an existing array and then create a “window” into it.

Suppose you have an array called a.

By using an index expression with two indices a[low : high] you can create a slice that uses a as its backing array.

All elements between low (inclusive) and high (exclusive) in the backing array will be part of the slice. low must be 0 or greater, and high can be at most the length of the array.

In code, this looks as follows. Play around and try different combinations, try to cause some errors.

package main

import "fmt"

func main() {
    // a is an array with 6 elements.
    a := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

    // s is a slice backed by a.
    s := a[1:4]

    fmt.Println(s)
}

If we map this example onto our earlier diagram, it will look like this.

Play

So how did we end up with the numbers and calculations in the diagram?

• Offset is the low index.
• Length calculated by high - low.
• Capacity calculated by array length - low.

An array reference and the two indices give us enough information to create a full slice.

Accessing elements

As you just saw, a slice does not store elements. It only refers to an array that actually contains the elements.

So when we talk about “elements in a slice”, we mean the elements in the backing array that are “part of the window”.

You can access these elements by using index expressions on a slice.

In other words, accessing a slice element will actually access a corresponding element in the backing array relative to the slice’s offset.

Suppose you have a slice s with an offset n and backing array a.

Index expression s[x] will access the element a[n + x] in the backing array.

Valid values for the index range from 0 up to the length of the slice (not the backing array). Providing an index out of range will panic.

To see these index expressions in action, run the example below.

package main

import "fmt"

func main() {
    // a is an array with 6 elements.
    a := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

    // s is a slice backed by a.
    s := a[1:4]

    // print each element in s using index expressions.
    fmt.Println(s[0])
    fmt.Println(s[1])
    fmt.Println(s[2])
}

If we diagram this example we get the following diagram:

Play

Slice index expressions can also be used to set values in the backing array.

package main

import "fmt"

func main() {
    // a is an array with 6 elements.
    a := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

    // s is a slice backed by a.
    s := a[1:4]

    // set the slice's first element.
    s[0] = "🐉"

    // verify the element was also changed in a.
    fmt.Println(a[1])
}

In the example above, a change made via the slice is reflected in the backing array.

Length and Capacity

The length and capacity of a slice can be accessed using the len and cap built-in functions.

If s is a slice:

• len(s) would get the length of that slice.
• cap(s) will get the capacity of that slice.

Using these functions

package main

import "fmt"

func main() {
    // a is an array with 6 elements.
    a := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

    // s is a slice backed by a.
    s := a[1:4]

    // print the length of the slice.
    fmt.Println(len(s))

    // print the capacity of the slice.
    fmt.Println(cap(s))
}

Build your own slice

To really solidify our understanding of the above behavior, we will emulate it by building our own slice.

Our slice is a learning tool, and not meant as a replacement for real slices. We will focus on building a useful mental model for users of the Go language. Our slice will:

• Only support string type elements (instead of supporting all types).
• Use reflection (real slices use unsafe pointers).

So with that said, let’s get started.

Slice struct

To begin, we need a struct to hold the data we discussed earlier.

type Slice struct {
    array    reflect.Value
    offset   int
    length   int
    capacity int
}

func (s Slice) Len() int {
    return s.length
}

func (s Slice) Cap() int {
    return s.capacity
}

We store the array reference as a reflect.Value, this is a type that can be used to represent and manipulate any Go value. Later functions will ensure this field will always contain an array value.

The other fields are all int types since they are always “whole numbers”. They will never be negative, so we could use the uint type instead. But, that would litter our code with type casts, which is a bit of a pain.

There are also the Len and Cap methods. These emulate the len and cap built-in functions.

SliceArray function

This function will slice an array and emulate the a[low : high] index expression.

SliceArray should accept three parameters and return a Slice.

func SliceArray(a any, low, high int) Slice {
    // ...
}

The a parameter is of type any. This type accepts every Go type.

Ideally we would like a to be a type that describes “a pointer to a string array of any size”, but that is not possible using just types in Go. We will implement this using reflection.

Why do we need a pointer? Without this pointer we would be working on the variable that is local to `SliceArray, we won’t be working with the variable that was provided to the function.

So how should this function work?

1. Validate that a is a non-nil pointer.
2. Validate that this pointer points to a “array of strings of any size” and get a reflect.Value for that array.
3. Validate the low and high bounds.
4. Return a Slice with the appropriate offset, length and capacity.

Let’s begin with the first step.

func SliceArray(a any, low, high int) Slice {
    // 1. check that a is a non-nil pointer.
    ptr := reflect.ValueOf(a)
    if ptr.Kind() != reflect.Pointer || ptr.IsNil() {
        panic("can only slice a non-nil pointer")
    }

    // ...
}

If you’re unfamiliar with reflection this can be a bit intimidating, let’s walk through it.

First we assign ptr by getting a reflect.Value of a. This enables us to get information about a on the “language level”.

We then panic if a either is:

• Not a “kind of” pointer.
• Is nil using IsNil().

We can now check if this pointer actually points to an array of strings.

func SliceArray(a any, low, high int) Slice {
    // 1. check that a is a non-nil pointer.
    ptr := reflect.ValueOf(a)
    if ptr.Kind() != reflect.Pointer || ptr.IsNil() {
        panic("can only slice a non-nil pointer")
    }

    // 2. check if a points to an array of strings.
    v := ptr.Elem()
    if v.Kind() != reflect.Array || v.Type().Elem().Kind() != reflect.String {
        panic("can only slice arrays of strings")
    }

    //...
}

We assign the element of the pointer to v. This is the reflect.Value of the value that the pointer points to.

We then panic if v:

• Is not a “kind of” array.
• The type of elements in v are not a “kind of” string.

If this executes succesfully we know that a is a pointer to an array of strings, and we have a reference to the backing array in v.

On to the third step, the bounds validation.

func SliceArray(a any, low, high int) Slice {
    // 1. check that a is a non-nil pointer.
    ptr := reflect.ValueOf(a)
    if ptr.Kind() != reflect.Pointer || ptr.IsNil() {
        panic("can only slice a non-nil pointer")
    }

    // 2. check if a points to an array of strings.
    v := ptr.Elem()
    if v.Kind() != reflect.Array || v.Type().Elem().Kind() != reflect.String {
        panic("can only slice arrays of strings")
    }

    // 3. validate the bounds.
    if low < 0 || high > v.Len() || low > high {
        panic(fmt.Sprintf("slice bounds out of range [%d:%d]", low, high))
    }
    
    // ...
}

Here we use v.Len() to get the length of the backing array.

The rules for bounds validation are as follows:

• low cannot be lower than 0.
• high must greater than array length.
• low must be smaller or equal to high.

Note that as a consequence of the last rule, it is valid to create an empty slice into an existing array.

Now the last step, creating the Slice struct.

func SliceArray(a any, low, high int) Slice {
    // 1. check that a is a non-nil pointer.
    ptr := reflect.ValueOf(a)
    if ptr.Kind() != reflect.Pointer || ptr.IsNil() {
        panic("can only slice a non-nil pointer")
    }

    // 2. check if a points to an array of strings.
    v := ptr.Elem()
    if v.Kind() != reflect.Array || v.Type().Elem().Kind() != reflect.String {
        panic("can only slice arrays of strings")
    }

    // 3. validate the bounds.
    if low < 0 || high > v.Len() || low > high {
        panic(fmt.Sprintf("slice bounds out of range [%d:%d]", low, high))
    }
    
    // 4. calculate offset, length and capacity and return the slice
    return Slice{
        array:    v,
        offset:   low,
        length:   high - low,
        capacity: v.Len() - low,
    }
}

Here we use the calculations we saw earlier to initialize the slice.

This wraps up the SliceArray function.

Get and Set methods

These functions will emulate the s[x] index expression. We will add two methods to the Slice because index expressions can be used to “get” and “set” elements.

Both methods should check if the index is in the range of the slice.

If the element is in range, we should “get” or “set” the appropriate element in the backing array.

Let’s begin with Get.

func (s Slice) Get(x int) string {
    // 1. Check if x is in range.
    if x < 0 || x >= s.length {
        panic(fmt.Sprintf("index out of range [%d] with length %d", x, s.length))
    }

    // 2. Retrieve the element.
    return s.array.Index(s.offset + x).String()
}

We first check if the index x is in range and panic if it is not.

Then we use Index method on s.array to get the offset + xth element.

The result of Index is another reflect.Value, so we need to call String() to return the underlying string value.

And now, the Set method.

func (s Slice) Set(x int, value string) {
    // 1. Check if x is in range.
    if x < 0 || x >= s.length {
        panic(fmt.Sprintf("index out of range [%d] with length %d", x, s.length))
    }

    // 2. Set the element value.
    s.array.Index(s.offset + x).SetString(value)
}

We again check if the index x is in range and access the reflect.Value of the element using Index.

But now, instead of reading the value, we call SetString on the element to update the underlying string value.

String method

We will implement one more method in this article: the String method.

Currently, if we were to format a Slice as a string, it would like something like this:

{{0x50aa40 0xc00007bb60 401} 1 3 5}

Not very user friendly.

We can define our own “default format” by implementing the Stringer interface from the fmt package.

This interface is implemented by defining a String method like this:

func (s Slice) String() string {
    out := "["
    for i := 0; i < s.length; i++ {
        if i > 0 {
            // add a space between elements
            out += " "
        }
        out += s.Get(i)
    }
    return out + "]"
}

Now our Slice will be formatted like a regular slice.

[🐟 🦒 🐢]

Demo

Time to take our own slice for a spin! Let’s also check how it compares to a regular slice.

package main

import "fmt"

func main() {
    // a and b are two arrays with 6 elements.
    a := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}
    b := [6]string{"🐊", "🐟", "🦒", "🐢", "🙈", "🐱"}

    // regular is a s slice backed by a.
    regular := a[1:4]
    regular[0] = "🐉"
    fmt.Println(regular)

    // custom is our own slice backed by b.
    custom := SliceArray(&b, 1, 4)
    custom.Set(0, "🐉")
    fmt.Println(custom)
}

package main

import (
	"fmt"
	"reflect"
)

type Slice struct {
    array    reflect.Value
    offset   int
    length   int
    capacity int
}

func (s Slice) Len() int {
    return s.length
}

func (s Slice) Cap() int {
    return s.capacity
}

func SliceArray(a any, low, high int) Slice {
    // 1. check that a is a non-nil pointer.
    ptr := reflect.ValueOf(a)
    if ptr.Kind() != reflect.Pointer || ptr.IsNil() {
        panic("can only slice a non-nil pointer")
    }

    // 2. check if a points to an array of strings.
    v := ptr.Elem()
    if v.Kind() != reflect.Array || v.Type().Elem().Kind() != reflect.String {
        panic("can only slice arrays of strings")
    }

    // 3. validate the bounds.
    if low < 0 || high > v.Len() || low > high {
        panic(fmt.Sprintf("slice bounds out of range [%d:%d]", low, high))
    }
    
    // 4. calculate offset, length and capacity and return the slice
    return Slice{
        array:    v,
        offset:   low,
        length:   high - low,
        capacity: v.Len() - low,
    }
}

func (s Slice) Get(x int) string {
    // 1. Check if x is in range.
    if x < 0 || x >= s.length {
        panic(fmt.Sprintf("index out of range [%d] with length %d", x, s.length))
    }

    // 2. Retrieve the element.
    return s.array.Index(s.offset + x).String()
}

func (s Slice) Set(x int, value string) {
    // 1. Check if x is in range.
    if x < 0 || x >= s.length {
        panic(fmt.Sprintf("index out of range [%d] with length %d", x, s.length))
    }

    // 2. Set the element value.
    s.array.Index(s.offset + x).SetString(value)
}

func (s Slice) String() string {
    out := "["
    for i := 0; i < s.length; i++ {
        if i > 0 {
            // add a space between elements
            out += " "
        }
        out += s.Get(i)
    }
    return out + "]"
}

As you can see, our slice and the real slice behave identically.

Summary

That’s it for this article, thanks for reading!

Hopefully you now have a solid understanding of:

• The difference between an array and a slice.
• How slices use arrays for storage.
• Why changing an element in a slice is reflected in the backing array (and vice versa).
• The way offset is used to access elements in a backing array.

In the next article we will take a look at the built-in append and copy functions.

More resources

• This series of articles covers similar ground to Rob Pike’s post on slices on the Go blog, it might be worthwhile to check that out as well.