Composable HTTP Handlers using generics

Implementing HTTP handlers in Go can be a bit of a chore. Most handlers will have to:

• Retrieve request data from parameters, body, headers etc.
• Validate this data: The “shape” of the data is often validated as soon as possible.
• Call out to business logic or a data store for further validation, processing and/or querying.
• Format responses for successful requests.
• Handle errors by sending different responses.

Even if you move this functionality to dedicated functions it will still get quite repetitive, handlers need to call those functions and deal with errors.

If you have an unwieldable number of endpoints, it’s probably worth making these repetitive patterns explicit.

As far as I know there are two common ways to do this:

1. Generate them based on a spec, using oapi-codegen for example.
2. Generalize them using Go code.

This article will focus on generalizing these handlers using generics.

My ideal handler

Ideally, a HTTP handler only deals with “web” concerns: HTTP headers, cookies, status codes etc.

Business logic, database queries or external service calls are delegated to other functions or methods. I’ll call them “target functions” in this article.

This means that the HTTP handler is essentially a translation or mapping function:

• Translate HTTP requests to types that a target function can deal with.
• Translate target function results or errors to HTTP responses.

In a diagram this would look like this:

I’d like these handlers to be:

• Composable: Work together with the existing middleware and routers.
• Flexible when translating types to/from requests and responses.
• More of a pattern than a library. Drop it in and adapt it as needed per project.

Non-Generic HTTP handler

Let’s begin by looking at a handler that calls a target function without involving generics.

For our example, we’ll imagine an endpoint that will create notes. The endpoint will accept requests with a JSON body like this:

{
  "note": "Hello world!"
}

And return a response with status code 201 and a JSON body containing the note and an assigned ID:

{
  "id": 123,
  "note": "Hello world!"
}

We will define two Go structs for the new note and the created note.

package main

type NewNote struct {
  Note string `json:"note"`
}

type Note struct {
  ID   int    `json:"id"`
  Note string `json:"note"`
}

Our target function will typically be defined as a method on a service of some sort and will have a signature that looks as follows:

func (s *Service) CreateNote(ctx context.Context, n NewNote) (Note, error) {
  // ...
}

How this CreateNote method works is not the concern of the HTTP handler, but you can imagine it will store the note in some kind of database, the database might also generate an ID.

In my experience, the HTTP handler will be defined on some kind of “server” struct, which holds a reference to the service (sometimes with an interface inbetween).

The example below contains a fairly rough implementation of such a HTTP Handler in server.go.

package main

import (
	"encoding/json"
	"log"
	"net/http"
)

type Server struct {
	svc *Service
}

func (s *Server) CreateNoteHandler(w http.ResponseWriter, r *http.Request) {
	var in NewNote

	// Retrieve data from request.
	err := json.NewDecoder(r.Body).Decode(&in)
	if err != nil {
		// Format error response
		http.Error(w, "invalid json", http.StatusBadRequest)
		return
	}

	// Call out to service.
	out, err := s.svc.CreateNote(r.Context(), in)
	if err != nil {
		// Format error response
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}

	// Format and write response
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusCreated)
	err = json.NewEncoder(w).Encode(out)
	if err != nil {
		log.Printf("failed to encode created note: %v", err)
		return
	}
}

package main

import (
	"context"
	"fmt"
)

// Service is responsible for managing notes.
// All behaviour is hardcoded for demo purposes.
type Service struct{}

func (s *Service) CreateNote(ctx context.Context, n NewNote) (Note, error) {
	fmt.Printf("CreateNote called: %+v\n", n)
	return Note{
		ID:   1203,
		Note: n.Note,
	}, nil
}

package main

type NewNote struct {
	Note string `json:"note"`
}

type Note struct {
	ID   int    `json:"id"`
	Note string `json:"note"`
}

package main

import (
	"fmt"
	"io"
	"log"
	"net/http/httptest"
	"strings"
)

func main() {
	server := &Server{
		svc: &Service{},
	}

	rr := httptest.NewRecorder()

	bdy := strings.NewReader(`{"note": "Hello world!"}`)
	req := httptest.NewRequest("GET", "/", bdy)

	server.CreateNoteHandler(rr, req)

	res := rr.Result()
	defer res.Body.Close()

	b, err := io.ReadAll(res.Body)
	if err != nil {
		log.Fatalf("failed to read response body: %v", err)
	}

	fmt.Printf("response status: %d, response body: %s\n", res.StatusCode, b)
}

You can see that some of the concerns listed in this article’s intro are dealt with in the HTTP handler.

In non-demo code you would probably move some functionality to helper methods, but you will still need to call these from the handler.

Suppose we want to add a new endpoint that supports updating the notes. This requires a new target function on the Service:

func (s *Service) UpdateNote(ctx context.Context, n Note) (Note, error) {
	// ...
}

But this in turn will also require a new HTTP handler that again decodes JSON, deals with errors, calls the service, encodes JSON etc.

Now, if you have a handful of endpoints this is not really an issue, just copy-paste and go on with your day.

But at some point the number of endpoints will make dealing with the HTTP handlers annoying: Making changes will take more effort and it’s easy for inconsistencies to sneak in.

Are you or your team struggling with aspects of Go?

Book a 30 minute check-in with me.

I can help with questions like:

• How should we structure our Go web app?
• Is this package a good choice for us?
• How do I get my team up to speed quickly?
• Can I pick your brain on {subject}?

Learn more

Looking forward to meeting you.

- Willem

Generic HTTP handler

We can make things a bit less tedious by using generic HTTP handlers.

The Handle function

One “trick” we will use is functions as values. Functions in Go can be passed around just like other values, as long as you define variables of the right type.

We will create a Handle function that:

1. Accepts a target function as input.
2. Outputs a HTTP handler.

In Go code this look would like this:

func Handle(f TargetFunc) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		// TODO: Implement handler and call f.
	})
}

This Handle function:

• Accepts a function f which is our target function (we’ll look at the types in a second).
• Returns a function that is converted to the http.HandlerFunc type. Which is an implementation of the http.Handler interface.

This gives us the design for the function, so what about the TargetFunc type?

Generic target function

We want a function type that allows us to match target functions such as the CreateNote and UpdateNote methods we saw earlier:

By defining a generic target function type we can make the Handle function work for all kinds of target functions. The syntax for this looks as follows:

type TargetFunc[In any, Out any] func(context.Context, In) (Out, error)

In and Out are type parameters. The any indicates that there are no restraints on the types, all types can be used in their places.

To use the TargetFunc type in Handle, we will need to make Handle aware of these type parameters as well:

func Handle[In any, Out any](f TargetFunc[In, Out]) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		// TODO: Implement handler and call f.
	})
}

Handler function

With the target function type done, we can implement the handler function.

We can copy paste the contents of our earlier non-generic HTTP handler and replace:

• The hardcoded type with the In type parameter
• The service method call with the target function call.

func Handle[In any, Out any](f TargetFunc[In, Out]) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		var in In

		// ...

		// Call out to target function
		out, err := f(r.Context(), in)

		// ...
	})
}

See the demo below for the entire function.

To create the handler for creating notes, we can now call Handle(svc.CreateNote).

But the big thing is, this will now work for any function that matches the TargetFunc signature.

Demo

This is pretty powerful, we can now use this single Handle function to create handlers for any endpoint that deals with JSON requests and responses.

The demo below shows the creation of both a CreateNote and UpdateNote handler using the generic Handle function.

package main

import (
	"context"
	"encoding/json"
	"log"
	"net/http"
)

type TargetFunc[In any, Out any] func(context.Context, In) (Out, error)

func Handle[In any, Out any](f TargetFunc[In, Out]) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		var in In

		// Retrieve data from request.
		err := json.NewDecoder(r.Body).Decode(&in)
		if err != nil {
			// Format error response
			http.Error(w, "invalid json", http.StatusBadRequest)
			return
		}

		// Call out to target function
		out, err := f(r.Context(), in)
		if err != nil {
			// Format error response
			http.Error(w, err.Error(), http.StatusBadRequest)
			return
		}

		// Format and write response
		w.Header().Set("Content-Type", "application/json")
		w.WriteHeader(http.StatusCreated)
		err = json.NewEncoder(w).Encode(out)
		if err != nil {
			log.Printf("failed to encode created note: %v", err)
			return
		}
	})
}

package main

import (
	"context"
  "fmt"
)

// Service is responsible for managing notes.
// All behaviour is hardcoded for demo purposes.
type Service struct{}

func (s *Service) CreateNote(ctx context.Context, n NewNote) (Note, error) {
	fmt.Printf("CreateNote called: %+v\n", n)
	return Note{
		ID:   1203,
		Note: n.Note,
	}, nil
}

func (s *Service) UpdateNote(ctx context.Context, n Note) (Note, error) {
	fmt.Printf("UpdateNote called: %+v\n", n)
  return n, nil
}

package main

type NewNote struct {
	Note string `json:"note"`
}

type Note struct {
	ID   int    `json:"id"`
	Note string `json:"note"`
}

package main

import (
	"fmt"
	"io"
	"log"
	"net/http"
	"net/http/httptest"
	"strings"
)

func main() {
	svc := &Service{}

	mux := &http.ServeMux{}
	mux.Handle("POST /notes", Handle(svc.CreateNote))
	mux.Handle("PUT /notes/{noteID}", Handle(svc.UpdateNote))

	requests := []struct {
		method string
		url    string
		data   string
	}{
		{http.MethodPost, "/notes", `{"note": "Hello world!"}`},
		{http.MethodPut, "/notes/1023", `{"note": "Updated content!"}`},
	}

	for _, r := range requests {
		rr := httptest.NewRecorder()
		req := httptest.NewRequest(r.method, r.url, strings.NewReader(r.data))

		mux.ServeHTTP(rr, req)

		res := rr.Result()
		defer res.Body.Close()

		b, err := io.ReadAll(res.Body)
		if err != nil {
			log.Fatalf("failed to read response body: %v", err)
		}

		fmt.Printf("response status: %d, response body: %s\n", res.StatusCode, b)
	}
}

Further improvements

Depending on what your application needs you can take the generic handler in different directions. Below I list some ideas that came to mind.

You can implement them in different ways depending on needs and taste:

• As parameters on Handle.
• As functional options so the caller can overwrite default values.
• As variants functions. For example: HandleInput and HandleOutput for endpoints that don’t have output or input data.

Variable status codes

In the above demo, the status code for the PUT request is 201, this is technically wrong since no new resource has been created.

Mapping request headers, urls etc

Above we only map request and response bodies using the encoding/json package. In a real application you will probably want to map other parts of the request as well.

There are packages like gorilla/schema, ggicci/httpin and go-playground/form that allow you to map parts of requests to structs using struct tags.

If you’re dealing with complex requests you could inject some kind of generic constructor function into the Handle function:

type ConstructorFunc[In any](r *http.Request)(In, error)

Alternatively, you could define an interface like this directly on the input type:

type RequestMapper interface {
	MapRequest(req *http.Request) error
}

And then call it in the handler function by first converting your input type to any and then doing a type assertion:

m, ok := any(v).(RequestMapper)
if ok {
	err = m.MapRequest(r)
	if err != nil {
		// handle error
	}
}

Redirect responses

For some endpoints you may want to return a redirect response when the target function is successful. You will likely need some data from the target function output to form the target url.

Injecting a function type like this could be a solution:

type RedirectFunc[Out any](Out)(string, int, error)

The string is for the URL and the int for a redirect status code.

Multiple content types

In the example above we only deal with JSON data, but a switch on the Content-Type and/or Accept headers would allow you to different content formats relatively easily.

Existing packages

After I posted this article on Reddit, some really cool packages were shared that use generics to easily build APIs:

• danielgtaylor/huma: A modern, simple, fast & flexible micro framework for building HTTP REST/RPC APIs in Go backed by OpenAPI 3 and JSON Schema.
• matt1484/chimera: Chi-based Module for Easy REST APIs.
• dkotik/htadaptor: Provides convenient generic domain logic adaptors for HTTP handlers.
• go-fuego/fuego: Go framework generating OpenAPI documentation from code.
• calvinmclean/babyapi: A Go CRUD API framework so simple a baby could use it.
• dolanor/rip: Typical REST routes using generics. Also be sure to check out the accompanying talk at Golab 2023 by Tanguy Herrmann.
• abemedia/go-don: Don is a fast & simple API framework written in Go.
• diamondburned/hrt: HRT implements a type-safe HTTP router.

If you don’t want to implement generic handlers yourself, be sure to take a look and try them out.

Summary

I hope this article gave you some practical suggestions for simplifying HTTP handlers using generics.

Let me know on Twitter or Mastodon if you have any comments or suggestions. Links can be found below.