Consuming REST APIs in Go - HTTP GET, PUT, POST and DELETE

Dec 20, 2019
golang rest api

In a previous post - Go REST API, we saw how to build a simple REST service in Golang. In this post, we will see how to consume an external API in Go - As developers, we often write applications/programs that fetch information from external APIs, and it’s essential to understand how to make HTTP requests using the libraries the language provides.

Introduction

Since we are going to learn how to consume a REST API, the quickest way to get started is to use a public REST API for our consumption. There are lots of free public APIs available, but my favourite one is jsonplaceholder. The APIs have a very simple schema with familiar entities (posts, comments, todos), and it doesn’t enforce authentication, so we don’t have to spend time for sign-up, apiKey generation, etc.

In this tutorial, we will see how to get the API response as a string and also how to map the response to a struct. A sample JSON representation of the todo resource looks like the following:

{
    "userId": 2,
    "id": 40,
    "title": "totam atque quo nesciunt",
    "completed": true
  }

The equivalent go struct looks like the one below:

// Todo struct
type Todo struct {
	UserID    int    `json:"userId"`
	ID        int    `json:"id"`
	Title     string `json:"title"`
	Completed bool   `json:"completed"`
}

I manually created this struct representing the information returned in the API response. However, this could be tedious task if the API response has 100’s of fields. Worry not 😄 We can use tools like json-to-go to automatically generate the Go struct definition, given a JSON response.

Imports

Now that we have our environment setup, it’s time to write some code and get our hands dirty. Let’s start by importing the set of packages mentioned in the code snippet below:

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
)

bytes - This package implements functions for the manipulation of byte slices (similar to the string package that provides methods for manipulating strings)

encoding/json - This package contains methods that are used to convert Go types to JSON and vice-versa (This conversion is called as encode/decode in Go, serialization/de-serialization or marshall/unmarshall in other languages).

fmt - This package implements formatted I/O functions similar to scanf and printf in C

io/ioutil - This package implements some I/O utility functions (For instance, reading the contents of a file , reading from a io.Reader, etc)

log - Has methods for formatting and printing log messages.

net/http - Contains methods for performing operations over HTTP. It provides HTTP server and client implementations and has abstractions for HTTP request, response, headers, etc.

HTTP Get

It’s time to get our hands dirty by implementing a HTTP Get request. The http package exposes a convenient Get method which can be used to make HTTP Get requests Let’s try to get the todo with id 1 using the code below:

func get() {
    fmt.Println("1. Performing Http Get...")
    resp, err := http.Get("https://jsonplaceholder.typicode.com/todos/1")
    if err != nil {
        log.Fatalln(err)
    }

    defer resp.Body.Close()
    bodyBytes, _ := ioutil.ReadAll(resp.Body)

    // Convert response body to string
    bodyString := string(bodyBytes)
    fmt.Println("API Response as String:\n" + bodyString)

    // Convert response body to Todo struct
    var todoStruct Todo
    json.Unmarshal(bodyBytes, &todoStruct)
    fmt.Printf("API Response as struct %+v\n", todoStruct)
}

The Get method is very simple - takes the URL as the argument, and returns a response and error. For successful API calls, err will be non-nil. However, if there was an error while making the API call (which means err will be equal to nil, we log and exit the program using the log.Fatalln() method.

Fatalln() is equivalent to calling Println() followed by os.Exit(1). The if condition below will have the same effect as the one in the code snippet above:

if err != nil {
    fmt.Println(err)
    os.Exit(1)
}

If you are calling the API as part of a bigger application, exiting the program is not a good idea - In that case, just log the error, return it to the caller and let the caller handle the rest.

The defer ensures that the resp.Body.Close() is executed at the end of method . Now, it is extremely important to close the response body. Not doing it will result in resource leaks, as discussed in this Stackoverflow post.

The ioutil.ReadAll method returns the response body as a []byte(a slice of bytes) and we can convert this slice of bytes to a string or the Todo struct we had defined earlier in this article.

HTTP Post

Now , let’s try to make a POST request, which is normally used to create a new resource on the server.

func post() {
    fmt.Println("2. Performing Http Post...")
    todo := Todo{1, 2, "lorem ipsum dolor sit amet", true}
    jsonReq, err := json.Marshal(todo)
    resp, err := http.Post("https://jsonplaceholder.typicode.com/todos", "application/json; charset=utf-8", bytes.NewBuffer(jsonReq))
    if err != nil {
        log.Fatalln(err)
    }

    defer resp.Body.Close()
    bodyBytes, _ := ioutil.ReadAll(resp.Body)

    // Convert response body to string
    bodyString := string(bodyBytes)
    fmt.Println(bodyString)

    // Convert response body to Todo struct
    var todoStruct Todo
    json.Unmarshal(bodyBytes, &todoStruct)
    fmt.Printf("%+v\n", todoStruct)
}

We start by initializing a todo struct, with some random values, and converting the struct to a slice of bytes ([]byte) using the json.Marshall() method - The resulting bytes slice is stored in the jsonReq variable.

The Post method takes 3 arguments - The Url of the API, the content-type, which is application/json in our case , and an instance of io.Reader. In order to use the jsonReq with the post method, we need to convert that to an instance of io.Reader, and this is where the bytes.NewBuffer() method comes into picture

The bytes.NewBuffer() takes the jsonReq bytes slice as input and returns a Buffer initialized from the contents of our jsonReq. As we can see from the buffer.go source, buffer has a read method which means it implements the io.Reader interface and it can be passed as an argument to the http.Post() method call.

The subsequent lines of code to handle errors, closing the response body, converting the response to struct/string is exactly the same as the Get method.

HTTP Put

This section describes how to make a PUT request, which is normally used to modify a resource on the server. The code to make PUT, DELETE requests is slightly different from GET/POST calls - This is because the http .Client interface has convenient high-level methods for Get and Put but not for other http methods like PUT, DELETE, etc. The reasoning behind this is that the go developers felt it’s unrealistic to provide high-level methods for all HTTP verbs, and decided to draw the line at Get and Post.

So, what does this mean for us? - Since there is no direct method available for PUT, We have to use the http.NewRequest() method and pass the http method type as the argument (PUT in our case). All in all, this results in a couple of more lines of code compared to the PUT/POST methods (Yeah, I know. Big deal, right? 😉)

func put() {
    fmt.Println("3. Performing Http Put...")
    todo := Todo{1, 2, "lorem ipsum dolor sit amet", true}
    jsonReq, err := json.Marshal(todo)
    req, err := http.NewRequest(http.MethodPut, "https://jsonplaceholder.typicode.com/todos/1", bytes.NewBuffer(jsonReq))
    req.Header.Set("Content-Type", "application/json; charset=utf-8")
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        log.Fatalln(err)
    }

    defer resp.Body.Close()
    bodyBytes, _ := ioutil.ReadAll(resp.Body)

    // Convert response body to string
    bodyString := string(bodyBytes)
    fmt.Println(bodyString)

    // Convert response body to Todo struct
    var todoStruct Todo
    json.Unmarshal(bodyBytes, &todoStruct)
    fmt.Printf("API Response as struct:\n%+v\n", todoStruct)
}

HTTP Delete

The Delete API call is very similar to PUT, except that we need to pass DELETE as the argument to http .NewRequest() method. I hope the below code snippet is self-explanatory, considering

func delete() {
    fmt.Println("4. Performing Http Delete...")
    todo := Todo{1, 2, "lorem ipsum dolor sit amet", true}
    jsonReq, err := json.Marshal(todo)
    req, err := http.NewRequest(http.MethodDelete, "https://jsonplaceholder.typicode.com/todos/1", bytes.NewBuffer(jsonReq))
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        log.Fatalln(err)
    }

    defer resp.Body.Close()
    bodyBytes, _ := ioutil.ReadAll(resp.Body)

    // Convert response body to string
    bodyString := string(bodyBytes)
    fmt.Println(bodyString)
}

One thing worth pointing out is that, for the POST/PUT/DELETE requests here, the resource will not actually be modified on the server, but will be faked as if it is.

Note:
One thing worth pointing out is that, for the POST/PUT/DELETE requests here, the resource will not actually be modified on the server, but will be faked as if it is. This is how the jsonplaceholder fake API behaves and shouldn’t be the case when we work with proper(non-fake) APIs

Conclusion

In this post, we discussed how to use the net/http package to call external APIs in Go, and I hope you found it useful. You can find the entire code on Github - Please feel free to reach out if there are any questions or suggestions. Do leave your thoughts in the Comments section below.