HTTP, JSON and APIs · Prasanta Kumar Behera

Call REST APIs safely from Go: an http.Client with timeouts, requests tied to a context, JSON decoding into structs, status-code checks, retries with backoff for the errors worth retrying, custom TLS, and the AWS and Google Cloud SDKs.

A client with a timeout, a request with a context

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "net/http"
    "os"
    "time"
)

type Release struct {
    TagName     string    `json:"tag_name"`
    PublishedAt time.Time `json:"published_at"`
}

func latestRelease(ctx context.Context, c *http.Client, repo string) (Release, error) {
    url := "https://api.github.com/repos/" + repo + "/releases/latest"
    req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
    if err != nil {
        return Release{}, err
    }
    req.Header.Set("Accept", "application/vnd.github+json")

    resp, err := c.Do(req)
    if err != nil {
        return Release{}, fmt.Errorf("get %s: %w", url, err)
    }
    defer resp.Body.Close()
    if resp.StatusCode != http.StatusOK {
        return Release{}, fmt.Errorf("get %s: unexpected status %s", url, resp.Status)
    }

    var r Release
    if err := json.NewDecoder(resp.Body).Decode(&r); err != nil {
        return Release{}, fmt.Errorf("decode release: %w", err)
    }
    return r, nil
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
    defer cancel()
    client := &http.Client{Timeout: 10 * time.Second}

    r, err := latestRelease(ctx, client, "kubernetes/kubernetes")
    if err != nil {
        fmt.Fprintln(os.Stderr, "error:", err)
        os.Exit(1)
    }
    fmt.Printf("%s published %s\n", r.TagName, r.PublishedAt.Format("2006-01-02"))
}

Three habits in that code:

Timeouts on the client and a context on the request.
Check the status code: a 500 is a successful HTTP round trip, not a Go error.
Close the body and decode with a json.Decoder straight from the stream.

Calling an API is like phoning a shop. Set a timer before you call (timeout), check what they actually said before writing it down (status code), and hang up when you're done (close the body), or the phone line stays busy for the next call.

JSON details that bite

Only exported fields are decoded; tags map JSON names to Go fields.
Unknown JSON fields are ignored by default; dec.DisallowUnknownFields() makes them errors (useful for config files).
For fields you don't know in advance, decode into map[string]any or json.RawMessage and handle later.
Times decode from RFC 3339 strings into time.Time. Go formats dates with the reference time 2006-01-02 15:04:05.

Retries with backoff

Retry only what can succeed later: network errors, 429, 502/503/504. Use exponential backoff with jitter and a limit, and respect the context:

package main

import (
    "context"
    "errors"
    "fmt"
    "math/rand"
    "time"
)

var errRetryable = errors.New("retryable")

func withRetry(ctx context.Context, attempts int, fn func() error) error {
    backoff := 500 * time.Millisecond
    var err error
    for i := 1; i <= attempts; i++ {
        if err = fn(); err == nil || !errors.Is(err, errRetryable) {
            return err
        }
        sleep := backoff + time.Duration(rand.Int63n(int64(backoff/2))) // jitter
        select {
        case <-time.After(sleep):
        case <-ctx.Done():
            return fmt.Errorf("giving up after %d attempts: %w", i, ctx.Err())
        }
        backoff *= 2
    }
    return fmt.Errorf("giving up after %d attempts: %w", attempts, err)
}

func main() {
    calls := 0
    err := withRetry(context.Background(), 4, func() error {
        calls++
        if calls < 3 {
            return fmt.Errorf("status 503: %w", errRetryable)
        }
        return nil
    })
    fmt.Println("calls:", calls, "err:", err)
}

Only retry idempotent operations (GET, PUT with the full object, DELETE) unless the API offers idempotency keys.

Custom TLS: internal CAs

pool, _ := x509.SystemCertPool()
pem, _ := os.ReadFile("/etc/pki/internal-ca.pem")
pool.AppendCertsFromPEM(pem)
client := &http.Client{
    Timeout:   10 * time.Second,
    Transport: &http.Transport{TLSClientConfig: &tls.Config{RootCAs: pool, MinVersion: tls.VersionTLS12}},
}

Add your CA instead of setting InsecureSkipVerify; that flag turns TLS into encryption without identity.

Cloud SDKs

Cloud	Package	Credentials
AWS	`github.com/aws/aws-sdk-go-v2/config` + service packages (`service/ec2`, `service/eks`)	`config.LoadDefaultConfig(ctx)`: env, profiles, IRSA/Pod Identity, instance roles
Google Cloud	`cloud.google.com/go/...` (e.g. `container/apiv1`)	Application Default Credentials: gcloud login, Workload Identity

Both find credentials automatically in the standard places, so the same binary works on a laptop, in CI with OIDC federation, and in a pod with workload identity. Never embed keys.

Try it: a release checker

Run the program above for kubernetes/kubernetes and argoproj/argo-cd.
Point it at https://httpbin.org/status/503 (or a local server returning 503) and wrap the call in withRetry.
Set the context timeout to 1 second and point it at https://httpbin.org/delay/5; read the error.
Add -o json output and pipe it into jq.
If you have AWS or Google Cloud access, list clusters (EKS ListClusters or GKE ListClusters) with the SDK and default credentials.

Going deeper: being a good API client

Reuse one http.Client (it pools connections); don't create one per request.
Respect Retry-After headers and rate limits; add a client-side rate limiter (golang.org/x/time/rate) for bulk jobs.
Set a clear User-Agent (kaudit/1.2.0) so API owners can find you in their logs.

Recap

Always a client timeout and a request context.
Check status codes, close bodies, decode JSON into tagged structs.
Retry only transient failures, with backoff, jitter, limits and the context.
Trust internal CAs explicitly; never skip verification.
Cloud SDKs pick up credentials from the environment: no keys in code.

This site is a public version of my personal engineering knowledge hub. It intentionally excludes confidential company information and internal operational details.