Data interchange between systems plays a pivotal role. data is transferred as a sequence of bytes (e.g. sending data from one process to another, either within the same machine or on different machine). One of the most prevalent formats for this purpose is JSON (JavaScript Object Notation). JSON offers a lightweight, human-readable, and easy-to-parse structure, making it a preferred choice for data transmission and storage.
Data Transmission and Storage: Base64 encoding is commonly employed for transmitting binary data over text-based protocols, such as email attachments or JSON payloads. Golang’s efficient encoding package makes it a go-to choice for developers working on applications where data needs to be represented in a compact, ASCII-safe format.
Base64 is a binary-to-text encoding scheme that represents binary data using a set of 64 ASCII characters. This allows binary data to be transmitted and stored as text, making it ideal for scenarios where the original binary format may not be supported.
In the domain of Go programming, efficient handling of JSON data is facilitated by built-in encoding and decoding functionalities to mapping between the Go types(structs) and JSON in the serialization format by standard library encoding/json
Understanding Serialization and Deserialization:
Serialization (marshaling):
Serialization is the process of taking a data structure in your language and converting it into a format suitable for transmission or storage which might be a sequence of bytes.
Encode (Serialization , Marshal) = structs -> [ ]bytes
Deserialization (unmarshalling):
Deserialization is the process of taking a sequence of bytes, and converting it back into a data structure in your language.
Decode (Deserialization , Unmarshal) = [ ]bytes -> structs
Examples Of JSON Encoding and Decoding
JSON Encoding in Go Example:
The Marshal function provide developers to convert Go objects into their JSON representation. Consider the following example:
package main
import (
"encoding/json"
"fmt"
)
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
}
func main() {
person := Person{Name: "John Doe", Age: 30}
jsonData, err := json.Marshal(person)
if err != nil {
fmt.Println("Error:", err)
return
}
fmt.Println(string(jsonData))
}
In this example, the Person struct is marshaled into its JSON representation using json.Marshal().
Tags such as json:"name" provide hints to the encoder about how to name the JSON fields. Running this program would yield JSON output representing the Person object.
JSON Decoding in Go Example:
On the flip side, decoding JSON data into Go structures is achieved through the Unmarshal function provided by the encoding/json package. This function populates a Go data structure from JSON data. Consider the following example:
package main
import (
"encoding/json"
"fmt"
)
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
}
func main() {
jsonData := []byte(`{"name":"Alice","age":25}`)
var person Person
err := json.Unmarshal(jsonData, &person)
if err != nil {
fmt.Println("Error:", err)
return
}
fmt.Println(person)
}
In this example, the JSON data representing a person is unmarshaled into the Person struct using json.Unmarshal(). The resulting person object contains the decoded data.
Converting HTTP Response Body to a Go Structure
When reading from HTTP resp.Body we get a sequence of bytes that needs to be converted into a Go structure.
resp.Body is of type io.ReadCloser, which is a combination of two interfaces io.Reader and io.Closer and is used by Go's HTTP client to:
read data from a stream
close a stream
type Closer interface {
Close() error
}
The io.Closer is an interface: meaning the concrete type behind it should implement it and it should have a method called Close that returns an error.
type Reader interface {
Read(p []byte) (n int, err error)
}
The io.Reader is an interface: meaning the concrete type behind it should implement it and it should have a method called Read that gets a slice of bytes to fill, and returns two things:
How many bytes it managed to fill
Error if there was any
Let's get a real example by fetches JSON API Endpoint from a specified URL using an HTTP GET request. It defines a struct named Todo representing a todo item. After fetching the JSON response, it reads the response body into a byte slice. Next, it uses the json.Unmarshal() function to deserialize the JSON data into a Todo struct instance. Finally, it prints the deserialized data, demonstrating the process of decoding JSON data into a Go struct.
there are two ways to achieve deserializing the JSON data
json.Unmarshal
json.NewDecoder
json.Unmarshal
using json.Unmarshal([]bytes, v) with the help of io.ReadAll(resp.Body) to read the response body into a byte slice
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"net/http"
)
type Todo struct {
UserID int `json:"userId"`
ID int `json:"id"`
Title string `json:"title"`
Completed bool `json:"completed"`
}
func main() {
url := "https://jsonplaceholder.typicode.com/todos/1"
resp, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusOK {
todoItem := Todo{}
// printComingJsonData()
// ### Convert Json back To Go struct
// ### 1 first way of deserializing decoding unmarshaling
bodyBytes, err := io.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
fmt.Printf("print incoming json body: (JSON) \n %+v \n", string(bodyBytes))
fmt.Println("----------------------------------------------------")
json.Unmarshal(bodyBytes, &todoItem)
fmt.Printf("Unmarshal data from API JSON to Go Struct: (JSON) -> (Go Struct) \n %+v \n", todoItem)
fmt.Println("----------------------------------------------------")
}
}
output:
print incoming json body: (JSON)
{
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
}
----------------------------------------------------
Unmarshal data from API JSON to Go Struct: (JSON) -> (Go Struct)
{UserID:1 ID:1 Title:delectus aut autem Completed:false}
----------------------------------------------------
json.NewDecoder
creates a new JSON decoder that initialized with the HTTP response body resp.Body. The Decode() method of the decoder is then invoked. This method reads JSON data from the input stream (in this case, the HTTP response body) and decodes it into the provided the struct. Essentially, it achieves the same result as json.Unmarshal() by decoding JSON data into a Go struct, but it provides more flexibility, especially when dealing with large JSON payloads or streaming data.
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
)
type Todo struct {
UserID int `json:"userId"`
ID int `json:"id"`
Title string `json:"title"`
Completed bool `json:"completed"`
}
func main() {
url := "https://jsonplaceholder.typicode.com/todos/1"
resp, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusOK {
todoItem := Todo{}
// ### 2 second way of deserializing decoding unmarshaling
json.NewDecoder(resp.Body).Decode(&todoItem)
fmt.Printf("Unmarshal data from API JSON to Go Struct: (JSON) -> (Go Struct) \n %+v \n", todoItem)
fmt.Println("----------------------------------------------------")
}
}
output:
─╯
Unmarshal data from API JSON to Go Struct: (JSON) -> (Go Struct)
{UserID:1 ID:1 Title:delectus aut autem Completed:false}
----------------------------------------------------
Converting Go Structure to JSON
Once we've successfully transformed JSON data from an API into Go structs through unmarshaling, let's explore how we can convert Go structs back into JSON format to serve as an HTTP response.
json.Marshal & MarshalIndent
json.Marshal converts the struct into a JSON-formatted byte slice. This process is known as JSON encoding or serialization. The resulting JSON data can be transmitted over networks, stored in files.
json.MarshalIndent(v any, prefix, indent string) is similar to Marshal but adds indentation and newlines to make the JSON output more human-readable.This is particularly useful for debugging and readability purposes, as it presents the JSON data in a more structured and visually appealing manner.
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
)
type Todo struct {
UserID int `json:"userId"`
ID int `json:"id"`
Title string `json:"title"`
Completed bool `json:"completed"`
}
func main() {
url := "https://jsonplaceholder.typicode.com/todos/1"
resp, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusOK {
todoItem := Todo{}
// ### 2 second way of deserializing decoding unmarshaling
json.NewDecoder(resp.Body).Decode(&todoItem)
fmt.Printf("Unmarshal data from API JSON to Go Struct: (JSON) -> (Go Struct) \n %+v \n", todoItem)
fmt.Println("----------------------------------------------------")
// --------------------------------------------
// ### Convert Structs back To Json
jsonData, err := json.Marshal(&todoItem)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Marshal Go Struct To Json: (Go Struct) -> (JSON) \n %+v \n", string(jsonData))
fmt.Println("----------------------------------------------------")
// #### Marshal with indent
jsonData2, err := json.MarshalIndent(&todoItem, "", "\t")
if err != nil {
log.Fatal(err)
}
fmt.Printf("MarshalIndent (Go Struct) -> (JSON) \n %+v \n", string(jsonData2))
fmt.Println("----------------------------------------------------")
}
}
output :
Unmarshal data from API JSON to Go Struct: (JSON) -> (Go Struct)
{UserID:1 ID:1 Title:delectus aut autem Completed:false}
----------------------------------------------------
Marshal Go Struct To Json: (Go Struct) -> (JSON)
{"userId":1,"id":1,"title":"delectus aut autem","completed":false}
----------------------------------------------------
MarshalIndent (Go Struct) -> (JSON)
{
"userId": 1,
"id": 1,
"title": "delectus aut autem",
"completed": false
}
----------------------------------------------------
Excluding struct fields from output
you can control which fields of a struct are included in the encoded JSON output by using struct tags. Struct tags are metadata attached to struct fields, and they can specify various properties, including how the field should be encoded or decoded.
Some fields may be sensitive, so in these circumstances, we would like the JSON encoder to ignore the field entirely—even when it is set. This is done using the special value - as the value argument to a json: struct tag.
This example ignoring userId and Id fields to be shown in the json response
type Todo struct {
UserID int `json:"-"`
ID int `json:"-"`
Title string `json:"title"`
Completed bool `json:"completed"`
}
output:
{
"title": "delectus aut autem",
"completed": false
}
Omitting fields with empty values from output
you can omit fields with empty values from the encoded JSON output by using the omitempty option in struct tags. This option tells the JSON encoder to skip fields that have zero values (such as empty strings, zero integers, nil pointers, etc.) during encoding.
so we will make the title empty to see how will the omit option tag output
type Todo struct {
UserID int `json:"-"`
ID int `json:"-"`
Title string `json:"title,omitempty"`
Completed bool `json:"completed"`
}
output:
{
"completed": false
}
there's no title field in the output