A little while ago, I decided to start learning Go. One of the things that I wanted to start out with was learning how to work with JSON. Coming from Python, the first thing I noticed was that working with JSON in Go is a bit more involved.
The following is what I was used to when serializing a Python dictionary into JSON:
from json import dump
d = {
'Automate the Boring Stuff with Python' : 'Al Sweigart',
'Fluent Python' : 'Luciano Ramalho',
'Learning Python' : 'Mark Lutz',
'Python Tricks' : 'Dan Bader',
}
with open('/var/tmp/dictionary.json', 'w') as f:
dump(d, f)
Or deserializing a JSON file into a Python dictionary:
from json import load
with open('/var/tmp/dictionary.json', 'r') as f:
d = load(f)
Working with JSON in Go requires a little more effort. In the following examples, I will serialize a struct into JSON and deserialize JSON into a struct:
To do this, I will be using package JSON. First, I will ‘Marshal’ a Go object into JSON and write it to disk. After that, I will ‘Unmarshal’ that JSON into a Go object again.
Marshal
Turning a Go object into JSON is done using the Marshal function. First, we start out defining the struct we will Marshal into JSON:
// The struct/embedded struct that we will turn into JSON:
type ExampleStruct struct {
Integer int
Float float64
String string
Bool bool
Slice []string
Map map[string]int
Struct SomeStruct `json:"embedded_struct"` // name the JSON field embedded_struct
}
type SomeStruct struct {
String string
IntSlice []int
Omitted string `json:"omitted,omitempty"` // omit when the field is empty
NotOmitted string
}
// Note that the struct fields must be capitalized for them to be eligible for serialization.
Notice that the struct contains some tags that are read and used by the JSON package we are using. In this example, the first tag sets the field name, and the other tag ensures the field is not considered when it is not defined. Check the Marshal function in the package to see what other tags are available.
Next, we construct the struct:
// Construct the ExampleStruct:
Example := ExampleStruct{
Integer: 100,
Float: 15.12,
String: "Word",
Bool: false,
Slice: []string{"go", "python", "c"},
Map: map[string]int{
"a": 1,
"b": 2,
"c": 3,
},
Struct: SomeStruct{
String: "abcdefg",
IntSlice: []int{1, 2, 3, 4},
},
}
// Print the struct:
fmt.Println(Example)
After constructing the Example struct we are working with, it is returned to screen:
{100 15.12 Word false [go python c] map[a:1 b:2 c:3] {abcdefg [1 2 3 4] }}
Now that we have the struct, we can Marshal it like so:
Example_marshalled, _ := json.Marshal(Example)
The Example_marshalled now contains a byte-slice. In this example, we passed the json.Marshal function a Value. If we chose to pass it a pointer (&Example), it would have produced the same output.
After Marshalling the struct into JSON, we can print it like so:
fmt.Printf("%T %v\n", Example_marshalled, Example_marshalled)
This returns the following:
[]uint8 [123 34 73 110 116 101 103 101 114 34 58 49 48 48 44 34 70 108 111 97 116 34 58 49 53 46 49 50 44 34 83 116 114 105 110 103 34 58 34 87 111 114 100 34 44 34 66 111 111 108 34 58 102 97 108 115 101 44 34 83 108 105 99 101 34 58 91 34 103 111 34 44 34 112 121 116 104 111 110 34 44 34 99 34 93 44 34 77 97 112 34 58 123 34 97 34 58 49 44 34 98 34 58 50 44 34 99 34 58 51 125 44 34 101 109 98 101 100 100 101 100 95 115 116 114 117 99 116 34 58 123 34 83 116 114 105 110 103 34 58 34 97 98 99 100 101 102 103 34 44 34 73 110 116 83 108 105 99 101 34 58 91 49 44 50 44 51 44 52 93 44 34 111 109 105 116 101 109 112 116 121 34 58 34 34 44 34 78 111 116 79 109 105 116 116 101 100 34 58 34 34 125 125]
To look at the values of this byte-slice, we can use string to turn it into something we can read:
fmt.Println(string(Example_marshalled))
This would return the following to screen:
{"Integer":100,"Float":15.12,"String":"Word","Bool":false,"Slice":["go","python","c"],"Map":{"a":1,"b":2,"c":3},"embedded_struct":{"String":"abcdefg","IntSlice":[1,2,3,4],"NotOmitted":""}}
To make it look prettier, we can turn to the json.MarshalIndent function:
prettyJSON, _ := json.MarshalIndent(Example, "", "\t") fmt.Println(string(prettyJSON))
This will return the JSON data to screen in a more readable format:
{
"Integer": 100,
"Float": 15.12,
"String": "Word",
"Bool": false,
"Slice": [
"go",
"python",
"c"
],
"Map": {
"a": 1,
"b": 2,
"c": 3
},
"embedded_struct": {
"String": "abcdefg",
"IntSlice": [
1,
2,
3,
4
],
"NotOmitted": ""
}
}
To store the JSON for later use, we can write it to a file using ioutil.WriteFile:
_ = ioutil.WriteFile("test.json", prettyJSON, 0644)
As an alternative to this approach, we can also follow a different patter. In the next example, we first create a file. After this file is created, we create a new encoder with json.NewEncoder, which is passed the file handler. We then call Encode on that and pass it the struct we want to encode:
// Alternatively, use an encoder:
f, _ := os.Create("encoder_test.json")
defer f.Close()
// pass filehandler to NewEncoder and Encode the Example struct:
_ = json.NewEncoder(f).Encode(&Example) // this writes the JSON to 'encoder_test.json'
The complete script can be found at the bottom.
Unmarshal
We previously Marshalled JSON into the test.json file. Now, we will deserialize that JSON into a Go object using the Unmarshal function.
To be able to serialize the JSON into a Go object, we first need to define the struct again:
// The struct/embedded struct that we will turn into JSON:
type ExampleStruct struct {
Integer int
Float float64
String string
Bool bool
Slice []string
Map map[string]int
Struct SomeStruct `json:"embedded_struct"`
}
// Note that the struct fields must be capitalized for them to be eligible for serialization.
type SomeStruct struct {
String string
IntSlice []int
Omitted string `json:"omitted,omitempty"`
NotOmitted string
}
// Contstruct an empty struct that will receive the JSONL
JSONdata := new(ExampleStruct)
After this, we read the content of the file:
jsonFile, err := os.Open("test.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
With all this in place, we can now Unmarshal the JSON into the struct:
// Unmarshal the JSON into the struct: _ = json.Unmarshal(byteValue, &JSONdata)
We can print the struct to screen to look at the content. And in addition to looking at the struct content, we can also use the json.MarshalIndent function again to look at a more readable representation of the JSON:
// Return the unmarshalled JSON data to screen by printing the struct: fmt.Println(JSONdata) // Make the JSON pretty: prettyJSON, _ := json.MarshalIndent(JSONdata, "", "\t") fmt.Println(string(prettyJSON))
The full script to Unmarshal the JSON is the following:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
)
// The struct/embedded struct that we will turn into JSON:
type ExampleStruct struct {
Integer int
Float float64
String string
Bool bool
Slice []string
Map map[string]int
Struct SomeStruct `json:"embedded_struct"`
}
// Note that the struct fields must be capitalized for them to be eligible for serialization.
type SomeStruct struct {
String string
IntSlice []int
Omitted string `json:"omitted,omitempty"`
NotOmitted string
}
func main() {
jsonFile, _ := os.Open("test.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
// Contstruct an empty struct that will receive the JSONL
JSONdata := new(ExampleStruct)
// Unmarshal the JSON into the struct:
_ = json.Unmarshal(byteValue, &JSONdata)
// Return the unmarshalled JSON data to screen by printing the struct:
fmt.Println(JSONdata)
// Make the JSON pretty:
prettyJSON, _ := json.MarshalIndent(JSONdata, "", "\t")
fmt.Println(string(prettyJSON))
}
Outer array
In case the JSON that you are working with is enclosed in an outer array, we can work with that in (almost) the same way. Suppose the JSON data we have been working with in this example would now be enclosed in an outer array. To deal with that, the struct definition can stay the same. What we can do to turn that JSON data into a Go object is tell our program to anticipate a slice of structs:
var JSONdata []ExampleStruct
The rest of the code would be the same:
jsonFile, _ := os.Open("outer_array_test.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
_ = json.Unmarshal(byteValue, &JSONdata)
fmt.Println(JSONdata)
Having enclosed two copies of the previously stored JSON test data in an outer array in the outer_array_test.json, this would give us the following output:
[{100 15.12 Word false [go python c] map[a:1 b:2 c:3] {abcdefg [1 2 3 4] }} {100 15.12 Word false [go python c] map[a:1 b:2 c:3] {abcdefg [1 2 3 4] }}]
Marshal example:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
)
// The struct/embedded struct that we will turn into JSON:
type ExampleStruct struct {
Integer int
Float float64
String string
Bool bool
Slice []string
Map map[string]int
Struct SomeStruct `json:"embedded_struct"` // name the JSON field embedded_struct
}
type SomeStruct struct {
String string
IntSlice []int
Omitted string `json:"omitempty"` // omit when the field is empty
NotOmitted string
}
// Note that the struct fields must be capitalized for them to be eligible for serialization.
func main() {
// Construct the ExampleStruct:
Example := ExampleStruct{
Integer: 100,
Float: 15.12,
String: "Word",
Bool: false,
Slice: []string{"go", "python", "c"},
Map: map[string]int{
"a": 1,
"b": 2,
"c": 3,
},
Struct: SomeStruct{
String: "abcdefg",
IntSlice: []int{1, 2, 3, 4},
},
}
// Print the struct:
fmt.Println(Example)
// Marshal the struct
Example_marshalled, _ := json.Marshal(Example) // also works with &Example
fmt.Printf("%T %v\n", Example_marshalled, Example_marshalled)
// Print the marshalled struct as a string(json.Marshal returns a byte-slice):
fmt.Println(string(Example_marshalled))
// Make the JSON pretty:
prettyJSON, _ := json.MarshalIndent(Example, "", "\t")
fmt.Println(string(prettyJSON))
// Write it to a file for later use:
_ = ioutil.WriteFile("test.json", prettyJSON, 0644) // this writes the JSON to 'test.json'
// Alternatively, use an encoder:
f, _ := os.Create("encoder_test.json")
defer f.Close()
// pass filehandler to NewEncoder and Encode the Example struct:
_ = json.NewEncoder(f).Encode(&Example) // this writes the JSON to 'encoder_test.json'
}
Unmarshal example:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
)
// The struct/embedded struct that we will turn into JSON:
type ExampleStruct struct {
Integer int
Float float64
String string
Bool bool
Slice []string
Map map[string]int
Struct SomeStruct `json:"embedded_struct"`
}
// Note that the struct fields must be capitalized for them to be eligible for serialization.
type SomeStruct struct {
String string
IntSlice []int
Omitted string `json:"omitted,omitempty"`
NotOmitted string
}
func main() {
// Contstruct an empty struct that will receive the JSONL
JSONdata := new(ExampleStruct)
jsonFile, _ := os.Open("test.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
// Unmarshal the JSON into the struct:
_ = json.Unmarshal(byteValue, &JSONdata)
// Return the unmarshalled JSON data to screen by printing the struct:
fmt.Println("Data stored test.json:\n\n", JSONdata, "\n\n")
// Make the JSON pretty:
prettyJSON, _ := json.MarshalIndent(JSONdata, "", "\t")
fmt.Println(string(prettyJSON))
}
Outer array example:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
)
// The struct/embedded struct that we will turn into JSON:
type ExampleStruct struct {
Integer int
Float float64
String string
Bool bool
Slice []string
Map map[string]int
Struct SomeStruct `json:"embedded_struct"`
}
// Note that the struct fields must be capitalized for them to be eligible for serialization.
type SomeStruct struct {
String string
IntSlice []int
Omitted string `json:"omitted,omitempty"`
NotOmitted string
}
func main() {
var JSONdata []ExampleStruct
jsonFile, _ := os.Open("outer_array_test.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
_ = json.Unmarshal(byteValue, &JSONdata)
fmt.Println(JSONdata)
// Make the JSON pretty:
prettyJSON, _ := json.MarshalIndent(JSONdata, "", "\t")
fmt.Println(string(prettyJSON))
}