Custom struct field tags in Golang
Structs in Golang represent one of the most common variable types and used practically everywhere, from dealing with configuration options to marshaling of JSON or XML documents using encoding/json or encoding/xml packages. Field tags are part of the struct's field definition and allow nice and easy way to store meta data about fields for many use cases (field mapping, data validation, ORM, etc).
Basics
What's interesting about structs in general? One of the most useful features of the struct is ability to specify field name mapping. It comes very handy if you deal with external services and do a lot of data transformations. Lets take a look at the following example:
type User struct {
Id int `json:"id"`
Name string `json:"name"`
Bio string `json:"about,omitempty"`
Active bool `json:"active"`
Admin bool `json:"-"`
CreatedAt time.Time `json:"created_at"`
}
In our User struct, tags are just strings following the field's type definition and enclosed in backticks. In our example we redefine field names for JSON encoding/decoding. What that means is when JSON encoder goes over the struct fields, it will use user-defined field names instead of default capitalized names. Here's the JSON output of the struct without custom tags, produced by json.Marshal call:
{
"Id": 1,
"Name": "John Doe",
"Bio": "Some Text",
"Active": true,
"Admin": false,
"CreatedAt": "2016-07-16T15:32:17.957714799Z"
}
As you can see, all fields in the example output correspond to their definitions on the User struct. Now, lets add custom JSON tags and see what happens:
{
"id": 1,
"name": "John Doe",
"about": "Some Text",
"active": true,
"created_at": "2016-07-16T15:32:17.957714799Z"
}
With custom tags we were able to reshape the output. Using json:"-" definition we tell encoder to skip the field altogether. Check out JSON and XML packages for more details and available tag options.
Roll Your Own
Now that we understand how struct tags are defined and used, lets try to make our own tag processor. To do so we need to inspect the struct and read the tag attributes. That's where the reflect package comes into play.
Pretend we're going to implement simple validation library that uses field tags to define some validation rules based on field's type. We always want to validate data before it gets saved to a database.
package main
import (
"fmt"
"reflect"
)
// Name of the struct tag used in examples
const tagName = "validate"
type User struct {
Id int `validate:"-"`
Name string `validate:"presence,min=2,max=32"`
Email string `validate:"email,required"`
}
func main() {
user := User{
Id: 1,
Name: "John Doe",
Email: "john@example",
}
// TypeOf returns the reflection Type that represents the dynamic type of variable.
// If variable is a nil interface value, TypeOf returns nil.
t := reflect.TypeOf(user)
// Get the type and kind of our user variable
fmt.Println("Type:", t.Name())
fmt.Println("Kind:", t.Kind())
// Iterate over all available fields and read the tag value
for i := 0; i < t.NumField(); i++ {
// Get the field, returns https://golang.org/pkg/reflect/#StructField
field := t.Field(i)
// Get the field tag value
tag := field.Tag.Get(tagName)
fmt.Printf("%d. %v (%v), tag: '%v'\n", i+1, field.Name, field.Type.Name(), tag)
}
}
Save and run the code. You'll see something like this:
Type: User
Kind: struct
1. Id (int), tag: '-'
2. Name (string), tag: 'presence,min=2,max=32'
3. Email (string), tag: 'email,required'
With reflect package we were able to extract basic information about our User struct, its type and kind and also list of all of its fields. As you can see, we also printed out a tag associated with each field. There's nothing magical about tags, field.Tag.Get method will return a string that matches the tag name (see docs) that we're free to do whatever we want.
To give you an idea how struct tags could be used to perform validations, i implemented a few validator types (numeric, string, email) using interface pattern. Here's the full runnable code sample:
package main
import (
"fmt"
"reflect"
"regexp"
"strings"
)
// Name of the struct tag used in examples.
const tagName = "validate"
// Regular expression to validate email address.
var mailRe = regexp.MustCompile(`\A[\w+\-.]+@[a-z\d\-]+(\.[a-z]+)*\.[a-z]+\z`)
// Generic data validator.
type Validator interface {
// Validate method performs validation and returns result and optional error.
Validate(interface{}) (bool, error)
}
// DefaultValidator does not perform any validations.
type DefaultValidator struct {
}
func (v DefaultValidator) Validate(val interface{}) (bool, error) {
return true, nil
}
// StringValidator validates string presence and/or its length.
type StringValidator struct {
Min int
Max int
}
func (v StringValidator) Validate(val interface{}) (bool, error) {
l := len(val.(string))
if l == 0 {
return false, fmt.Errorf("cannot be blank")
}
if l < v.Min {
return false, fmt.Errorf("should be at least %v chars long", v.Min)
}
if v.Max >= v.Min && l > v.Max {
return false, fmt.Errorf("should be less than %v chars long", v.Max)
}
return true, nil
}
// NumberValidator performs numerical value validation.
// Its limited to int type for simplicity.
type NumberValidator struct {
Min int
Max int
}
func (v NumberValidator) Validate(val interface{}) (bool, error) {
num := val.(int)
if num < v.Min {
return false, fmt.Errorf("should be greater than %v", v.Min)
}
if v.Max >= v.Min && num > v.Max {
return false, fmt.Errorf("should be less than %v", v.Max)
}
return true, nil
}
// EmailValidator checks if string is a valid email address.
type EmailValidator struct {
}
func (v EmailValidator) Validate(val interface{}) (bool, error) {
if !mailRe.MatchString(val.(string)) {
return false, fmt.Errorf("is not a valid email address")
}
return true, nil
}
// Returns validator struct corresponding to validation type
func getValidatorFromTag(tag string) Validator {
args := strings.Split(tag, ",")
switch args[0] {
case "number":
validator := NumberValidator{}
fmt.Sscanf(strings.Join(args[1:], ","), "min=%d,max=%d", &validator.Min, &validator.Max)
return validator
case "string":
validator := StringValidator{}
fmt.Sscanf(strings.Join(args[1:], ","), "min=%d,max=%d", &validator.Min, &validator.Max)
return validator
case "email":
return EmailValidator{}
}
return DefaultValidator{}
}
// Performs actual data validation using validator definitions on the struct
func validateStruct(s interface{}) []error {
errs := []error{}
// ValueOf returns a Value representing the run-time data
v := reflect.ValueOf(s)
for i := 0; i < v.NumField(); i++ {
// Get the field tag value
tag := v.Type().Field(i).Tag.Get(tagName)
// Skip if tag is not defined or ignored
if tag == "" || tag == "-" {
continue
}
// Get a validator that corresponds to a tag
validator := getValidatorFromTag(tag)
// Perform validation
valid, err := validator.Validate(v.Field(i).Interface())
// Append error to results
if !valid && err != nil {
errs = append(errs, fmt.Errorf("%s %s", v.Type().Field(i).Name, err.Error()))
}
}
return errs
}
type User struct {
Id int `validate:"number,min=1,max=1000"`
Name string `validate:"string,min=2,max=10"`
Bio string `validate:"string"`
Email string `validate:"email"`
}
func main() {
user := User{
Id: 0,
Name: "superlongstring",
Bio: "",
Email: "foobar",
}
fmt.Println("Errors:")
for i, err := range validateStruct(user) {
fmt.Printf("\t%d. %s\n", i+1, err.Error())
}
}
On User struct we define a validation on Id field that should check if the value is in proper range (1-1000). Name field value is a string and validation should check the length. Bio field value is a string and we just require it to be present. And finally, Email field value should be a legit email address (at least formatted as email). Example User struct's field are all invalid and after running the code you will see the following output:
Errors:
1. Id should be greater than 1
2. Name should be less than 10 chars long
3. Bio cannot be blank
4. Email is not a valid email address
The main different between last example and the one before (where we used basic reflection of the type) is that we use reflect.ValueOf instead of reflectTypeOf. It's needed so we can grab the field value using v.Field(i).Interface() which gives us an interface that we can pass for validation. We can also get type of the field using v.Type().Field(i).
References
My particular example shows a basic use case, but there a plenty of other ways how to deal with interfaces and dynamic programming in Go. Here's a few open source projects that use struct tags extensively:
- gorm - Database ORM for SQLite, Postgres and MySQL.
- govalidator - Package of validators and sanitizers for strings, numerics, slices and structs.
- mgo - The MongoDB database driver for Go.
Also checkout example code in gist.