Git toolkit for Go

Heroku introduced git push heroku master deployment workflow to the world and became the staple of platform engineering back in the day. Not that developers learned about Git, which was great btw, but the overall user experience was the selling point. Just push the code and watch it getting built and shipped live, right in your terminal! It felt almost magical.

Once switching to Git, we didn't even need any additional tools to deploy applications. Heroku would just plug into the existing repo with an additional git remote (ex: git remote add heroku ...) so basically deployments were done by pushing the code into a separate repo. You could have multiple remotes as well, like staging and production, for different environments.

The magic behind all that awesomeness was powered by a fairly new (at the time) Git version control system, handling all repository operations with hooks, and entry points via SSH and HTTP protocols. Over the years I've made several implementations of the git-push workflows for internal needs, finally deciding to package it up as an open-source project (ca 2016). Enter Gitkit.

Gitkit

Project is currently hosted on Github, written in Go, and essentially provides building blocks for Git-oriented applications. For example, you can use it as a utility for testing real repository interactions, building internal static site generation pipelines, among other cases. Or anything that needs to run when you git push. Think of:

$ git remote add deploy http://deploys.mycorp.com/myapp.git # <- powered by gitkit
$ git push deploy master

Gitkit is fairly simple but extremely handy since it provides both HTTP and SSH servers to handle incoming git requests (push/pull). A few lines of configuration code is all it takes to get going:

$ go get github.com/sosedoff/gitkit

Bare minimum example:

package main

import (
  "log"
  "net/http"
  "github.com/sosedoff/gitkit"
)

func main() {
  service := gitkit.New(gitkit.Config{
    Dir:        "/path/to/repos",
    AutoCreate: true,
  })

  // Configure git server. Will create git repos path if it does not exist.
  // We will also create individual repositories on the fly as they're requested.
  if err := service.Setup(); err != nil {
    log.Fatal(err)
  }

  http.Handle("/", service)

  // Start HTTP server
  if err := http.ListenAndServe(":5000", nil); err != nil {
    log.Fatal(err)
  }
}

Start the server with:

go run main.go
# this should start server on http://localhost:5000

Now we're ready for some action:

$ git clone http://localhost:5000/test.git /tmp/test
# Cloning into '/tmp/test'...
# warning: You appear to have cloned an empty repository.
# Checking connectivity... done.

$ cd /tmp/test
$ touch sample

$ git add sample
$ git commit -am "First commit"
# [master (root-commit) fe40c98] First commit
# 1 file changed, 0 insertions(+), 0 deletions(-)
# create mode 100644 sample

$ git push origin master
# Counting objects: 3, done.
# Writing objects: 100% (3/3), 213 bytes | 0 bytes/s, done.
# Total 3 (delta 0), reused 0 (delta 0)
# To http://localhost:5000/test.git
# * [new branch]      master -> master

Voila, now we can push or pull changes from our own server. Adding authentication is super straightforward:

// Here's the user-defined authentication function.
// If return value is false or error is set, user's request will be rejected.
// You can hook up your database/redis/cache for authentication purposes.
service.AuthFunc = func(cred gitkit.Credential, req *gitkit.Request) (bool, error) {
  log.Println("user auth request for repo:", cred.Username, cred.Password, req.RepoName)
  return cred.Username == "hello", nil
}

At this point we have everything in place to start building push workflows. The first script to run when handling a push from a client is pre-receive. It takes a list of references that are being pushed from stdin; if it exits non-zero, none of them are accepted. You can use this hook to do things like make sure none of the updated references are non-fast-forwards, or to do access control for all the refs and files they’re modifying with the push.

Gitkit provides a Receiver module:

receiver := gitkit.Receiver{
  MasterOnly:  false,         // if set to true, only pushes to master branch will be allowed
  TmpDir:      "/tmp/gitkit", // directory for temporary git checkouts
  HandlerFunc: receive,       // your handler function
}

// Git hook data is provided via STDIN
if err := receiver.Handle(os.Stdin); err != nil {
  log.Println("Error:", err)
  os.Exit(1) // terminating with non-zero status will cancel push
}

Or we can use Hooks configuration for running general scripts:

hooks := &gitkit.HookScripts{
  PreReceive: `echo "Hello World!"`,
}

// Configure git service
service := gitkit.New(gitkit.Config{
  // ...
  AutoHooks:  true,
  Hooks:      hooks,
})

When wired up correctly, pre-receive hook will run and we'll see the output:

$ git push origin master
# ...
# remote: Hello World! <----------------- pre-receive hook
# ...

Hook scripts could be written in pretty much any language. This is how Heroku initially worked: all bundle installation, precompilation steps were run as part of the pre-receive hook and if that failed, the whole git push payload is rejected. Make a change, then rinse and repeat until it works. Gitkit shell out to standard git for all operations so it should work on any system that has it installed.

Conclusion

Gitkit gives you a good enough foundation to run push workflows, all without detailed knowledge of git internals. While I don't really use the project these days, it's still alive. Check out Readme, there's tons of examples and additional information. I had fun hacking on it (some during Gophercon 2016) and glad others find the package useful.