DISCOVERY

November 15th, 2022

Building and Testing Go Code using Please Build and GitHub Actions

Please Build

GitHub Actions

Docker

GoLand

Bazel

Starlark

YAML

Go

During software development, it's important to test logic and perform static code analysis to ensure programs meet certain standards. Build tools and CI/CD platforms help developers automate these tasks. There are many different build tools, such as Make, Bazel, and Please, along with many CI/CD platforms, such as Jenkins, TravisCI, and GitHub Actions (to name a few). Due to a wide variety of choices, it can be overwhelming for engineers to pick the best option for their codebase.

Although I've experimented a bit with TravisCI in the past, most of my CI/CD work up until now has been on Jenkins. Outside of language or platform specific build tools (such as CMake or Webpack), I haven't used many language-agnostic build tools in my code up to this point. While designing my go-programming repository, which contains Go programming language code samples, I decided to work with a new build tool and CI/CD platform.

For the build tool, I decided to use Please Build, a language agnostic build platform derived from Google's internal Blaze build tool, which was ported to the open source community as Bazel1. For the CI/CD platform, I decided to use GitHub Actions, which is built into GitHub. In this article, I'll discuss the basics of Please Build and GitHub Actions, along with how I integrated both into my go-programming repository.

The go-programming repository consists of many subdirectories containing Go code. This Go code contains unit tests that I ensure are successful and libraries that I build and compile into machine specific binaries. For example, the goroutines subdirectory contains unit tests, such as basics_test.go, along with libraries, such as goroutine_example.go.

I use the GoLand IDE by Jetbrains for Go development. One of the great features of Jetbrains IDEs, GoLand included, are run configurations. Run configurations enable an easy way to run or test Go code through the GoLand UI2. Run configurations can optionally run on a Docker container, and configurations can be saved and committed to a repository in XML files.

In the go-programming repository, run configurations are stored in a .run directory. These run configurations execute on Docker containers, which are defined in a base directory. Currently, all my run configurations execute on docker containers derived from a base Dockerfile, which is specified in base/Dockerfile, with the following setup.

FROM golang:1.18.3-alpine3.16 RUN apk add gcc libc-dev linux-headers RUN curl https://get.please.build > please.sh && \ bash ./please.sh ENV PATH="${PATH}:/root/.please/bin" STOPSIGNAL SIGTERM

Most importantly, the base Dockerfile installs Please Build from the URL https://get.please.build and appends its installation directory onto the PATH environment variable. Go is installed and configured in the golang image, which is specified as the base image using the FROM golang:1.18.3-alpine3.16 command.

Each subdirectory in the repository has its own Dockerfile, using base/Dockerfile as its base image. Subdirectory Dockerfiles contain commands needed to execute the code within the subdirectory. For example, the goroutines subdirectory contains the following Dockerfile

FROM ajarombek/go-alpine-linux-programming:latest WORKDIR src COPY . . ENTRYPOINT ["go", "test", "-v", "."]

This Dockerfile uses the base image ajarombek/go-alpine-linux-programming:latest, which is available on DockerHub. It copies files from my repository onto the containers filesystem and runs the command go test -v ., which executes all the Go tests.

This Dockerfile is also used in a run configuration for the repository. GoLand run configurations are XML files, and in my repository they specify a docker container to run. For the goroutines subdirectory, I created a goroutines.run.xml run configuration which starts a Docker container.

<component name="ProjectRunConfigurationManager"> <configuration default="false" name="goroutines" type="docker-deploy" factoryName="dockerfile" server-name="Docker"> <deployment type="dockerfile"> <settings> <option name="containerName" value="go_programming_goroutines" /> <option name="sourceFilePath" value="goroutines/Dockerfile" /> </settings> </deployment> <method v="2" /> </configuration> </component>

This run configuration specifies the creation of a Docker image named go_programming_goroutines created using the Dockerfile goroutines/Dockerfile. Putting all the pieces together, the following image shows what it looks like to trigger the run configuration and view its console log.

Run configurations are great for manually executing tests or running code. However, for CI/CD purposes, operations like executing tests, compiling code, and packaging binaries should be automated. This is where Please Build and GitHub Actions can help.

Please Build

Please Build is a language agnostic build system derived from Bazel and Blaze, which are open source and proprietary build systems originating at Google. Like Bazel and Blaze, Please Build uses the Starlark programming language for its build files. Build files, commonly named BUILD, contain build targets, which are units of deployable code3. Build targets are written using rules, which appear as Python functions. Rules are configurable using arguments and they determine what occurs when a build target is run. Builds are run in isolation from application source code; they are given a separate plz-out directory to run it. Build results are also cached, which makes subsequent builds on unchanged code very fast.

Starlark Programming Language

Starlark is a programming language that is syntactically similar to and inspired by Python4. Starlark is used in BUILD files for the Blaze and Bazel build tools, along with their derivatives such as Please Build. Starlark has different language rules than Python despite its syntactic similarities; Starlark is neither a superset or subset of Python.

When configuring Please Build in a repository, a plz init command is run from the root of the codebase. This command creates a .plzconfig file in the repository, which configures the execution of Please commands5. This is how .plzconfig appears in my go-programming repository.

[please] version = 16.22.1 [build] PassEnv = PATH path = $PATH:/usr/local/bin:/usr/bin:/bin [buildenv] test-env = plz

.plzconfig lists out different build options to use in the repository6. version = 16.22.1 sets the version of Please Build to use while performing builds. PassEnv = PATH takes the PATH environment variable and makes it accessible while Please Build rules are executed. path = $PATH:/usr/local/bin:/usr/bin:/bin takes this PATH environment variable and appends /usr/local/bin:/usr/bin:/bin to it. Finally, test-env = plz creates a new environment variable named TEST_ENV and assigns it a value of plz. TEST_ENV is accessible while running Please Build commands. These are just a few of the configuration options available with Please Build; a full list of options are available in the Please Build documentation.

As previously mentioned, Please Build consists of BUILD files written in Starlark. In my go-programming repository, the root directory contains a BUILD file with the following contents.

package(default_visibility = ["PUBLIC"]) go_toolchain( name = "go_download", version = "1.18", ) go_module( name = "testify", module = "github.com/stretchr/testify", install = ["..."], version = "v1.7.0", deps = [ ":go_difflib", ":go_yaml", ":go_spew", ":objx", ], ) go_module( name = "go_difflib", module = "github.com/pmezard/go-difflib", install = ["..."], version = "v1.0.0", deps = [":go_download"], ) go_module( name = "go_yaml", module = "gopkg.in/yaml.v3", install = ["..."], version = "v3.0.0-20200313102051-9f266ea9e77c", deps = [":go_download"], ) go_module( name = "go_spew", module = "github.com/davecgh/go-spew", install = ["..."], version = "v1.1.0", deps = [":go_download"], ) go_module( name = "objx", module = "github.com/stretchr/objx", install = ["..."], version = "v0.1.0", deps = [":go_download"], )

On the first line of code, package(), a Please Build built-in function, sets configuration details for all the ensuing rules6. In my code, it sets the visibility of all the rules in the file to PUBLIC, which makes them accessible to all the other rules in the repository7.

The next rule, go_toolchain(), downloads Go for other go_ prefixed rules to use. Based on my configurations, go_toolchain() downloads version 1.18 of Go. The name argument of go_toolchain() defines a unique name for the rule that can be used as a dependency for other rules. name is considered a common argument, and exists on every rule in Please Build8.

The first go_module() rule, named testify, downloads and installs a Go module that is used by other go_ prefixed rules. This Go module contains assertion functions which I use in unit tests. The full path of the module is github.com/stretchr/testify and the version I install is v1.7.0. testify depends on four Go modules itself, which are defined in the deps argument. Each dependency has its own go_module() rule in the BUILD file. The colon (:) before each dependency name informs Please Build that the dependency is another rule declared in the same package.

With that in mind, notice how the rules in this BUILD file define a dependency tree. testify is dependent on four dependencies, go_difflib, go_yaml, go_spew, objx, which in turn are all dependent on go_download. This is often the case in Please Build, where rules build on each other to create a functional build system for an application.

Rules in the root BUILD file create a foundation for other rules in the repository. Let's take a look at another BUILD file. unit-testing/BUILD defines rules that run unit tests for a Go program. The unit-testing directory contains code samples I created based on my learnings from chapter 11 of The Go Programming Language. The BUILD file contains two rules.

go_library( name = "license_plates", srcs = ["license_plates.go"], deps = ["//:go_download"], labels = ["unit_testing"], ) go_test( name = "test", srcs = ["license_plates_test.go"], deps = [":license_plates"], labels = [ "unit_testing", "test", ], )

First, go_library() defines a rule, named license_plates, that creates a Go library usable by other Go programs. license_plates contains a single Go file license_plates.go and is dependent on //:go_download, which downloads and installs Go. The //: prefix to the dependency tells Please Build that go_download is a rule located in the root of the repository.

Second, go_test() defines a rule, named test, to run Go tests. test contains a single Go file license_plates_test.go, which implements multiple unit test cases, and is dependent on the license_plates rule.

test is executed from the command line to verify that all Go tests are successful. Running a test rule is done with a plz test command; in my scenario, plz test //unit-testing:test -vvv runs my test rule. //unit-testing:test specifies the path and name of the rule to execute, and -vvv sets the verbosity level of the console output logs. Running this command in my repository generates an output similar to the following log.

... //unit-testing:test 5 tests run in 121ms; 5 passed TestValidCountryCode PASS 0s TestInvalidCountryCode PASS 0s TestCollected PASS 0s Collected PASS 0s CountryCode PASS 0s 1 test target and 5 tests run; 5 passed. Total time: 15.22s real, 120ms compute.

Let's look at a similar example in my reflection directory, which analyzes how to use reflection to gain runtime information about types in Go programs. reflection contains zero library files and two unit test files, with the following BUILD configuration.

filegroup( name = "test_files", srcs = glob(["*.go"]), ) go_test( name = "test", srcs = [":test_files"], deps = [ "//:go_download", "//:testify", ], labels = [ "reflection", "test" ], )

Instead of explicitly writing names of Go files in the srcs argument of the go_test() rule, I use a separate filegroup() rule to collect all the files matching the glob pattern *.go. In the end, running these tests is the same as it was in the unit-testing directory, with a plz test //reflection:test -vvv command.

Let's look at one more example in my goroutines/channel_example directory. channel_example contains a single Go application file, and has the following BUILD configuration.

go_binary( name = "binary", srcs = ["channel_example.go"], out = "channel_example", deps = ["//:go_download"], labels = [ "goroutines", "binary", ], )

The go_binary() rule creates an executable binary file from compiled Go code. After Go is installed from the //:go_download dependency rule, a single channel_example.go file is compiled. Running a binary rule is done with a plz build command; in my scenario, plz build //goroutines/channel_example:binary -vvv runs the binary rule. Running this command in my repository generates an output similar to the following log.

... Build finished; total time 12.67s, incrementality 66.7%. Outputs: //goroutines/channel_example:binary: plz-out/bin/goroutines/channel_example/channel_example

The plz build command created a binary file within a plz-out/bin directory. From a command prompt (in my case running Bash), this binary file can be executed.

./plz-out/bin/goroutines/channel_example/channel_example

Every subdirectory in my go-programming repository contains a BUILD file with additional rules in case you want to explore more. One last thing to mention is that re-running any Please Build command without altering the source code results in a faster build. This is because stages of builds are cached, similar to layers on images being cached in a Dockerfile build. For example, the previous plz build command, plz build //goroutines/channel_example:binary -vvv, took 12.67 seconds the first time it ran on my machine. Running it a second time took just 140 milliseconds.

... Build finished; total time 140ms, incrementality 100.0%. Outputs: //goroutines/channel_example:binary: plz-out/bin/goroutines/channel_example/channel_example

GitHub Actions

GitHub Actions is a CI/CD platform integrated with the GitHub version control hosting service. GitHub Actions allows engineers to create workflows that run on virtual machines, containers, or self-hosted infrastructure10. Workflows, written in YAML, contain one or many jobs that run on a schedule or respond to events within a repository, such as code commits or pull request creation.

GitHub Actions workflows exist in a .github/workflows directory for GitHub repositories. In my repository, this directory contains two YAML files, each corresponding to a workflow. Let's look at the first workflow, go_tests.yml, which runs all the Go tests in the go-programming repository.

name: Go Tests on: push: branches: ["main", "feature/*"] pull_request: branches: ["main"] schedule: - cron: "0 5 * * 1,3,5" workflow_dispatch: jobs: go_tests: runs-on: ubuntu-latest container: ajarombek/go-alpine-linux-programming:latest steps: - run: echo "Job running on a ${{ runner.os }} server" - name: Check out repository code uses: actions/checkout@v3 - run: echo "Checked out branch '${{ github.ref }}' of the ${{ github.repository }} repository" - name: Files installed from repository run: ls -ltra - name: Run Go tests using Please Build run: plz test //... -i test -vvv

This YAML document creates a workflow named Go Tests. The on dictionary configures events that trigger the workflow along with an additional cron schedule for the workflow to run on. on contains four keys, push, pull_request, schedule, and workflow_dispatch. push triggers the workflow anytime code is pushed to the main branch or any branch matching the feature/* pattern. pull_request triggers the workflow anytime a pull request is made that merges code into the main branch. schedule sets a cron schedule for the workflow to run on, in my case 5:00 AM every Monday, Wednesday, and Friday. Finally, workflow_dispatch enables manual triggers of the pipeline from the GitHub UI or API11.

The jobs dictionary configures one or many jobs within a workflow. My workflow contains a single job named go_tests. The runs-on key-value pair determines the machine and operating system to run on, in my case an Ubuntu distribution of Linux (ubuntu-latest). On top of Ubuntu, my job runs in a Docker container, with the image name and tag specified under the container key-value pair. The value under container, ajarombek/go-alpine-linux-programming:latest, is a Docker image stored on DockerHub and defined in a Dockerfile in my repository.

steps is a list of steps within a job. Steps with a run key execute its corresponding value from the shell as a command line program12. name keys give steps a name that are displayed in the GitHub UI and uses keys run actions for a step13.

Actions are a unit of execution within GitHub Actions. They are programs, often written in JavaScript, that perform commonly repeated tasks14. In my case, I use a actions/checkout@v3 action, which is maintained by GitHub. actions/checkout@v3 is used to checkout a GitHub repository.

As a whole, my Go Tests workflow runs on a custom Docker container, performs a checkout of my go-programming repository, and runs a plz test //... -i test -vvv command, which executes all Go tests in my code. The -i test flag in the plz test command runs every Please Build rule that has a label of value test.

Results of GitHub actions are viewable in repositories on GitHub. One way to view actions is to click on the "Actions" tab at the top of a repository page.

In the list of workflow results under the "Actions" tab, clicking on one of them navigates to the following page, which gives an overview of the workflow run and the results of individual jobs.

My "Go Tests" workflow contains a single go_tests job, and clicking on it displays the execution logs.

GitHub Actions send an email when workflows fail, which is a great way to stay alerted on the status of a codebase. I only demonstrated the basics of GitHub Actions in this article, and there is a lot more for me to learn, but GitHub Actions is a very promising CI/CD framework that I intend to use extensively in the future.

Combining the "Please Build" build system and GitHub Actions CI/CD platform was a great way to test Go code in both my development environment and on GitHub's hosted platform. I plan to expand my usage of both technologies into other projects and learn more advanced features. All the code discussed in this article is available in my go-programming repository on GitHub.