During software development, it's important to test logic and perform static code analysis to ensure programs meet certain standards. Build tools and CI/CD platforms help developers automate these tasks. There are many different build tools, such as Make, Bazel, and Please, along with many CI/CD platforms, such as Jenkins, TravisCI, and GitHub Actions (to name a few). Due to a wide variety of choices, it can be overwhelming for engineers to pick the best option for their codebase.
Although I've experimented a bit with TravisCI in the past, most of my CI/CD work up until now has been on Jenkins. Outside of language or platform specific build tools (such as CMake or Webpack), I haven't used many language-agnostic build tools in my code up to this point. While designing my go-programming repository, which contains Go programming language code samples, I decided to work with a new build tool and CI/CD platform.
For the build tool, I decided to use Please Build, a language agnostic build platform derived from Google's internal Blaze build tool, which was ported to the open source community as Bazel1. For the CI/CD platform, I decided to use GitHub Actions, which is built into GitHub. In this article, I'll discuss the basics of Please Build and GitHub Actions, along with how I integrated both into my go-programming repository.
The go-programming repository consists of many subdirectories containing Go code. This Go code contains unit tests that I ensure are successful and libraries that I build and compile into machine specific binaries. For example, the goroutines subdirectory contains unit tests, such as basics_test.go, along with libraries, such as goroutine_example.go.
I use the GoLand IDE by Jetbrains for Go development. One of the great features of Jetbrains IDEs, GoLand included, are run configurations. Run configurations enable an easy way to run or test Go code through the GoLand UI2. Run configurations can optionally run on a Docker container, and configurations can be saved and committed to a repository in XML files.
In the go-programming repository, run configurations are stored in a .run directory. These run configurations execute on Docker containers, which are defined in a base directory. Currently, all my run configurations execute on docker containers derived from a base Dockerfile, which is specified in base/Dockerfile, with the following setup.
Most importantly, the base Dockerfile installs Please Build from the URL https://get.please.build and appends its installation directory onto the
PATH environment variable. Go is installed and configured in the
golang image, which is specified as the base image using the
FROM golang:1.18.3-alpine3.16 command.
Each subdirectory in the repository has its own Dockerfile, using base/Dockerfile as its base image. Subdirectory Dockerfiles contain commands needed to execute the code within the subdirectory. For example, the goroutines subdirectory contains the following Dockerfile
This Dockerfile uses the base image
ajarombek/go-alpine-linux-programming:latest, which is available on DockerHub. It copies files from my repository onto the containers filesystem and runs the command
go test -v ., which executes all the Go tests.
This Dockerfile is also used in a run configuration for the repository. GoLand run configurations are XML files, and in my repository they specify a docker container to run. For the goroutines subdirectory, I created a goroutines.run.xml run configuration which starts a Docker container.
This run configuration specifies the creation of a Docker image named
go_programming_goroutines created using the Dockerfile
goroutines/Dockerfile. Putting all the pieces together, the following image shows what it looks like to trigger the run configuration and view its console log.
Run configurations are great for manually executing tests or running code. However, for CI/CD purposes, operations like executing tests, compiling code, and packaging binaries should be automated. This is where Please Build and GitHub Actions can help.
Please Build is a language agnostic build system derived from Bazel and Blaze, which are open source and proprietary build systems originating at Google. Like Bazel and Blaze, Please Build uses the Starlark programming language for its build files. Build files, commonly named BUILD, contain build targets, which are units of deployable code3. Build targets are written using rules, which appear as Python functions. Rules are configurable using arguments and they determine what occurs when a build target is run. Builds are run in isolation from application source code; they are given a separate plz-out directory to run it. Build results are also cached, which makes subsequent builds on unchanged code very fast.
Starlark Programming Language
Starlark is a programming language that is syntactically similar to and inspired by Python4. Starlark is used in BUILD files for the Blaze and Bazel build tools, along with their derivatives such as Please Build. Starlark has different language rules than Python despite its syntactic similarities; Starlark is neither a superset or subset of Python.
When configuring Please Build in a repository, a
plz init command is run from the root of the codebase. This command creates a .plzconfig file in the repository, which configures the execution of Please commands5. This is how .plzconfig appears in my go-programming repository.
.plzconfig lists out different build options to use in the repository6.
version = 16.22.1 sets the version of Please Build to use while performing builds.
PassEnv = PATH takes the
PATH environment variable and makes it accessible while Please Build rules are executed.
path = $PATH:/usr/local/bin:/usr/bin:/bin takes this
PATH environment variable and appends
/usr/local/bin:/usr/bin:/bin to it. Finally,
test-env = plz creates a new environment variable named
TEST_ENV and assigns it a value of
TEST_ENV is accessible while running Please Build commands. These are just a few of the configuration options available with Please Build; a full list of options are available in the Please Build documentation.
On the first line of code,
package(), a Please Build built-in function, sets configuration details for all the ensuing rules6. In my code, it sets the visibility of all the rules in the file to
PUBLIC, which makes them accessible to all the other rules in the repository7.
The next rule,
go_toolchain(), downloads Go for other
go_ prefixed rules to use. Based on my configurations,
go_toolchain() downloads version 1.18 of Go. The
name argument of
go_toolchain() defines a unique name for the rule that can be used as a dependency for other rules.
name is considered a common argument, and exists on every rule in Please Build8.
go_module() rule, named
testify, downloads and installs a Go module that is used by other
go_ prefixed rules. This Go module contains assertion functions which I use in unit tests. The full path of the module is
github.com/stretchr/testify and the version I install is
testify depends on four Go modules itself, which are defined in the
deps argument. Each dependency has its own
go_module() rule in the BUILD file. The colon (
:) before each dependency name informs Please Build that the dependency is another rule declared in the same package.
With that in mind, notice how the rules in this BUILD file define a dependency tree.
testify is dependent on four dependencies,
objx, which in turn are all dependent on
go_download. This is often the case in Please Build, where rules build on each other to create a functional build system for an application.
Rules in the root BUILD file create a foundation for other rules in the repository. Let's take a look at another BUILD file. unit-testing/BUILD defines rules that run unit tests for a Go program. The unit-testing directory contains code samples I created based on my learnings from chapter 11 of The Go Programming Language. The BUILD file contains two rules.
go_library() defines a rule, named
license_plates, that creates a Go library usable by other Go programs.
license_plates contains a single Go file
license_plates.go and is dependent on
//:go_download, which downloads and installs Go. The
//: prefix to the dependency tells Please Build that
go_download is a rule located in the root of the repository.
go_test() defines a rule, named
test, to run Go tests.
test contains a single Go file
license_plates_test.go, which implements multiple unit test cases, and is dependent on the
test is executed from the command line to verify that all Go tests are successful. Running a test rule is done with a
plz test command; in my scenario,
plz test //unit-testing:test -vvv runs my
//unit-testing:test specifies the path and name of the rule to execute, and
-vvv sets the verbosity level of the console output logs. Running this command in my repository generates an output similar to the following log.
Let's look at a similar example in my reflection directory, which analyzes how to use reflection to gain runtime information about types in Go programs. reflection contains zero library files and two unit test files, with the following BUILD configuration.
Instead of explicitly writing names of Go files in the
srcs argument of the
go_test() rule, I use a separate
filegroup() rule to collect all the files matching the glob pattern
*.go. In the end, running these tests is the same as it was in the unit-testing directory, with a
plz test //reflection:test -vvv command.
go_binary() rule creates an executable binary file from compiled Go code. After Go is installed from the
//:go_download dependency rule, a single
channel_example.go file is compiled. Running a binary rule is done with a
plz build command; in my scenario,
plz build //goroutines/channel_example:binary -vvv runs the
binary rule. Running this command in my repository generates an output similar to the following log.
plz build command created a binary file within a plz-out/bin directory. From a command prompt (in my case running Bash), this binary file can be executed.
Every subdirectory in my go-programming repository contains a BUILD file with additional rules in case you want to explore more. One last thing to mention is that re-running any Please Build command without altering the source code results in a faster build. This is because stages of builds are cached, similar to layers on images being cached in a Dockerfile build. For example, the previous
plz build command,
plz build //goroutines/channel_example:binary -vvv, took 12.67 seconds the first time it ran on my machine. Running it a second time took just 140 milliseconds.
GitHub Actions is a CI/CD platform integrated with the GitHub version control hosting service. GitHub Actions allows engineers to create workflows that run on virtual machines, containers, or self-hosted infrastructure10. Workflows, written in YAML, contain one or many jobs that run on a schedule or respond to events within a repository, such as code commits or pull request creation.
GitHub Actions workflows exist in a .github/workflows directory for GitHub repositories. In my repository, this directory contains two YAML files, each corresponding to a workflow. Let's look at the first workflow, go_tests.yml, which runs all the Go tests in the go-programming repository.
This YAML document creates a workflow named
Go Tests. The
on dictionary configures events that trigger the workflow along with an additional cron schedule for the workflow to run on.
on contains four keys,
push triggers the workflow anytime code is pushed to the
main branch or any branch matching the
pull_request triggers the workflow anytime a pull request is made that merges code into the
schedule sets a cron schedule for the workflow to run on, in my case 5:00 AM every Monday, Wednesday, and Friday. Finally,
workflow_dispatch enables manual triggers of the pipeline from the GitHub UI or API11.
jobs dictionary configures one or many jobs within a workflow. My workflow contains a single job named
runs-on key-value pair determines the machine and operating system to run on, in my case an Ubuntu distribution of Linux (
ubuntu-latest). On top of Ubuntu, my job runs in a Docker container, with the image name and tag specified under the
container key-value pair. The value under
ajarombek/go-alpine-linux-programming:latest, is a Docker image stored on DockerHub and defined in a Dockerfile in my repository.
steps is a list of steps within a job. Steps with a
run key execute its corresponding value from the shell as a command line program12.
name keys give steps a name that are displayed in the GitHub UI and
uses keys run actions for a step13.
actions/checkout@v3 action, which is maintained by GitHub.
actions/checkout@v3 is used to checkout a GitHub repository.
As a whole, my
Go Tests workflow runs on a custom Docker container, performs a checkout of my go-programming repository, and runs a
plz test //... -i test -vvv command, which executes all Go tests in my code. The
-i test flag in the
plz test command runs every Please Build rule that has a label of value
Results of GitHub actions are viewable in repositories on GitHub. One way to view actions is to click on the "Actions" tab at the top of a repository page.
In the list of workflow results under the "Actions" tab, clicking on one of them navigates to the following page, which gives an overview of the workflow run and the results of individual jobs.
My "Go Tests" workflow contains a single
go_tests job, and clicking on it displays the execution logs.
GitHub Actions send an email when workflows fail, which is a great way to stay alerted on the status of a codebase. I only demonstrated the basics of GitHub Actions in this article, and there is a lot more for me to learn, but GitHub Actions is a very promising CI/CD framework that I intend to use extensively in the future.
Combining the "Please Build" build system and GitHub Actions CI/CD platform was a great way to test Go code in both my development environment and on GitHub's hosted platform. I plan to expand my usage of both technologies into other projects and learn more advanced features. All the code discussed in this article is available in my go-programming repository on GitHub.