May 13th, 2019
Orchestrating Containers with Kubernetes Part I: Concepts
I recently wrote a series of articles on Docker and containerized applications. In those articles I worked with a single Docker container at a time. Each container was mapped to a port on its host machines IP, making it accessible from the browser. In my case, the host machine was an EC2 instance.
While the single container approach works, it has a number of limitations and drawbacks. First off, a single container isn't scalable. As web traffic to an application increases, the load becomes too much for a single container to handle. Secondly, there isn't a zero-downtime approach to release a new version of an application. The container running the old version has to stop and a container with the new version has to start in its place. While both these limitations are deal breakers in themselves, the worst part about the single container approach is that its a single point of failure. If the container stops or the application crashes, the entire website goes down. This makes deploying a production application on a single container inadequate.
October 10th, 2021
Writing Kubernetes Tests with Go
These days, most of my application infrastructure exists on Docker containers, orchestrated by Kubernetes. My AWS account has a Kubernetes cluster, which is hosted using EKS (Elastic Kubernetes Service). Since two of my production applications (jarombek.com and saintsxctf.com) run on this Kubernetes cluster, the health of their infrastructure is critical. To help ensure that the Kubernetes cluster is running properly, I created tests which check the state of my Kubernetes objects and ensure that they exist on the EKS cluster as expected.
This article explores my Kubernetes test suite, which is written in Go and leverages the Kubernetes Go Client. It also describes how the test suite is run on an automated schedule, alerting me when test failures occur.
May 20th, 2019
Orchestrating Containers with Kubernetes Part II: Single Node Cluster
In my previous Kubernetes article, I went over the concepts of container orchestration and the architecture of a Kubernetes cluster. In this article, I'm building a single node Kubernetes cluster that runs a Node.js application.
The Node.js application is the same one I used in my article on Containerization. The Kubernetes cluster environment is very similar to my Docker playground. The single node Kubernetes cluster is created with a CloudFormation template wrapped in Terraform. It installs Docker, kubeadm, kubectl, kubelet, and kubernetes-cni. You can check out the infrastructure code on GitHub.
September 29th, 2020
Jenkins Server Modern Infrastructure with Kubernetes on EKS
In a prior article, I discussed a Jenkins server I created on AWS EC2 and EFS. In this article I’ll discuss the second generation of that infrastructure, which uses Docker containers orchestrated by Kubernetes on an EKS cluster.
March 27th, 2022
Running a MySQL Database Client on Kubernetes
In the first version of my SaintsXCTF application, one underdeveloped aspect of the technology stack was the MySQL database infrastructure. The only way to access the production database was to create a bastion host and interact with it via the command line. This bastion host was a server that was only accessible from my IP address and could only interact with my MySQL database. All other network ports were closed.
While this was an okay start, I really wanted a user interface to interact with the MySQL database with similar functionality to a local MySQL IDE, such as DataGrip. After researching different options, I decided to use phpMyAdmin, a MySQL administrative client that can run on a web server.
October 25th, 2021
SaintsXCTF Version 2.0: Kubernetes Infrastructure
The infrastructure for the React/TypeScript frontend and Flask/Python backend for my website saintsxctf.com is hosted on Kubernetes. My Kubernetes infrastructure is hosted on a cluster, which is managed by AWS EKS. This article outlines the Kubernetes infrastructure and walks through Terraform code which configures and builds the infrastructure.
September 28th, 2020
Building an AWS EKS cluster with Terraform
Recently I made the decision to move my applications to Kubernetes, specifically hosted in an EKS cluster on AWS. Before making this decision, my applications (saintsxctf.com and jarombek.com) were hosted using different methods. saintsxctf.com was hosted on autoscaled AWS EC2 instances and jarombek.com was hosted on AWS ECS. I also had prototypes using different hosting methods and a Jenkins server which was hosted on EC2 instances. Moving all these applications to Kubernetes unifies the deployment process and allows me to take advantage of containerization and container orchestration.
In this article, I'll discuss the process for setting up my EKS cluster with Terraform. I'll also detail my experience deploying ALB Ingress Controller and External DNS pods on the cluster.
October 19th, 2021
Creating a Go Module for Reusable Test Functions
Recently I wrote tests for my Kubernetes infrastructure in Go. These tests are split across multiple different repositories. However, there is a lot of overlap in testing logic between the test suites in each repository. In attempts to follow good programming practices and keep my code DRY, I split out the common code between the repositories into reusable functions. These functions exist in their own Go module, which is imported into the test suites as a dependency.
Go modules are part of Go's dependency management system1. They consist of a collection of packages, which are defied in a go.mod file. Go modules can be used as dependencies in other modules, as is the case with my reusable test function module and my Kubernetes test modules.
June 14th, 2021
SaintsXCTF Version 2.0: Architectural Overview
On December 24th 2016, I released my first website saintsxctf.com. I was still a senior in college at the time, and used my limited software development knowledge from classes and a summer internship to build the application. SaintsXCTF is a training log designed for my college Cross Country and Track & Field teams at St. Lawrence University. Running competitively in college had a major impact on my life, and I was really proud to create the website to assist my teammates and coaches. Shortly after releasing the website, I created an Android application and an iOS application for SaintsXCTF. With SaintsXCTF accessible via web browsers and mobile applications, I felt my development work was complete and moved on to other programming projects.
As I began my professional software engineering career in the summer of 2017, I gradually learned industry best practices and became more well rounded as a developer. At this point, certain shortcomings and misguided assumptions about my SaintsXCTF applications became apparent. First, the core web application and API did not follow the latest industry standards. Second, all three applications were not properly tested and were prone to degradation if left unchecked. Third, the security & infrastructure of the applications was very basic and not fault tolerant. Lastly, my assumption that releasing the applications meant my work was done proved to be incorrect. As all software engineers know, the work just begins when an application is initially released.
July 31st, 2021
Building a GraphQL React Prototype
Last year, I started using GraphQL at my job. I decided to create some GraphQL prototypes in my spare time, to get better acquainted with the GraphQL ecosystem. In 2018 I learned the basics of GraphQL and wrote two articles about my experience, but never dove into using GraphQL in real world applications. The GraphQL React prototype discussed in this article along with my Apollo prototype are the beginnings of that production application journey. In the future, I plan on using GraphQL for the API layer of my applications.
The GraphQL prototype discussed in this article is a React front-end application that connects to a GitHub GraphQL API. The API provides details about my repositories, and React displays those details in a dashboard. The dashboard is shown below.
September 24th, 2021
Creating Reverse Proxies with Nginx and Docker
Many of my applications contain frontend components and API components. These two components are loosely coupled but communicate with each other over HTTPS. For example, my saintsxctf.com application has a React frontend which communicates with a Flask REST API backend, along with other API Gateway REST APIs. One way to accomplish communication from a frontend to an API is by explicitly writing the URLs of the APIs in the frontend code. This works fine, but it also exposes information about API origin servers to clients. Origin server information exposure can be avoided by passing all API traffic through the same URL as the frontend application. This is accomplished using a reverse proxy.
The following image shows my SaintsXCTF website, and how the URL of the API is hidden from clients. If users inspect the website's network traffic, they see HTTPS requests sent to the reverse proxy server for saintsxctf.com, instead of the actual API server api.saintsxctf.com.
September 27th, 2020
Jenkins Server Legacy Infrastructure on EC2 and EFS
Back in 2018, I created a Jenkins server which automated tasks for my applications. Jenkins is a continuous integration and continuous delivery (CI/CD) tool which I've written about in the past. When I first created the Jenkins server, I had a few jobs which ran unit tests, but I never took full advantage of them. Over the past two years, I've gained a greater appreciation for CI/CD tools and their ability to save time deploying code and building confidence in codebases by automating tests. Nowadays all my applications have automated test and deployment jobs on Jenkins.
Since 2018 the Jenkins ecosystem has evolved along with my understanding of cloud concepts. My original Jenkins server was hosted on an AWS EC2 instance which utilized AWS EFS for persistent storage. In the spring of 2020, I decided to rewrite the Jenkins server infrastructure. With my added knowledge of containerization with Docker and container orchestration with Kubernetes, I hosted the Jenkins server on AWS EKS as part of a Kubernetes deployment. In this article, I discuss the original EC2 Jenkins server and its creation process with Terraform. In an upcoming article, I'll discuss the Kubernetes Jenkins server infrastructure.