Databricks is a data warehouse and data lake platform designed around Apache Spark. I use Databricks extensively at work, but also use Databricks and Apache Spark for personal applications. This summer, I attended my first software engineering conference, the Databricks Data+AI Summit in San Francisco. The conference was a great way to stay up-to-date on the latest trends in the data engineering industry.

During software development, it's important to test logic and perform static code analysis to ensure programs meet certain standards. Build tools and CI/CD platforms help developers automate these tasks. There are many different build tools, such as Make, Bazel, and Please, along with many CI/CD platforms, such as Jenkins, TravisCI, and GitHub Actions (to name a few). Due to a wide variety of choices, it can be overwhelming for engineers to pick the best option for their codebase.

Although I've experimented a bit with TravisCI in the past, most of my CI/CD work up until now has been on Jenkins. Outside of language or platform specific build tools (such as CMake or Webpack), I haven't used many language-agnostic build tools in my code up to this point. While designing my go-programming repository, which contains Go programming language code samples, I decided to work with a new build tool and CI/CD platform.

Data alignment in operating systems is something I’ve been aware of throughout my young career, mostly thanks to my love for reading software engineering books. However, it’s not a concept that I’ve explored in depth. In this article, I’ll discuss the basics of data alignment and how it applies to programming languages like Go.

Recently I read The Go Programming Language in hopes to better understand Go and discover what makes it a unique language. One Go feature that peaked my interest was goroutines. Goroutines are a mechanism for achieving concurrency and parallelism in Go programs. Many programming languages provide libraries that use multiple threads or processes to achieve concurrency, but Go takes a more distinct approach.

Go's use of Goroutines to achieve concurrent programming lead me to ask many questions, all of which I'll attempt to answer in this article. The questions are as follows:

Recently at work, I’ve used Splunk Enterprise extensively to analyze application logs. In Splunk, I’m able to query logs, trigger alerts when scenarios occur, and create dashboards to visualize logs for fellow engineers and business analysts. Splunk provides its own query language to interact with logs on its platform; Splunk Processing Language (SPL) is used for searching, alerting, and dashboard creation¹.

Although Splunk querying was part of my work-life for years, I never explored it on my own or used it in any personal projects. Recently, I decided to run a local Splunk Enterprise server to experiment with its basic properties. This groundwork may lead to my adoption of Splunk for monitoring applications logs. In this article, I describe the basics of Splunk and how to set up a local instance on Docker. I also discuss basic SPL queries and dashboards, and how some queries can be run before you upload any logs of your own.

In the first version of my SaintsXCTF application, one underdeveloped aspect of the technology stack was the MySQL database infrastructure. The only way to access the production database was to create a bastion host and interact with it via the command line. This bastion host was a server that was only accessible from my IP address and could only interact with my MySQL database. All other network ports were closed.

While this was an okay start, I really wanted a user interface to interact with the MySQL database with similar functionality to a local MySQL IDE, such as DataGrip. After researching different options, I decided to use phpMyAdmin, a MySQL administrative client that can run on a web server.

In my previous article, I discussed SwiftUI and how to integrate SwiftUI components into a UIKit application. SwiftUI is a (relatively) new framework for creating user interfaces in Apple ecosystem applications. User interfaces built using SwiftUI are configured completely in code with Swift. In this article, I discuss how to UI test SwiftUI views in an iOS application.

SwiftUI is a new framework that creates user interfaces for iOS and other Apple operating system applications. SwiftUI is completely programmatic (user interfaces in SwiftUI are built exclusively by writing Swift code). This is in stark contrast to previous frameworks such as UIKit which use a combination of Swift or Objective-C code and storyboards. Storyboards are graphical interfaces which allow engineers or designers to build user interfaces without writing code.

As someone who prefers solutions that involve writing code exclusively, SwiftUI is a welcome addition to the iOS landscape and a positive change in direction from Apple. Using GUIs such as storyboard to build interfaces have many downsides, such as difficulties when working in large groups on a single UI file and confusing merge conflicts when using version control systems. Since I'm very passionate about programming (as most software engineers are), altering values in a GUI to change a UI is a lot less exciting than coding!

My SaintsXCTF application has multiple APIs. One of these APIs, auth.saintsxctf.com, is hosted on AWS API Gateway and AWS Lambda. Its main purpose is to provide an authentication mechanism for the application. In this article, I discuss endpoints within this API and how they provide application authentication using JWTs.

My SaintsXCTF application has multiple APIs. One of these APIs, fn.saintsxctf.com, is hosted on AWS API Gateway and AWS Lambda. Its main purpose is to send emails to users based on certain website actions, such as changing a password or registering. In this article, I discuss the infrastructure needed to make the API, the source code of the API endpoints, and the approach used to deploy the API to AWS.

Over the last six months, I’ve ued Apache Airflow extensively at work. Airflow is a platform and framework for building and automating data pipelines¹. Airflow data pipelines are written in Python and interoperate with many different technologies, such as databases, cloud platforms, containers, and more. Often, Airflow is used in the realms of data analytics and machine learning.

While Airflow data pipelines are written in Python, the software they automate and schedule do not need to be Python related. Nonetheless, the fact that Airflow's language is Python makes data pipelines highly configurable and customizable. Since Python is very popular and simpler to learn compared to other languages, most engineers will be able to work with Airflow easily.

In a prior article, I discussed my Flask API written for SaintsXCTF, an application which allows users and their teammates to log running exercises and keep track of their running mileage. As with any piece of software, this Flask API requires extensive testing to keep it functional and maintainable. In this article, I discuss the automated testing approach I implemented for the API.