RETROSPECTIVE

September 24th, 2021

Creating Reverse Proxies with Nginx and Docker

Nginx

Docker

Kubernetes

uWSGI

Many of my applications contain frontend components and API components. These two components are loosely coupled but communicate with each other over HTTPS. For example, my saintsxctf.com application has a React frontend which communicates with a Flask REST API backend, along with other API Gateway REST APIs. One way to accomplish communication from a frontend to an API is by explicitly writing the URLs of the APIs in the frontend code. This works fine, but it also exposes information about API origin servers to clients. Origin server information exposure can be avoided by passing all API traffic through the same URL as the frontend application. This is accomplished using a reverse proxy.

The following image shows my SaintsXCTF website, and how the URL of the API is hidden from clients. If users inspect the website's network traffic, they see HTTPS requests sent to the reverse proxy server for saintsxctf.com, instead of the actual API server api.saintsxctf.com.

This article discusses reverse proxy servers, shows examples from my applications, and provides application source code; allowing you to achieve similar results. More specifically, it looks at creating Nginx reverse proxies in Docker containers.

When discussing Nginx reverse proxies, there are two distinct technologies to unpack: Nginx and reverse proxies.

Nginx

Nginx (pronounced Engine-X) is an open source web server which can be used as a reverse proxy1. It is commonly used for serving content from HTTP requests, as well as caching and load balancing requests. Nginx is comparable in popularity with Apache HTTP Server, which is another open source web server. Nginx has its own configuration language, which engineers use to adjust the web server's behavior2.

Reverse Proxy

A reverse proxy server sits between a client and one or more backend servers. Reverse proxy servers intake requests from a client, and appropriately distributes those requests to servers sitting behind it. When a reverse proxy server sends responses from it's backend servers to a client, it does so without alluding to the existence of backend servers at all. From the client's perspective, responses originate from the reverse proxy server itself. Reverse proxy servers are used to hide backend servers, cache responses, load balance, and more3.

Using Nginx's configuration language, servers can be configured as reverse proxy servers. While these servers can exist on physical machines or virtual machines, the Nginx servers discussed in this article exist on Docker containers.

I have multiple applications which use Nginx reverse proxy servers on Docker containers, including the SaintsXCTF web application, SaintsXCTF API, Apollo client prototype, Apollo server prototype, and GraphQL React prototype. All these proxy server containers are orchestrated using Kubernetes and hosted on AWS EKS. For the remainder of the article, I'll look specifically at the Nginx configuration for my SaintsXCTF application. Nginx code from my other applications can be viewed on GitHub using the links above.

The SaintsXCTF application has two Nginx reverse proxies, one for the web application and one for the main API. The application also has smaller APIs which are hosted using AWS API Gateway. As I mentioned earlier, the Nginx reverse proxies are hosted on Docker containers orchestrated on Kubernetes. My Kubernetes infrastructure is hosted on an AWS EKS cluster in my AWS account. Below is an infrastructure diagram showing the reverse proxy servers.

The web application Kubernetes deployment consists of a single container running an Nginx server. The API Kubernetes deployment consists of two containers: an Nginx server and a uWSGI server. For the API, the Nginx server is the reverse proxy server and the uWSGI server is the application server, hosting the Python/Flask API.

Let's look at the Docker and Nginx configurations for these containers, back to front; starting with the uWSGI container for the API code. The two important configuration files for the uWSGI container are a Dockerfile and a uwsgi.ini file.

# Dockerfile FROM python:3.8 LABEL maintainer="andrew@jarombek.com" \ version="1.0.0" \ description="Dockerfile for the Flask SaintsXCTF API in Production" RUN apt-get update \ && apt-get install g++ RUN pip install pipenv \ && pip install uwsgi RUN mkdir /src WORKDIR /src COPY Pipfile . COPY Pipfile.lock . RUN pipenv install --system COPY . . ENV FLASK_ENV production ENV ENV prod COPY credentials .aws/ ENV AWS_DEFAULT_REGION us-east-1 ENV AWS_SHARED_CREDENTIALS_FILE .aws/credentials STOPSIGNAL SIGTERM EXPOSE 5000 CMD ["uwsgi", "--ini", "uwsgi.ini"]
; uwsgi.ini [uwsgi] protocol = uwsgi module = main callable = app master = true processes = 5 ; When using an Nginx reverse proxy, use 'socket' socket = :5000 ; When using uWSGI as a server that awaits HTTP requests, use 'http-socket' ; http-socket = :5000

Don't get bogged down by the details of the Dockerfile; the important takeaway is that it installs uWSGI with pip install uwsgi and starts the uWSGI application server with the configuration specified in uwsgi.ini. It does so using the CMD ["uwsgi", "--ini", "uwsgi.ini"] entrypoint.

The uWSGI configuration file is configured to open a socket on port 5000 for incoming requests. These requests are passed from the Nginx reverse proxy server4. The Nginx container for the API has two important configuration files: the Dockerfile and an nginx.conf file.

# Dockerfile FROM nginx:latest LABEL maintainer="andrew@jarombek.com" \ version="1.0.0" \ description="Dockerfile for the Nginx Reverse Proxy to the SaintsXCTF API in Production" RUN rm /etc/nginx/conf.d/default.conf COPY nginx.conf /etc/nginx/conf.d STOPSIGNAL SIGTERM EXPOSE 80
# nginx.conf server { listen 80; root /usr/share/nginx/html; location / { try_files $uri @api; } location @api { include uwsgi_params; uwsgi_pass saints-xctf-api-flask:5000; } }

The important parts of the Dockerfile are the RUN and COPY commands. These commands delete the default Nginx configuration and add the custom Nginx configuration, respectively. The custom Nginx configuration exists in the nginx.conf file.

In the Nginx configuration, the server block creates a virtual server, which in my case is a reverse proxy server5. Inside the server block are two location blocks; each location determines what actions occur when incoming requests are made to certain URLs6. The remaining configuration is a standard Flask/uWSGI Nginx setup found in the Flask documentation7. The uwsgi_pass directive makes the Nginx server a reverse proxy server that proxies requests to the uWSGI application server located at saints-xctf-api-flask:5000. This URL is the DNS location of the uWSGI Docker container on my Kubernetes cluster.

With the uWSGI and Nginx containers created, the infrastructure for my API is in place. The Docker containers are configured to run locally with Docker Compose in a docker-compose-api.yml file and in production with Kubernetes in a main.tf file.

The web application infrastructure consists of a single Nginx Docker container. The two important configuration files for the web application container are a Dockerfile and a nginx.conf file.

# Dockerfile FROM 739088120071.dkr.ecr.us-east-1.amazonaws.com/saints-xctf-web-base:latest AS base WORKDIR src RUN yarn build FROM nginx AS host LABEL maintainer="andrew@jarombek.com" \ version="1.0.0" \ description="Dockerfile for running the SaintsXCTF web application." RUN rm /etc/nginx/conf.d/default.conf COPY nginx.conf /etc/nginx/conf.d COPY --from=base /src/dist /usr/share/nginx/html
# nginx.conf server { listen 80; location / { root /usr/share/nginx/html; try_files $uri /index.html; } location /api { rewrite /api/(.*) /$1 break; proxy_pass https://api.saintsxctf.com; } location /auth { rewrite /auth/(.*) /$1 break; proxy_pass https://auth.saintsxctf.com; proxy_redirect off; proxy_ssl_server_name on; proxy_ssl_protocols TLSv1 TLSv1.1 TLSv1.2; proxy_buffering off; proxy_set_header Host $proxy_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forward-For $proxy_add_x_forwarded_for; proxy_set_header X-NginX-Proxy true; } location /fn { rewrite /fn/(.*) /$1 break; proxy_pass https://fn.saintsxctf.com; proxy_redirect off; proxy_ssl_server_name on; proxy_ssl_protocols TLSv1 TLSv1.1 TLSv1.2; proxy_buffering off; proxy_set_header Host $proxy_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forward-For $proxy_add_x_forwarded_for; proxy_set_header X-NginX-Proxy true; } location /asset { rewrite /asset/(.*) /$1 break; proxy_pass https://asset.saintsxctf.com; proxy_redirect off; proxy_ssl_server_name on; proxy_ssl_protocols TLSv1 TLSv1.1 TLSv1.2; proxy_buffering off; proxy_set_header Host $proxy_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forward-For $proxy_add_x_forwarded_for; proxy_set_header X-NginX-Proxy true; } location /uasset { rewrite /uasset/(.*) /prod/$1 break; proxy_pass https://uasset.saintsxctf.com; proxy_redirect off; proxy_ssl_server_name on; proxy_ssl_protocols TLSv1 TLSv1.1 TLSv1.2; proxy_buffering off; proxy_set_header Host $proxy_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forward-For $proxy_add_x_forwarded_for; proxy_set_header X-NginX-Proxy true; } location /s3 { rewrite /s3/(.*) /$1 break; proxy_pass https://s3.amazonaws.com; } }

The Dockerfile for the web application should look similar to the Nginx Dockerfile for the API. The differences include the use of a base image which installs npm dependencies using yarn, the creation of JavaScript bundles for the frontend application via RUN yarn build, and JavaScript bundles being copied from the base image to the Nginx image via COPY --from=base /src/dist /usr/share/nginx/html.

The Nginx reverse proxy server configuration for the web application is a bit more complex. The server block for the web application has many location blocks. The first location block serves the React.js web application, while each subsequent block proxies requests to different backend servers. These proxies use proxy_pass directives instead of uwsgi_pass directives, since the backend servers are simple HTTPS servers instead of uWSGI applciation servers8.

Similar to the API, the web application is configured to run in production with Kubernetes in a main.tf file. Instead of using Docker Compose locally like the API, the web application configures a local reverse proxy using webpack dev server9. The configuration for webpack dev server exists in a webpack.config.js file.

Reverse proxy servers are a great way to hide backend server locations from end users. I also find them very elegant, as they remove the need to hardcode server URLs in application code. Code for the web application and API referenced throughout this article is available on GitHub.