Docker and Docker Compose

Containers

Vocab

  • Development Environment (dev)
    • The environment where you write your code
    • Ex. Your laptop
    • Add features; Find and eliminate bugs
  • Production environment (prod)
    • The environment where your app will eventually live
    • The live server with real end users
    • Do everything we can to avoid bugs in production

Deployment Headaches

  • It works on my laptop!
  • Run your code in production and it's broken
  • Many causes
    • Different version of compiler/interpreter
    • Dependencies not linked
    • Hard-coded paths
    • Different environment variables
    • etc.

Virtual Machines

  • Simulate an entire machine
  • Run the virtual machine (VM) in your development environment for testing
  • Run an exact copy of the VM on the production server
  • No more surprise deployment issues
  • Simulating an entire machine can be inefficient
    • If you've run a VM on your laptop you know how slow this can get

Containers

  • Effectively, lightweight VMs
  • Efficiently shares resources across containers
    • As opposed to VMs which require dedicated ram/cpu/disk

Security

  • Can't break out of a VM/Container [Without sophisticated attacks]
  • If an attacker compromises the server, they can only access what you put in the container
    • Can't "rm -rf /" your entire machine
    • Patch the exploited vulnerability and rebuild the image
  • The attacker can still cause significant damage and steal private data
    • They just can't destroy your server box

Security

  • Sometimes an app has to allow code injection attacks to function
    • Autolab
    • AWS
    • Digital Ocean
  • Run user code in their own VM/Container
    • One malicious user cannot take down the entire service

Docker

  • Docker is software that's used to create containers
  • Install Docker in your development environment to test containers
  • Install Docker in your production environment and run the same containers

Dockerfile

  • To start working with Docker, write a Dockerfile
  • This file contains all the instructions needed to build a Docker image
    • Some similarities to a Makefile

Dockerfile

  • Let's explore this sample Dockerfile
  • This Dockerfile creates an image for a node.js app
    • You don't need to know node.js. We're using this example to learn Docker
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • The first line of your Dockerfile will specify the base image
  • This image is downloaded and the rest of your Dockerfile adds to this image
  • In this example: We start with Ubuntu 26.04
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • Use WORKDIR to change your current working directory
    • Same as "cd"
  • This Dockerfile sets the working directory to /app
  • You'll know where your files are if you ever have to navigate the container
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • Use the RUN keyword to run commands in the base image
  • We are starting with a fresh Ubuntu install
    • Need to run any commands needed for your app to run
  • This is the same as running these commands in the terminal
  • Use flags like -y to ensure everything is automated
  • All commands run as root
    • No need for sudo
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • Use COPY to copy local files into your image COPY <local_src> <image_dest>
  • Need to copy all files needed for your app
  • Destination is relative to the working directory that we set on line 3
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • Install any dependencies needed for your app
  • Copy in any files specifying your dependencies (e.g. go.mod and go.sum)
  • Very useful to do this before copying your code
  • Docker has a very good caching system and won't repeat steps where nothing changed
    • Only reinstall dependencies when your dependencies change, not every time your code changes
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • Get your app ready to run
    • Copy all the code for your app into the image
    • If needed, compile your code after copying it
      • Test compiling and running your code locally first (e.g. If you've only ever ran go run . you'll need to learn how to compile and run separately)
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • Finally, use CMD to run your app
  • Important: Do not use RUN to run your app!
  • RUN will execute the command when the image is being built
  • CMD will execute when the container is run
FROM ubuntu:26.04

WORKDIR /app

RUN apt-get update --fix-missing
RUN apt-get install -y nodejs
RUN apt-get install -y npm

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Dockerfile

  • There are many base images to choose from
  • Start with an image with your language installed to simplify your Dockerfile
    • Search "docker language" to find images/tutorials for your other language
  • This image starts with node and npm installed so we can remove the installation lines
FROM node:24

WORKDIR /app

COPY package.json package.json
RUN npm install

COPY server.js server.js
COPY src/ src/

CMD ["node", "server.js"]

Running Your App

  • Now that we have a Dockerfile, build an image using this command
    • docker build -t my-image .
  • But we're interested in containers
    • Images can be used to create many containers (Like a class being used to create objects)
  • To create a container from an image, run this command
    • docker run -p 8080:8080 my-mage
    • The -p 8080:8080 will map local port 8080 to container port 8080
    • Need to map the port for your app to be accessible from outside the container (e.g. From your browser)
  • Your app is now running in a docker container

Running Your App

  • When preparing your app to run in a container
    • Do not use localhost [in your code]
    • Use 0.0.0.0 as the host instead
    • This allows your app to be accessed from outside the container by listening on all network interfaces
  • Go does this by default when using the net package

Docker Compose

Docker Compose

  • Docker compose manages building/running docker images/containers
  • Build and run with one command docker compose up --build --force-recreate --renew-anon-volumes
  • No need to use docker build and docker run
  • Will be used to manage multiple containers
    • Separate container for your database
  • Configure using a file named compose.yaml, compose.yml, or docker-compose.yml

Docker Compose

  • CAUTION: DO NOT store passwords in plain text like this!
    • Fine for Homework 1
    • We will explore a better setup for homework 2
  • The compose.yaml file uses the YAML format
    • White space (indentation) matters
    • Format: <key>:<value>
    • List values prefixed with -
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • List of all the services for docker compose to run
  • A docker container is created for each service
  • This compose file will create 2 containers
    • One for a mySQL database named mysql
    • One for a web server named app
  • The name of each service will become its hostname when using network protocols across containers
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • For each service, specify how to build its image
  • Use image to specify an image to use
    • Same as having a 1 line Dockerfile
    • FROM mysql
    • Useful when you need very little customization
  • Use build to specify a local Dockerfile
    • Specify the directory where the Dockerfile can be found
    • Using . to specify the current directory
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • Map a local port to a container port
  • Same as using -p 8080:8080 when running a single container
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • Use environment to set any needed environment variables
  • When using MySQL, the image expects certain environment variables that are used for setup
    • This is how you setup your username/password and the name of your database
    • Hint: Postgres uses the same concept, but with different variable names
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • You can also set environment variables in your app container
    • Makes for a cleaner setup and can avoid storing plain-text passwords in your code
    • When we learn about authentication, we will discus a better way to handle passwords
      • Brief version: Use a .env file and use ${VARIABLE_NAME} to read your variables
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • We have a detrimental race condition
  • Docker compose will start both containers concurrently
  • Your code will likely connect to your database on startup
    • What if the database is still starting up and not accepting connections yet?
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • Many different solutions to the race
  • Find and implement your favorite approach:
    • Handle rejected connections in your code gracefully and retry
    • Use docker compose's depends_on with healthcheck (After reading the documentation)
    • Use a 3rd party solution like ufoscout's solution on GitHub
services:
  mysql:
    image: mysql
    environment:
      MYSQL_ROOT_PASSWORD: '7PtypF^*6&to@PAhwaw5'
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
  app:
    build: .
    environment:
      MYSQL_USER: mysql_username
      MYSQL_PASSWORD: 'WTc&$5#jAd6jRe9gJ0nL'
      MYSQL_DATABASE: my_db
    ports:
      - "8080:8080"

Docker Compose

  • When communicating across containers, the service name is the container's hostname

  • Connection string:

    postgres://user:pass@HOST:5432/db

  • Running with go run .

    HOST = localhost

  • Running with docker compose up

    HOST = postgres

services:
  postgres:
    image: postgres
    ...
  app:
    build: .
    ...
    ports:
      - "8080:8080"

Docker Compose

  • Note that you choose your service names

  • Be consistent!

    HOST = my_super_cool_service_name

  • Must match the service name in compose.yaml

services:
  my_super_cool_service_name:
    image: postgres
    ...
  app:
    build: .
    ...
    ports:
      - "8080:8080"

Running Your App

  • To run your app
    • docker compose up
  • To rebuild and restart the containers
    • docker compose up --build --force-recreate --renew-anon-volumes
    • Will ensure your newest code is running in the containers
    • Renewing anonymous volumes will destroy your database
      • Remove this if you want your data to persist
  • To restart the container without rebuilding
    • docker compose restart

Further Reading