Understanding Docker and Containers: A Comprehensive Guide

What are Containers

Containers are lightweight, portable units that package an application with its dependencies, libraries, and configuration files, allowing it to run consistently across different environments. Unlike virtual machines, which include an entire operating system, containers share the host OS kernel, making them more efficient in terms of resource usage and startup time. Containers isolate applications at the process level, enhancing portability and scalability, though they share the host kernel and do not provide full OS isolation like VMs.

Key characteristics of containers include:

  • Portability: Run consistently on any system with a compatible container runtime.
  • Lightweight: Minimal overhead compared to VMs, as they don’t include a full OS.
  • Isolation: Processes are isolated, preventing interference between applications.
  • Scalability: Easily replicate and deploy containers for distributed systems.

Containers are the foundation of modern application development, enabling microservices architectures and cloud-native solutions.

Docker - An Overview

Docker is an open-source platform that simplifies the creation, deployment, and management of containers. It provides tools to build, share, and run containerized applications efficiently. Docker uses a client-server architecture, where the Docker client communicates with the Docker daemon, which handles container operations. The core components include:

  • Docker Engine: The runtime that builds and runs containers.
  • Docker Images: Read-only templates containing the application, dependencies, and runtime environment.
  • Docker Containers: Running instances of Docker images.
  • Dockerfile: A script defining how to build a Docker image.

Docker simplifies workflows by standardizing environments, reducing the “it works on my machine” problem, and enabling seamless deployment across development, testing, and production.

Docker Hub

Docker Hub is a cloud-based registry service for sharing and managing Docker images. It serves as a centralized repository where developers can store, distribute, and collaborate on container images. Key features include:

  • Public and Private Repositories: Store images publicly for community use or privately for secure access.
  • Official Images: Curated, secure images for popular software (e.g., Nginx, MySQL, Python).
  • Automated Builds: Automatically build images from GitHub or Bitbucket repositories.
  • Team Collaboration: Manage access and permissions for teams.

Docker Hub is integral for discovering pre-built images and sharing custom ones, streamlining development workflows. Note: Docker Hub enforces rate limits for image pulls, especially for unauthenticated users.

Docker on Windows, Mac, Linux

Docker runs natively on Linux, as it leverages the Linux kernel’s containerization features. For Windows and Mac, Docker Desktop provides a user-friendly interface and virtualization layer to run containers.

  • Windows: Docker Desktop primarily uses WSL 2 (Windows Subsystem for Linux) to run Linux containers. Hyper-V is an alternative backend for older setups. Windows containers are also supported for native Windows applications, but these are distinct from Linux containers.
  • Mac: Docker Desktop uses a lightweight Linux VM (via HyperKit) to run containers, as macOS does not natively support Linux containers.
  • Linux: Docker runs directly on the host kernel, offering optimal performance. Installation is straightforward via package managers like apt or yum.

Docker Desktop on Windows and Mac includes a GUI for managing containers, images, and volumes, while Linux users typically rely on the command line or third-party tools.

Command Line Examples of Docker Desktop

Here are common Docker commands to manage containers and images:

# Pull an image from Docker Hub
docker pull nginx:latest

# Run a container from an image
docker run -d -p 8080:80 --name my-nginx nginx:latest

# List running containers
docker ps

# List all containers (including stopped)
docker ps -a

# Show logs of a running container
docker logs my-nginx

# Access a running container's shell:
docker exec -it my-nginx /bin/bash

# Stop a running container
docker stop my-nginx

# Remove a container
docker rm my-nginx

These commands demonstrate basic operations. Use docker --help for more options or docker <command> --help for specific details.

GPU Passthrough

GPU passthrough allows containers to access the host’s GPU, enabling high-performance tasks like machine learning, gaming, or rendering. Docker supports GPU passthrough primarily for NVIDIA GPUs via the NVIDIA Container Toolkit.

Setup for GPU Passthrough

  • Install the NVIDIA Container Toolkit (modern replacement for nvidia-docker2).
  • Ensure the NVIDIA drivers are installed on the host.
  • Run containers with the --gpus flag:

Note! On Windows 11 the latest NVIDIA Windows GPU Driver will fully support WSL 2. With CUDA support in the driver, existing applications (compiled elsewhere on a Linux system for the same target GPU) can run unmodified within the WSL environment.
Once a Windows NVIDIA GPU driver is installed on the system, CUDA becomes available within WSL 2. The CUDA driver installed on Windows host will be stubbed inside the WSL 2 as libcuda.so, therefore users must not install any NVIDIA GPU Linux driver within WSL 2. 

# Run a container with GPU access
docker run --gpus all -it --rm nvidia/cuda:11.0-base nvidia-smi

This command runs a container with access to all available GPUs and executes nvidia-smi to verify GPU availability. GPU passthrough is critical for compute-intensive workloads. Support for AMD or Intel GPUs is experimental and may require alternative runtimes like ROCm.

7 - Use Cases

Docker and containers have transformed various industries with their versatility. Common use cases include:

  • Microservices: Break down applications into small, independent services for easier scaling and maintenance.
  • CI/CD Pipelines: Use containers to create consistent build and test environments in tools like Jenkins or GitLab CI.
  • Cloud-Native Development: Deploy containers on platforms like Kubernetes or AWS ECS for scalable cloud applications.
  • Development Environments: Create reproducible dev environments with identical dependencies across teams.
  • Machine Learning: Run ML workloads with GPU-accelerated containers for training and inference.
  • Legacy Application Modernization: Containerize monolithic apps to simplify migration to the cloud.
  • Edge Computing: Run containers on edge or IoT devices where lightweight execution is critical.

By enabling consistency, scalability, and portability, Docker and containers have become essential for modern software development and deployment.