An open-source platform called Docker makes designing, shipping, and deploying applications simple. It runs an application in an isolated environment by compiling its dependencies into a so-called container. for additional information on Docker. In a normal case, several services, such as a database and load balancing, are required to support an application.
We'll look at Docker Compose's assistance with setting up many services in this article. Also, we will see a demonstration of installing and utilizing Docker Compose. Let's try to understand docker-compose simply.
Docker Compose will execute a YAML-based multi-container application. The YAML file consists of all configurations needed to deploy containers Docker Compose, which is integrated with Docker Swarm, and provides directions for building and deploying containers. With Docker Compose, each container is constructed to run on a single host.
Key Concepts in Docker Compose
Docker Compose is a powerful tool for managing multi-container applications, and mastering its key components—like services, networks, volumes, and environment variables—can greatly enhance its usage. Let’s break down these concepts and how they work within a Docker Compose file.
Docker Compose configurations are mainly stored in a file named docker-compose.yml, which uses YAML format to define an application’s environment. This file includes all the necessary details to set up and run your application, such as services, networks, and volumes. To use Docker Compose in an effective way you have to know the structure of this file.
Key Elements of the YAML Configuration
- Version: It defines the format of the Compose file, by ensuring compatibility with specific Docker Compose features and syntax.
- Services: The
services section lists each containerized service required for the application. Each service can have its configuration options, such as which image to use, environment variables, and resource limits. - Networks: In network section, you can define custom networks that enable communication between containers. Additionally, it allows you to specify network drivers and custom settings for organizing container interactions.
- Volumes: Volumes allow for data persistence across container restarts and can be shared between containers if needed. They enable you to store data outside the container's lifecycle, making it useful for shared storage or preserving application state.
Example docker-compose.yml
Here’s a sample Compose file that defines two services, a shared network, and a volume:
version: '3.8'
services:
web:
image: nginx:latest
ports:
- "80:80"
networks:
- frontend
volumes:
- shared-volume:/usr/share/nginx/html
depends_on:
- app
app:
image: node:14
working_dir: /app
command: node server.js # Specify a command
networks:
- frontend
volumes:
- shared-volume:/app/data
networks:
frontend:
driver: bridge
volumes:
shared-volume: # Remove incorrect syntax
Explanation:
- The
web service runs an Nginx container, and app runs a Node.js container. - Both services connect through the
frontend network, allowing them to communicate. - The
shared-volume volume is mounted in both containers, providing shared storage for files.
Docker Compose Services
In Docker Compose, every component of your application operates as a separate service, with each service running a single container tailored to a specific role—such as a database, web server, or cache. These services are defined within the `services` section of the `docker-compose.yml` file. This section lets you configure each service individually, specifying details like the Docker image to pull, environment variables, network connections, and storage options. Through this setup, you can control how each part of your application interacts, ensuring smooth communication and resource management across the services.
Key Service Configuration Options
- Image: Image option defines which Docker image we are going to use by the service from the Docker Hub or any other registry.
- Build: Instead of pulling an image, you can build one locally by specifying a directory containing a Dockerfile. The build is ideal for including custom code in your application.
- Ports: This setting maps a container's internal ports to those on the host machine, enabling access to services from outside the container.
- Volumes: Volumes attach persistent storage to a service ensuring that the data remains accessible even when a container restart.
- Environment: Environment variables allow you to pass configurations or sensitive information, like database credentials or API keys, to the service.
- Depends_on: Depends_on controls the startup order of services, ensuring that certain containers are running before others begin.
Example of docker-compose.yml Configuration
Here’s a sample configuration that demonstrates how these options are used:
version: '3.8'
services:
db:
image: postgres:13
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=pass
volumes:
- db_data:/var/lib/postgresql/data
web:
build: ./web
ports:
- "5000:5000"
volumes:
- web_data:/usr/src/app
environment:
- DATABASE_URL=postgres://user:pass@db:5432/mydb
depends_on:
- db
volumes:
db_data:
web_data:
Explanation:
- The
db service runs a PostgreSQL container. It uses environment variables to set up a database username and password, and stores data on the db_data volume to ensure it’s retained. - The
web service is built from a Dockerfile in the ./web directory and exposes port 5000. The web_data volume is mounted to store application files persistently. It depends on the db service, ensuring the database is available when the web service starts.
Docker Compose Networks
Docker Compose deployments use networks to allow secure communications between the services. Services defined in a docker-compose.yml file are by default placed on one network and are able to connect to each other without any additional setup. For more strict control, you can create additional networks and assign services to them in order to control the way they communicate or to separate some groups of services as the need arises.
Key Network Configuration Options
- Driver: Driver are used in the network driver type, such as
bridge (the default for local networks) or overlay (for multi-host networks in Docker Swarm), which determines how services connect to each other. - Driver Options (
driver_opts): Driver Options(driver_opts) allows for additional settings on the network driver, useful for fine-tuning network behavior to meet specific needs. - IP Address Management (
ipam):IP address management configures network-level IP settings, like subnets and IP ranges, to give you greater control over the IP address space assigned to your services.
Example docker-compose.yml with Custom Networks
Below is an example Compose file that sets up two networks, one for database communication and another for web access.
version: '3.8'
services:
db:
image: postgres:13
networks:
- backend
web:
image: nginx:latest
networks:
- frontend
- backend
ports:
- "80:80"
networks:
frontend:
driver: bridge
backend:
driver: bridge
ipam:
config:
- subnet: 172.16.238.0/24
Explanation:
- The
db service uses thebackend network, isolating it from the frontend network to limit access. - The
web service is connected to both frontend and backend networks, allowing it to communicate with the db service while remaining accessible via the frontend network. - The
backend network includes IPAM settings with a specific subnet range, ensuring custom IP address management.
Docker Compose Volumes
Volumesin docker compose are used to persist data created or used by the docker containers. By doing so they enable the data to persist even if containers are stopped or removed in your docker-compose. Within a docker-compose. yml file, the volumes section describes all the volumes that are attached to the services allowing you to manage data that exists independently of the container lifecycle.
Key Volume Configuration Options
- External: Set true to signify that the volume was created externally, outside of Docker Compose (such as via docker volume create) and simply referenced in the configuration.
- Driver: This indicates which volume driver the volume should use, and it controls how are these volumes being handled. By default, the driver is local, but other options are also available.
- Driver Options (driver_opts): Additional options to customize the volume driver like filesystem type or different storage parameters.
Example docker-compose.yml with Volumes
Here’s a practical example showing how to configure a volume for a Pos-tgreSQL database, ensuring that its data is stored persistently.
version: '3.8'
services:
db:
image: postgres:13
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=pass
volumes:
- db_data:/var/lib/postgresql/data
volumes:
db_data:
driver: local
driver_opts:
type: none
o: bind
device: /path/to/local/db_data
Explanation
- The
db service runs a PostgreSQL container, with its data stored in the db_data volume. This setup ensures that the database information remains intact across restarts or removals of the container. - The
db_data volume is configured to use the localdriver, and it has driver options set to create a bind mount pointing to a specific path on the host system (/path/to/local/db_data). This means that all database files are saved in that designated directory on the host. - By using volumes in this way, you can keep essential data safe and easily accessible, separate from the container itself.
Docker Compose Environment Variables
Environment variables are a simple and effective way to pass configuration settings from your host operating system through Docker Compose in order to get to your services. You can set these variables directly on the service definition by using the environment section or load them from an external file.
How to Set Environment Variables in Docker Compose?
- Inline: you may declare env vars directly in the service definition.This approach is simple and gives everything in one place.
- env_file: This option allows you to load environment variables from an external file, making it easier to manage configuration, especially when dealing with many variables.
Example docker-compose.yml Using Environment Variables
Here’s an example that demonstrates both methods of setting environment variables for a web application and a database service.
version: '3.8'
services:
db:
image: postgres:13
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=pass
volumes: - db_data:/var/lib/postgresql/data
web: image: my-web-app:latest
build: ./web
environment:
- DATABASE_URL=postgres://user:pass@db:5432/mydb
env_file:
- .env
volumes:
db_data:
Explanation
- In the
db service, the POSTGRES_USER and POSTGRES_PASSWORD environment variables are defined inline, specifying the database credentials directly. - The
web service uses an inline variable for DATABASE_URL, which connects to the PostgreSQL database. Additionally, it loads environment variables from an external file named .env. This file can contain various settings, such as API keys, application configurations, and other sensitive information.
With a good understanding of these basic principles, developers are ready to use Docker Compose to manage and orchestrate applications that can be quite complex and involve many Docker containers.
Install Docker Compose
We can run Docker Compose on macOs, Widows, and 64-bit Linux.
- For any significant activity, Docker Compose depends on Docker Engine. Depending on your arrangement, we must ensure that Docker Engine is installed either locally or remotely.
- A desktop system such as Docker for Mac and Windows comes with Docker Compose preinstalled.
- Install Docker first as instructed in Docker installation on the Linux system before beginning the installation of Docker Compose.
Install Docker Compose on Ubuntu - A Step-By-Step Guide
Step 1: Update the package Manager
- The following scripts will install the most recent version of Docker Compose and update the package management.
sudo apt-get update
Step 2: Download the Software
- Here we are using the Ubuntu flavor in the Linux Operating system. So the package manager is "apt-get" If you want to install it in Redhat Linux then the package manager will be "yum".
Step 3: Apply Permissions
- Apply the Executable permissions to the software with the following commands:
sudo chmod +x /usr/local/bin/docker-compose
Step 4: Verify the Download Software
- Verify the whether the docker compose is successfully installed or not with the following command:
docker-compose --version
Docker Container
A docker container is a lightweight Linux-based system that packages all the libraries and dependencies of an application, prebuilt and ready to be executed. It is an isolated running image that makes the application feel like the whole system is dedicated to it. Many large organizations are moving towards containers from VMs as they are light and simple to use and maintain. But when it comes to using containers for real-world applications, usually one container is not sufficient. For example, Let's assume Netflix uses a microservices architecture. Then it needs services for authentication, Login, Database, Payment, etc, and for each of these services, we want to run a separate container. It is preferred for a container to have only a single purpose.
Now, imagine writing separate docker files, and managing configuration and networks for each container. This is where Docker Compose comes into the picture and makes our lives easy.
Why Docker Compose?
As discussed earlier, a real-world application has a separate container for each of its services. And we know that each container needs to have a Dockerfile. It means we will have to write maybe hundreds of docker files and then manage everything about the containers individually, That's cumbersome.
Hence we use docker-compose, which is a tool that helps in the definition and running of multi-container applications. With the help of Docker Compose you can start and stop the services by using its YAML file. Docker-compose allows us to start and stop all of the services with just a few simple commands and a single YAML file for each configuration.
In contrast to utilizing a prebuilt image from Docker Hub, which you may configure with the docker-compose.yaml file, if you are using a custom image, you will need to declare its configurations in a separate Dockerfile. These are the features that docker-compose support:
- All the services are isolated running on a single host.
- Containers are recreated only when there is some change.
- The volume data is not reset when creating new containers, volumes are preserved.
- Movement of variables and composition within environments.
- It creates a virtual network for easy interaction within the environments.
Now, let's see how we can use docker-compose, using a simple project.
How to Use Docker Compose?
In this project, we will create a straightforward Restfull API that will return a list of fruits. We will use a flask for this purpose. And a PHP application will request this service and show it in the browser. Both services will run in their own containers.
Step 1: Create Project Directory
- First, Create a separate directory for our complete project. Use the following command.
mkdir dockerComposeProject
- Move inside the directory.
cd dockerComposeProject
Step 2: Create API
we will create a custom image that will use Python to serve our Restful API defined below. Then the service will be further configured using aDockerfile.
- Then create a subdirectory for the service we will name it product. and move into the same.
mkdir product
cd product
Inside the product folder, create a file named requirements.txt and add the following dependencies:
flask
flask-restful
Step 3: Build Python api.py
- The following is the python file that helps in making an API call:
- Create a Dockerfile to define the container in which the above API will run.
from flask import Flask
from flask_restful import Resource, Api
# create a flask object
app = Flask(__name__)
api = Api(app)
# creating a class for Fruits that will hold
# the accessors
class Fruits(Resource):
def get(self):
# returns a dictionary with fruits
return {
'fruits': ['Mango', 'Pomegranate', 'Orange', 'Litchi']
}
# adds the resources at the root route
api.add_resource(Fruits, '/')
# if this file is being executed then run the service
if __name__ == '__main__':
# run the service
app.run(host='0.0.0.0', port=80, debug=True)
Step 4: Create Dockerfile For Python API
FROM python:3
WORKDIR /usr/src/app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "api.py"]
FROM accepts an image name and a version that the docker will download from the docker hub. The current working directory's contents can be copied to the location where the server expects the code to be by using the copy command. Moreover, the CMD command takes a list of commands to start the service once the container has been started.
Step 5: Create PHP HTML Website
Let's create a simple website using PHP that will use our API.
- Move to the parent directory and create another subdirectory for the website.
cd ..
mkdir website
cd website
index.php
<!DOCTYPE html>
<html lang="en">
<head>
<title>Fruit Service</title>
</head>
<body>
<h1>Welcome to India's Fruit Shop</h1>
<ul>
<?php
$json = file_get_contents('http://fruit-service');
$obj = json_decode($json);
$fruits = $obj->fruits;
foreach ($fruits as $fruit){
echo "<li>$fruit</li>";
}
?>
</ul>
</body>
</html>
- Now create a compose file where we will define and configure the two services, API and the website.
- Move out of the website subdirectory using the following code.
cd ..
- And then create the file name as . docker-compose.yaml
Step 6: Create Docker-compose.yaml file
- The following is the sample docker compose file code:
version: "3"
services:
fruit-service:
build: ./product
volumes:
- ./product:/usr/src/app
ports:
- 5001:80
website:
image: php:apache
volumes:
- ./website:/var/www/html
ports:
- 5000:80
depends_on:
- fruit-service
Docker-compose.yaml File
The first line is optional where we specify the version of the docker-compose tool. Next services define a list of services that our application is going to use. The first service is fruit service which is our API and the second one is our website. The fruit service has a property build that contains the dockerfile that is to be built and created as an image. Volumes define storage mapping between the host and the container so that we can make live changes. Finally, port property exposes the containers port 80 through the host's 5001.
The website service does not use a custom image but we download the PHP image from the Docker hub and then map the websites folder that contains our index.php to /var/www/html (PHP expects the code to be at this location). Ports expose the container port. Finally, the depends_on specifies all the services on which the current service depends.
- The folder structure after creating all the required files and directory will be as follows:
Run the application stack with Docker Compose
- Now that we have our docker-compose.yml file, we can run it.
- To start the application, enter the following command.
docker-compose up -d
Now all the services will start and our website will be ready to be used at localhost:5000.
- Open your browser and enter localhost:5000.
Output
- To stop the application, either press CTRL + C or
docker-compose stop
Advantages of Docker Compose
The following are the advantages of Docker Compose:
- Simplifies Multi-Container Management: Docker Compose facilitates with features such as define, configure, and run multiple containers with a single YAML file, streamlining the management of complex applications.
- Facilitates Environment Consistency: It facilitates with the development, testing, and production environments that are consistent with reducing the risk of environment-related issues.
- Automates Multi-Container Workflows: With Docker Compose, you can easily automate the setup and teardown of multi-container environments, making it ideal for CI/CD pipelines and development workflows.
- Efficient Resource Management: It enables efficient allocation and management of resources across multiple containers, improving application performance and scalability.
Disadvantages of Docker Compose
The following are the disadvantages of Docker Compose:
- Limited Scalability: Docker Compose is not developed for large scaling mechanism which can limit its effectiveness for managing complex deployments.
- Single Host Limitation: Docker Compose will operate on a single host, making it unsuitable for distributed applications with requiring multi-host orchestration.
- Basic Load Balancing: It lacks with advanced load balancing and auto-scaling features found in more robust orchestration tools like Kubernetes.
- Less Robust Monitoring: Docker Compose provides minimal built-in monitoring and logging capabilities compared to more comprehensive solutions.
Important Docker Compose Commands
Command | Description | Example |
|---|
docker-compose up | This command starts all the services defined in your docker-compose.yml file. It creates the necessary containers, networks, and volumes if they don’t already exist. You can run it in the background by adding the -d option. | docker-compose up -d |
|---|
docker-compose down | Use this command to stop and remove all the containers, networks, and volumes that were created by docker-compose up. It’s a good way to clean up resources when you no longer need the application running. | docker-compose down |
|---|
docker-compose ps | This command lists all the containers associated with your Compose application, showing their current status and other helpful information. It’s great for monitoring which services are up and running. | docker-compose ps |
|---|
docker-compose logs | This command lets you view the logs generated by your services. If you want to focus on a specific service, you can specify its name to filter the logs, which is useful for troubleshooting. | docker-compose logs web |
|---|
docker-compose exec | With this command, you can run a command inside one of the running service containers. It’s particularly useful for debugging or interacting with your services directly. | docker-compose exec db psql -U user -d mydb |
|---|
docker-compose build | This command builds or rebuilds the images specified in your docker-compose.yml file. It’s handy when you’ve made changes to your Dockerfiles or want to update your images. | docker-compose build |
|---|
docker-compose pull | Use this command to pull the latest images for your services from their respective registries. It ensures that you have the most current versions before starting your application. | docker-compose pull |
|---|
docker-compose start | This command starts containers that are already defined in your Compose file without recreating them. It’s a quick way to get your services running again after they’ve been stopped. | docker-compose start |
|---|
docker-compose stop | This command stops the running containers but keeps them intact, so you can start them up again later using docker-compose start. | docker-compose stop |
|---|
docker-compose config | This command validates and displays the configuration from your docker-compose.yml file. It’s a useful way to check for any errors before you deploy your application. | docker-compose config |
|---|
Best Practices of Docker Compose
The following are the some of the best practices of Docker Compose:
- Use Environment Variables: It is suggestable to store configuration values and secrets in environment variables to keep your
docker-compose.yml clean and secure. - Keep Services Lightweight: It is preferred to design each service to handle a single responsibility to ensure modularity and ease of maintenance.
- Leverage Volumes: Usage of volumes with enhancing in maintaining the persistent data storage, allowing data to persist across container restarts and updates.
- Version Control Your Compose Files: It is preferred to maintain your
docker-compose.yml file in version control (e.g., Git) to track changes and collaborate with your team effectively.
Features of Docker Compose
The following are the features of Docker Compose:
- Multi-Container Deployment: it facilitates with easily define and run applications with multiple containers using a single YAML file.
- Service Isolation: Each service runs in its own container, with ensuring the isolation and reducing conflicts between services.
- Simplified Configuration: It helps in centralizing all the configurations, including networking, volumes, and dependencies, in the
docker-compose.yml file. - Scalability: It provides the effortlessly scaling of services up or down with a single command, allowing for flexible and dynamic resource management.
Conclusion
In this article, we learned about Docker Compose, and why and when to use it. And demonstrated its usuage through a simple project. It helps in automating the creating process of containers through services, networks and volumes with through respective keywords availability. Through using docker compose management and automation of containers and its volumes, networks will be easier.
Similar Reads
DevOps Tutorial DevOps is a combination of two words: "Development" and "Operations." It’s a modern approach where software developers and software operations teams work together throughout the entire software life cycle.The goals of DevOps are:Faster and continuous software releases.Reduces manual errors through a
7 min read
Introduction
What is DevOps ?DevOps is all about automating and streamlining the software development lifecycle so that code moves from development to production quickly, reliably, and securely.Here is how the DevOps model flow works:Stages of DevOps are:Build Stage1. Developers write and organize code, using version control to
6 min read
DevOps LifecycleThe DevOps lifecycle is a structured approach that integrates development (Dev) and operations (Ops) teams to streamline software delivery. It focuses on collaboration, automation, and continuous feedback across key phases planning, coding, building, testing, releasing, deploying, operating, and mon
10 min read
The Evolution of DevOps - 3 Major Trends for FutureDevOps is a software engineering culture and practice that aims to unify software development and operations. It is an approach to software development that emphasizes collaboration, communication, and integration between software developers and IT operations. DevOps has come a long way since its in
7 min read
Version Control
Continuous Integration (CI) & Continuous Deployment (CD)
Containerization
Orchestration
Infrastructure as Code (IaC)
Monitoring and Logging
Microsoft Teams vs Slack Both Microsoft Teams and Slack are the communication channels used by organizations to communicate with their employees. Microsoft Teams was developed in 2017 whereas Slack was created in 2013. Microsoft Teams is mainly used in large organizations and is integrated with Office 365 enhancing the feat
4 min read
Security in DevOps