Docker Volume & Docker Network [Day 19 Task]

1. Docker Volume

A Docker volume is a mechanism provided by Docker to manage and persist data used by containers.

Containers are generally designed to be ephemeral, meaning that their filesystem is isolated and does not persist after the container stops or is removed. So, the data doesn't persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it. However, in many cases, you might want to keep certain data or files even after the container has stopped or been replaced, such as databases, configuration files, or application data.

Docker volumes address this need by providing a way to store and manage data outside of the container itself, allowing data to persist across container lifecycles. Docker has types of Volumes:

Volumes user-friendly volume names managed by Docker.
Bind mounts mounting a specific host directory into the container.

The volumes are stored on the host, independent of the container life cycle. This allows users to back up data and share file systems between containers easily.

Volumes: Volumes are stored in a part of the host filesystem, managed by Docker (/var/lib/docker/volumes/ on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker.

A given volume can be mounted into multiple containers simultaneously. When no running container uses a volume, the volume is still available to Docker and not removed automatically. But we can remove unused volumes using Docker volume prune.

we can create a Volume by using docker volume create <volume name>

To check whether the volume was created or not, use the command docker volume ls
Bind mounts: Bind mounts may be stored anywhere on the host system. They may be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time. Bind mounts have limited functionality compared to volumes.

When we use a bind mount, a file or directory on the host machine is mounted into a container. The file or directory is referenced by its full path on the host machine. The file or directory does not need to exist on the Docker host already. It is created on-demand if it does not yet exist.
Tmpfsmounts: tmpfs mounts are stored in the host system's memory only and are never written to the host system's filesystem. It is not persisted on disk, either on the Docker host or within a container.

2. Docker Network

For Docker containers to communicate with each other and the outside world via the host machine, there has to be a layer of networking involved. Docker supports different types of networks, each fit for certain use cases.

For example, building an application that runs on a single Docker container will have a different network setup as compared to a web application with a cluster with database, application, and load balancers that span multiple containers that need to communicate with each other.

Types Of Network

The most common network types are bridge, overlay, and macvlan.

Bridge Networks:

Bridge networking is the most common network type. It is limited to containers within a single host running the Docker engine. Bridge networks are easy to create, manage and troubleshoot.

For the containers on the bridge network to communicate or be reachable from the outside world, port mapping needs to be configured. As an example, consider you can have a Docker container running a web service on port 80. Because this container is attached to the bridge network on a private subnet, a port on the host system like 8000 needs to be mapped to port 80 on the container for outside traffic to reach the web service.

Syntax:

$ docker network create -d bridge mynetwork [network name]

Overlay Network:

An overlay network uses software virtualization to create additional layers of network abstraction running on top of a physical network. In Docker, an overlay network driver is used for multi-host network communication. This driver utilizes Virtual Extensible LAN (VXLAN) technology which provides portability between cloud, on-premise and virtual environments. VXLAN solves common portability limitations by extending layer 2 subnets across layer 3 network boundaries, hence containers can run on foreign IP subnets.

Syntax:

$ docker network create -d overlay --subnet=192.168.10.0/24 my-overlay-net

--subnet parameter to specify the network block that Docker will use to assign IP addresses to the containers.

Macvlan Networks:

The macvlan driver is used to connect Docker containers directly to the host network interfaces through layer 2 segmentation. No use of port mapping or network address translation (NAT) is needed and containers can be assigned a public IP address that is accessible from the outside world. Latency in Macvlan networks is low since packets are routed directly from the Docker host network interface controller (NIC) to the containers.

Macvlan has to be configured per host and has support for physical NIC, sub-interface, network bonded interfaces and even teamed interfaces.

3. Task 1

Create a multi-container docker-compose file that will bring UP and bring DOWN containers in a single shot ( Example - Create application and database container ).

Use the docker-compose-up command with the -d flag to start a multi-container application in detached mode.
Use the docker-compose scale command to increase or decrease the number of replicas for a specific service. You can also add replicas in the deployment file for auto-scaling.
Use the docker-compose ps command to view the status of all containers, and docker-compose logs to view the logs of a specific service.
Use the docker-compose down command to stop and remove all containers, networks, and volumes associated with the application.

Step 1: Create a multi-container docker-compose file that will bring UP and bring DOWN containers in a single shot ( Example - Create application and database container ).

By using the docker-compose -d up command to start the multi-container application in detached mode.

Step2:

By using the docker-compose up -d --scale webserver=3 command you can scale your container service.

Note: I faced an error of port 8001 already allocated. To solve this issue I use the Dynamic Port Assignment: If you don't want to manually manage port numbers, you can let Docker Compose dynamically assign ports to your containers. To do this, remove the ports configuration from your docker-compose.yml file for the compose-webserver service.

Step3:

By using the docker-compose ps command you can check your running container status.

Step 4:

You can check your container logs of a particular service.

Step 5: Make the container down :

docker-compose down: It stops and removes the containers, networks and volumes

4. Task 2

Learn how to use Docker Volumes and Named Volumes to share files and directories between multiple containers.
Create two or more containers that read and write data to the same volume using the docker run --mount command.
Verify that the data is the same in all containers by using the docker exec command to run commands inside each container.
Use the docker volume ls command to list all volumes and the docker volume rm command to remove the volume when you're done.

use Docker Volumes and Named Volumes to share files and directories between multiple containers.

Create a volume as per your choice and then list the volume list.
Create two or more containers that read and write data to the same volume using the docker run --mount command.

Create two containers and attached the same name volume to them by using the --mount flag.
Verify that the data is the same in all containers by using the docker exec command to run commands inside each container.

Here you can see two containers are using the same data [data1].
Use the docker volume ls command to list all volumes and the docker volume rm command to remove the volume when you're done.

If you want to remove volume, first you have to stop both of the containers and remove them completely. Then you can remove the volume.