Docker overview
by Wenwei Weng
Docker is becoming a very popular in virtualization world.
What is Docker? It is an open-source project that automates the deployment of applications inside linux container.
What is Linux container? It provides “operation-system level virtualization” on Linux, i.e. it provides a way for user to create a userspace enviroment(root file system) on top of underlying host linux kernel.
What is needed from Linux kernel? In general, it requires kernel 3.10+, with namespace, cgroup, plus libcontainer.
The following diagram shows the overall picture of Docker.
In this post, I will share what I learned about Docker.
Docker main components
- Host: the machine that is running the containers.
- Image: a hierarchy of files, with meta-data for how to run a container.
- Container: a contained running process, started from an image.
- Registry: a repository of images.
- Volume: storage outside the container.
- Dockerfile: a script for creating images.
Docker installation
There a a few Linux distributions supports docker, such as ubuntu, centos, Redhat, Fedora, coreos.
Let’s take ubuntu 14.04 desktop system which runs kernel 3.16 as example.
root@weng-VirtualBox:/home/weng# uname -a
Linux weng-VirtualBox 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 17:43:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
root@weng-VirtualBox:/home/weng# cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.2 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.2 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
root@weng-VirtualBox:/home/weng# In ubuntu, typically we can use apt-get to install application. However for Docker, the situation is a bit different. Because Docker is under actively development, we want to install pretty new version, as we can see below, the Docker version in ubuntu 14.04 repository, it is a bit too old: version 1.0.1
root@weng-VirtualBox:/home/weng# apt-cache search docker
pidgin - graphical multi-protocol instant messaging client for X
docker - System tray for KDE3/GNOME2 docklet applications
docker.io - Linux container runtime
karbon - vector graphics application for the Calligra Suite
kdocker - lets you dock any application into the system tray
vim-syntax-docker - Docker container engine - Vim highlighting syntax files
root@weng-VirtualBox:/home/weng# apt-cache policy docker.io
docker.io:
Installed: (none)
Candidate: 1.0.1~dfsg1-0ubuntu1~ubuntu0.14.04.1
Version table:
1.0.1~dfsg1-0ubuntu1~ubuntu0.14.04.1 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/universe amd64 Packages
0.9.1~dfsg1-2 0
500 http://us.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
root@weng-VirtualBox:/home/weng# The following shows how to configure to install the latest available Docker:
root@weng-VirtualBox:/home/weng# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --homedir /tmp/tmp.XDMDaqn5G2 --no-auto-check-trustdb --trust-model always --keyring /etc/apt/trusted.
gpg --primary-keyring /etc/apt/trusted.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
gpg: requesting key A88D21E9 from hkp server keyserver.ubuntu.com gpg: key A88D21E9: public key "Docker Release Tool (releasedocker)
<docker@dotcloud.com>" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
root@weng-VirtualBox:/home/weng# sh -c "echo deb https://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
root@weng-VirtualBox:/home/weng# apt-get update
....
root@weng-VirtualBox:/home/weng# apt-get install lxc-dockerAfter installation, we can verify the status of Docker:
root@weng-VirtualBox:/home/weng# docker --version
Docker version 1.6.2, build 7c8fca2
root@weng-VirtualBox:/home/weng# docker info
Containers: 17
Images: 39
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 73
Dirperm1 Supported: true
Execution Driver: native-0.2
Kernel Version: 3.16.0-30-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 1
Total Memory: 1.955 GiB
Name: weng-VirtualBox
ID: 2GAF:7I7A:E467:UWN2:6GDS:EXNL:QDOX:ZORL:I6RR:JSXV:2Q3K:54QR
WARNING: No swap limit support
root@weng-VirtualBox:/home/weng# For system like Centos, to install latest, the following instructions are to be run to install binary directly:
# To install, run the following command as root:
curl -sSL -O https://get.docker.com/builds/Linux/x86_64/docker-1.6.2 && chmod +x docker-1.6.2 && sudo mv docker-1.6.2 /usr/local/bin/docker
# Then start docker in daemon mode:
sudo /usr/local/bin/docker -dFor more information about installation of docker, check [here] (https://docs.docker.com/installation/)
How Does Docker Work?
Docker is implemented as a client-server mode; The Docker daemon runs on the Host and it is accessed via a socket connection from the client. The client may, but does not have to, be on the same machine as the daemon. The Docker CLI client works the same way as any other client but it is usually connected through a Unix domain socket instead of a TCP socket. The daemon receives commands from the client and manages the containers on the Host where it is running.
root@weng-VirtualBox:/home/weng# ps -ef | grep docker
root 1311 1 0 22:48 ? 00:00:00 /usr/bin/docker -d
root 2773 2311 0 23:04 pts/1 00:00:00 grep --color=auto docker
root@weng-VirtualBox:/home/weng# which docker
/usr/bin/docker
root@weng-VirtualBox:/home/weng# In above, it show Docker daemon runs as regular linux host process 1311.
“/usr/bin/docker” is to be invoked as Docker client to send commands to Docker daemon.
When Docker daemon runs, it uses /etc/default/docker as a configuration file:
root@weng-VirtualBox:/home/weng# cat /etc/default/docker
# Docker Upstart and SysVinit configuration file
# Customize location of Docker binary (especially for development testing).
#DOCKER="/usr/local/bin/docker"
# Use DOCKER_OPTS to modify the daemon startup options.
#DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4"
# If you need Docker to use an HTTP proxy, it can also be specified here.
#export http_proxy="http://127.0.0.1:3128/"
# This is also a handy place to tweak where Docker's temporary files go.
#export TMPDIR="/mnt/bigdrive/docker-tmp"
root@weng-VirtualBox:/home/weng# Run Docker container
The following are commands to for client to interact with containers:
# Commands for interacting with containers
$ docker create # creates a container but does not start it.
$ docker run # creates and starts a container.
$ docker stop # stops it.
$ docker start # will start it again.
$ docker restart # restarts a container.
$ docker rm # deletes a container.
$ docker kill # sends a SIGKILL to a container.
$ docker attach # will connect to a running container.
$ docker wait # blocks until container stops.
$ docker exec # executes a command in a running container.For more details, use “docker help”.
Below is an example of starting a container using Fedora image:
root@weng-VirtualBox:/home/weng# docker run -it fedora /bin/bash
Unable to find image 'fedora:latest' locally
latest: Pulling from fedora
48ecf305d2cf: Pull complete
ded7cd95e059: Already exists
fedora:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:10ba981a70632d7764c21deae25c6521db6d39730e1dd8caff90719013858a7b
Status: Downloaded newer image for fedora:latest
[root@c3cfa2c8ba3f /]# cat /etc/os-release
NAME=Fedora
VERSION="22 (Twenty Two)"
ID=fedora
VERSION_ID=22
PRETTY_NAME="Fedora 22 (Twenty Two)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:22"
HOME_URL="https://fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=22
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=22
PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy
[root@c3cfa2c8ba3f /]# uname -a
Linux c3cfa2c8ba3f 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 17:43:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@c3cfa2c8ba3f /]# ls /dev/
console full kcore null pts shm stdin tty zero
fd fuse mqueue ptmx random stderr stdout urandom
[root@c3cfa2c8ba3f /]# Here there are a few points to make:
- Command "docker run -it fedora /bin/bash" is invoked by user so that docker client send requests to docker daemon through UNIX socket to start a conatiner using fedora docker image, then run /bin/bash command.
- In the output of first line "Unable to find image 'fedora:latest' locally", it means no fedora image available, then it goes to default docker registry to fetch fedora docker image, nice!
- After fedora image is pulled and verified, container is started as instance ID c3cfa2c8ba3f, and runs command /bin/bash.
- cat /etc/os-release does tell that container is running Fedora image.
- "uname -a" tells it runs on top of kernel 3.16.0-30-generic from ubuntu build, which is host kernel.
- "ls /dev" shows very limted number of device files, because these are only devices available to container by default. Additional device can be exposed to container by using "--device" option.
From another xterm, we can verify the container running:
root@weng-VirtualBox:/home/weng# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c3cfa2c8ba3f fedora:latest "/bin/bash" 23 minutes ago Up 23 minutes stupefied_perlman
root@weng-VirtualBox:/home/weng# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
dockerlab/webserver 0.2.0 371419fd7fc5 9 hours ago 180.1 MB
localhost:5000/devenv-weng latest 430f440ac0e0 12 hours ago 226 MB
wengcentos/webserver version-0.1 5db16a40e56e 30 hours ago 295 MB
ubuntu 14.04 fa81ed084842 9 days ago 188.3 MB
ubuntu latest fa81ed084842 9 days ago 188.3 MB
ubuntu 12.04 5c97af892079 9 days ago 133.4 MB
fedora latest ded7cd95e059 13 days ago 186.5 MB
registry latest d849e35be7b0 2 weeks ago 413.9 MB
cirros latest d8de71c04044 7 weeks ago 7.698 MB
centos latest fd44297e2ddb 7 weeks ago 215.7 MB
root@weng-VirtualBox:/home/weng# It is interesting to note that even after exiting from container, the container is still there, meaning the configuration/meta-data about the conatiner is still in the host, unless using “docker rm $containerID” to remove it. See example below “docker ps -a” shows all existing containers, even thoug only one container is active.
root@weng-VirtualBox:/home/weng# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c3cfa2c8ba3f fedora:latest "/bin/bash" 37 minutes ago Up 37 minutes stupefied_perlman
cbf8cb80fb88 ubuntu:latest "/bin/bash" 39 minutes ago Exited (0) 39 minutes ago stupefied_fermi
0f28e4e66ffb ubuntu:latest "/bin/bash" 7 hours ago Exited (0) About an hour ago fervent_hypatia
736872d3afef ubuntu:latest "/bin/bash" 7 hours ago Exited (0) 7 hours ago angry_lalande
125e90b41314 ubuntu:latest "/bin/bash" 7 hours ago agitated_bell
7a06a0f8d2cb ubuntu:latest "/bin/bash" 7 hours ago Exited (0) 7 hours ago berserk_babbage
6d1dfd58d2e7 dockerlab/webserver:0.2.0 "/usr/sbin/apache2 - 9 hours ago Exited (0) 8 hours ago 0.0.0.0:8989->80/tcp hopeful_galileo
17648f98b2eb 566a8b1441e5f5224476ee76d83e944b50ce714a246489e76a76ab851bc6f40f:latest "/bin/sh -c 'apt-get 9 hours ago Exited (1) 9 hours ago serene_pike
28769fdee95d ubuntu:latest "/bin/bash" 10 hours ago Exited (0) 9 hours ago sharp_banach
7c32935fbec1 ubuntu:latest "/bin/bash" 12 hours ago Exited (0) 12 hours ago reverent_jones
0a27428cedda registry:latest "docker-registry" 12 hours ago Exited (0) 8 hours ago 0.0.0.0:5000->5000/tcp adoring_einstein
140c03c2aea3 wengcentos/webserver:version-0.1 "/bin/bash" 31 hours ago Exited (0) 8 hours ago modest_galileo
58a5029f6c3b ubuntu:latest "/bin/bash" 33 hours ago Exited (0) 12 hours ago reverent_stallman
dd07b42d33ae ubuntu:latest "/bin/bash" 33 hours ago Exited (0) 33 hours ago trusting_curie
b22f1cd231f4 centos:latest "/bin/bash" 33 hours ago Exited (1) 31 hours ago modest_engelbart
30b927bd5b2c cirros:latest "/bin/sh" 33 hours ago Exited (0) 8 hours ago stupefied_morse
cce87dd65fa2 cirros:latest "/bin/sh" 33 hours ago Exited (0) 33 hours ago boring_lovelace
acf99358d8c6 cirros:latest "/bin/bash" 33 hours ago clever_babbage
root@weng-VirtualBox:/home/weng#
root@weng-VirtualBox:/home/weng# docker rm acf99358d8c6
acf99358d8c6
root@weng-VirtualBox:/home/weng# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c3cfa2c8ba3f fedora:latest "/bin/bash" 38 minutes ago Up 38 minutes stupefied_perlman
cbf8cb80fb88 ubuntu:latest "/bin/bash" 40 minutes ago Exited (0) 40 minutes ago stupefied_fermi
0f28e4e66ffb ubuntu:latest "/bin/bash" 7 hours ago Exited (0) About an hour ago fervent_hypatia
736872d3afef ubuntu:latest "/bin/bash" 7 hours ago Exited (0) 7 hours ago angry_lalande
125e90b41314 ubuntu:latest "/bin/bash" 7 hours ago agitated_bell
7a06a0f8d2cb ubuntu:latest "/bin/bash" 7 hours ago Exited (0) 7 hours ago berserk_babbage
6d1dfd58d2e7 dockerlab/webserver:0.2.0 "/usr/sbin/apache2 - 9 hours ago Exited (0) 8 hours ago 0.0.0.0:8989->80/tcp hopeful_galileo
17648f98b2eb 566a8b1441e5f5224476ee76d83e944b50ce714a246489e76a76ab851bc6f40f:latest "/bin/sh -c 'apt-get 9 hours ago Exited (1) 9 hours ago serene_pike
28769fdee95d ubuntu:latest "/bin/bash" 10 hours ago Exited (0) 9 hours ago sharp_banach
7c32935fbec1 ubuntu:latest "/bin/bash" 12 hours ago Exited (0) 12 hours ago reverent_jones
0a27428cedda registry:latest "docker-registry" 12 hours ago Exited (0) 8 hours ago 0.0.0.0:5000->5000/tcp adoring_einstein
140c03c2aea3 wengcentos/webserver:version-0.1 "/bin/bash" 31 hours ago Exited (0) 8 hours ago modest_galileo
58a5029f6c3b ubuntu:latest "/bin/bash" 33 hours ago Exited (0) 12 hours ago reverent_stallman
dd07b42d33ae ubuntu:latest "/bin/bash" 33 hours ago Exited (0) 33 hours ago trusting_curie
b22f1cd231f4 centos:latest "/bin/bash" 33 hours ago Exited (1) 31 hours ago modest_engelbart
30b927bd5b2c cirros:latest "/bin/sh" 33 hours ago Exited (0) 8 hours ago stupefied_morse
cce87dd65fa2 cirros:latest "/bin/sh" 33 hours ago Exited (0) 33 hours ago boring_lovelace
root@weng-VirtualBox:/home/weng# Docker image
One of most amazing features of Docker to me is the Docker image management. It is done in an excellent fashion.
- Docker containers are constructed by sequentially mounting a set of file systems from one or more images.
- A Docker image is a file system layer with an optional parent image reference.
- Layered images tend to supply one specific feature on top of parent image
- Upper layer files mask files at lower layers with the same pathname
- An image with no parent image is called a Base image, e.g. ubuntu, centos, Debian, fedora, cirros
- Image file systems are immutable within container.
- Allow one image to support multiple container instances with repeatable results
- Reduces the disk and memory footprint of a given set of containers which share the same read only images
- Conatiners have a writable file system
- The container file system is initially empty
- All writes go to this file system and overlay any matching underlying image files
- In this way, container file systems contain only the delta between their file system state and that of their underlying images.
- The view from the top down, including all file systems in the stack, is called the union file system.
- Docker bootfs is a special layer.
- Supplies the in memory file system interface to the kernel
- Supplies the kernel library interface
Docker filesystem need backend support.
- Docker depends on the efficient use of layered images. Various file system features in kernel implement file system layering.
- Each container has two layers: 1. init layer : child of specified image; 2. child of init layer: container specific data.
- Committing a container creates a new image layer based on the image the container was created from.
- Docker backends file system supported (in preference order)
- aufs: Advanced multi layered Unification FileSystem.
- btrfs
- devicemapper: LVM
- vfs: no shared storage
- OverlayFS
- Container independant Volumes:
- write-heavy IO (database, logs)
- host volumes can be accessed across multiple concurrent containers
Docker Repositories & Registries
- A repository is a hosted collection of tagged images that together create the file system for a container.
- A registry is a service that stores repositories and provides an HTTP API for managing the upload and downloading of repositories.
- Default registry used by docker engine is Docker hub public registry: http://hub.docker.com
- Companies can deploy private registries using open source solution like Docker registry from Docker Inc, Artifactory.
Dockerfile
Dockerfile is just like a shell script, and it is understood by docker, can build image by parsing Dockefile.
Below is an example of Dockerfile
Subscribe via RSS