★ Docker Registry or How to Run your own Private Docker Image Repository

Docker allows to bundle artifacts and configurations in an image. These images run as light weight system-level virtual machines. In my previous articles, I showed how to use Docker in general  and how to use networking. In this article, I will show you how to manage images in a private Docker registry — basically a git repository for images — and run a production grade registry. As in my previous articles, a fully working example is available on Github to get you started with your own private Docker registry.

Every single command in a Dockerfile yields a new Docker image with an individual id similar to a commit in git. This commit can be tagged for easy reference with a Docker Tag. In addition, tags are the means to share images on public and private repositories.

Tags

When you build a Docker image using a Dockerfile, you can set a tag for the final image by passing the parameter -t <tag>. You can also tag any other image with docker tag <image> <tag>. The expression tag has evolved in Docker quite a lot and its meaning blurred. The syntax for a tag is repository:[tag]. So we have a recursive definition of tag ;)

In general, a repository is a collection of images which are hosted in a registry. When you use the public Docker.io repository — also called the Docker Index —, your repository is <username>/<reponame>_ — cf [1].

For example, I shared the registry-demo container to the Docker Index by first tagging and then pushing it:¹

> docker build lukaspustina/registry-demo registry
> docker push lukaspustina/registry-demo

In case you want to use a private registry, you need to set the URL to your private registry as the username. First tag it accordingly and then push it — assuming your registry is listing on localhost port 5000:

> docker tag lukaspustina/registry-demo localhost:5000/registry-demo
> docker push localhost:5000/registry-demo

Registry

You can download the docker registry from here. The configuration of a registry is straight forward and described in the corresponding Readme. You can define different modes a registry should run called flavors, e.g. for debugging, different storage backend etc. Docker registry supports multiple storage backend. You can choose from local file system, Amazon S3, OpenStack Glance etc.

If you run a production registry with multiple workers, it is crucial to set a “secretkey” in the configuration file _config/config.yml to a 64 characters long value. Otherwise, you will get very confusing errors if you run more than one worker thread.

In order to retrieve an image from a repository, run docker pull <name> where name denotes the reponame_ given above. If you want to select a specifically tagged version, you can supply the tag with the -t option.

For an image from the Docker Index, run
> docker pull lukaspustina/registry-demo

and for a private registry, run
> docker pull localhost:5000/registry-demo

Demo

You can find my demo project on GitHub. You can either run it directly on a Docker compatible OS like Linux and OS X or use Vagrantfile to setup a Virtual Box — see the Readme on GitHib for details.

The demo consists of two steps. The first step builds a production-ready Docker Image containing the Docker registry and two Python images for 2.7 and 3.3 which are pushed to the registry.² For this demo, the two Python version are used as two versions of the same artifact ‘python’ to demonstrate how to use tags for versioning images of the same repository.
> make build

You can see how the docker layers are uploaded to the registry:

Pushing repository localhost:5000/python (2 tags)
Image 511136ea3c5a Image successfully pushed
Image cc7385a89304 Image successfully pushed
Image 24ba2ee5d982 Image successfully pushed
Image a49658c12a4f Image successfully pushed
Image 7adcddf9f908 Image successfully pushed
Image 818c514e00c0 Image successfully pushed
Image 480cb1b9f62c Image successfully pushed
Image 9eeddd8616b7 Image successfully pushed
Image c26adcc0bc2a Image successfully pushed
Pushing tag for rev [c26adcc0bc2a] on
  {http://localhost:5000/v1/repositories/python/tags/python-2}
Image c04e8b52daba Image successfully pushed
Image 1ebdf4e79ef1 Image successfully pushed
Pushing tag for rev [1ebdf4e79ef1] on
  {http://localhost:5000/v1/repositories/python/tags/python-3}

You can also see that tags for the two Python version are also transmitted allowing a user to pull only a specific version.

The second step pulls the Python images and run them demonstrating the whole life cycle of build, pushing, pulling, and running image with your own, private Docker registry.
> make start-registry pull run-python-2 run-python-3 stop-registry

You can see how the individual layers of the Python images are gathered and afterwards run:

docker pull localhost:5000/python
Pulling repository localhost:5000/python
1ebdf4e79ef1: Download complete
c26adcc0bc2a: Download complete
511136ea3c5a: Download complete
cc7385a89304: Download complete
24ba2ee5d982: Download complete
a49658c12a4f: Download complete
7adcddf9f908: Download complete
818c514e00c0: Download complete
480cb1b9f62c: Download complete
c04e8b52daba: Download complete
9eeddd8616b7: Download complete
...
docker run localhost:5000/python:python-2
Python version is 2.7.5+
docker run localhost:5000/python:python-3
Python version is 3.3.2+

Running a Production Registry

At CenterDevice we are running a Docker registry on our Jenkins CI server. The main command line to run the registry is
> SETTINGS_FLAVOR=prod DOCKER_REGISTRY_CONFIG=/etc/docker/config.yml gunicorn -k gevent -b localhost:5000 --max-requests 100 --graceful-timeout 3600 -t 3600 -w 8 wsgi:application

which sets the flavor and the location of the configuration file. In addition, it starts 8 worker threads and listens only on localhost. Since the current version of Docker registry has no authentication, we use ssh tunnels for this purpose. The registry instance only listens on localhost. If you want to connect to the registry, you need to create a tunnel first with
> ssh -N -L 5000:localhost:5000 user@server

This requires a valid user on our server and enforcing authentication and authorization. For the storage backend we use S3.

The Dockerfile  in the GitHub demo project is almost the same as the one we use on our Jenkins. Please feel free adapt it to your needs. The main difference is that the demo uses local file storage, but the config.yml template file is already prepared for S3. See the Readme for details. If you’re feeling lazy, you also find this Docker image on Docker Index.

There’s more

At CenterDevice, we like to use private Docker registries because they allow us to safely share Docker images in our organization. We maintain all our backend service as well as our app images in a private registry. In this way, a developer only needs to pull changed images to update his development environment. In the future, we plan to use these images for production environments, too.

Docker is evolving very quickly, with a minor version update planed each month. So stay tuned for more articles about how we use Docker at CenterDevice (German readers can read more about it here).

Footnotes

1. You can run my shared registry directly: > docker run --name registry -d -p 5000:5000 -v <absolute local path>:/docker-registry-storage lukaspustina/registry-demo.↵
2. Please be patient as the build process takes quite some time.↵