Docker GitLab & GitLab CI

GitHub is a popular website for hosting code using the git version control system. It's free for open source projects and plans for hosting 5 private projects start at $7/month. With $5/month you can host hundreds of projects on your DigitalOcean droplet with GitLab. GitLab is an open source alternative to GitHub. It looks like and works very similarly to GitHub. After using it for a while, you may even like it more than GitHub.

GitLab CI (Continuous Integration) is an application for running automated tests. It's similar to Travis CI that's integrated with GitHub, which is also free for public open source projects only.

Here's how to to set up GitLab and GitLab CI on DigitalOcean in minutes. We're going to use Docker for deployment which will let you get up and running quickly and will run GitLab in isolated containers which means you can try this on your existing droplets.

This has been tested with the following software versions:

• Ubuntu Server 14.04 LTS
• GitLab 7.4.2
• GitLab CI 5.1.0

Docker

Docker is a deployment tool. Applications deployed with Docker are already pre-built (and can be rebuilt from scratch) and are run in isolated containers. It means fast deployment and Dockerized applications won't interfere with other services running on your server. It's also very simple to remove Docker applications without leaving anything behind.

Installation

Ubuntu Server ships an old version of Docker. It is recommended to use the latest stable version.

We'll need to add a custom package repository and install the docker package.

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
sudo sh -c "echo deb https://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
sudo apt-get update
sudo apt-get install -y lxc-docker

You could also install Docker simply by downloading an installation script and executing it. The advantage of this method is that it's a simple one-liner and can also perform other necessary actions for the installation. However, security experts frown upon doing this since the contents of the script may change in the future and execute other unexpected commands.

curl -sSL https://get.docker.com/ubuntu/ | sudo sh

Using Docker requires administrative privileges i.e. you would have to run all commands as the super user (root) with the command sudo docker. Add yourself to the docker group so that you don't have to use sudo all the time.

sudo gpasswd -a ${USER} docker
sudo service docker restart

If you're getting permission denied errors when using the docker command without sudo after this, log out and log in again.

GitLab

GitLab is a ruby application and will run in its own isolated container.

In addition, GitLab requires two databases to run. We're going to run them in separate containers as well.

Redis

Redis is an in-memory database which is used for storing temporary data. Restarting this database will cause all data to be lost but this is expected behavior.

Launch redis:

docker run --name=gitlab-redis -d sameersbn/redis:latest

First, Docker will check if an image (i.e. pre-installed application) called redis, created by user sameersbn with the latest version exists. If it doesn't, it will download it from the Docker cloud which is a collection of freely available Docker images. If you want to download an image but not run it yet, you can do it with docker pull sameersbn/redis:latest.

latest specifies the version to download and is the default version so it can be omitted. So docker pull sameersbn/redis will also download the latest version. This guide was written when the latest version was x.x.x. If there's problems with the latest version, replace latest with x.x.x i.e. docker pull sameersbn/redis:x.x.x.

You can see the downloaded images with docker images.

$ docker images
REPOSITORY         TAG       IMAGE ID         CREATED          VIRTUAL SIZE
sameersbn/redis    latest    a7a9de7127c6     5 days ago       207.4 MB

A Docker image is like a template (a class in programming terms). A Docker container is a running version of this template (a concrete instance in programming terms). You can run several containers from the image. So you would have one redis image and you could run two redis servers created from this image.

You can see the list of running containers with docker ps. You can see all containers (including stopped ones) with docker ps -a.

$ docker ps
CONTAINER ID    IMAGE                    COMMAND    CREATED      STATUS     PORTS      NAMES
bfd4a817b99f    sameersbn/redis:latest   "/start"   2 days ago   Up 2 days  6379/tcp   gitlab-redis

Docker by default will assign random names to containers so it's useful to name them and --name=gitlab-redis does just that.

-d means that the container will be launched in background as a daemon. You can see the output with docker logs gitlab-redis.

PostgreSQL

Launch PostgreSQL

docker run --name=gitlab-postgresql -d \
  -e 'DB_NAME=gitlab' -e 'DB_USER=gitlab' -e 'DB_PASS=gitlab' \
  -v /opt/gitlab/db:/var/lib/postgresql \
  sameersbn/postgresql

GitLab

Launch GitLab

docker run --name=gitlab -d \
  -e 'GITLAB_PORT=10080' -e 'GITLAB_SSH_PORT=10022' \
  -e 'GITLAB_BACKUPS=daily' \
  -p 10022:22 -p 10080:80 \
  -v /opt/gitlab/repos:/home/git/data \
  -v /var/run/docker.sock:/run/docker.sock \
  -v $(which docker):/bin/docker \
  --link gitlab-postgresql:postgresql \
  --link gitlab-redis:redisio \
  sameersbn/gitlab

This may take a while. You can see what's going on with docker logs gitlab.

Go to http://localhost:10080 and log in with

• Username: root
• Password: 5iveL!fe

GitLab CI

Launch redis

docker run --name=gitlab-ci-redis -d sameersbn/redis

Launch PostgreSQL

docker run --name=gitlab-ci-postgresql -d \
  -e 'DB_NAME=gitlab_ci_production' -e 'DB_USER=gitlab_ci' -e 'DB_PASS=gitlab_ci' \
  -v /opt/gitlab-ci/db:/var/lib/postgresql \
  sameersbn/postgresql

Launch GitLab CI

docker run --name=gitlab-ci -d \
  -p 11080:80 \
  -v /opt/gitlab-ci/data:/home/gitlab_ci/data \
  -v /var/run/docker.sock:/run/docker.sock \
  -v $(which docker):/bin/docker \
  --link gitlab:gitlab \
  --link gitlab-ci-postgresql:postgresql \
  --link gitlab-ci-redis:redisio \
  sameersbn/gitlab-ci

Go to http://localhost:11080 and log in with your GitLab credentials.

GitLab CI Runners

Note down the registration token at http://localhost:11080/admin/runners.

export CI_TOKEN=xxx
export CI_HOST=`hostname -I | cut -d' ' -f1`

Mono C#/.NET

Let's create a runner for testing C#/.NET applications with mono.

You're probably going to modify it several times to get it right, so we're going also going to create a Dockerfile and Makefile instead of using a one-liner.

mkdir -p ~/gitlab-ci-runners/mono
cd ~/gitlab-ci-runners/mono

Dockerfile

cat > Dockerfile <<END
FROM sameersbn/gitlab-ci-runner:latest
MAINTAINER me@localhost

RUN apt-get update && apt-get install -y curl && \
    curl -s http://download.mono-project.com/repo/xamarin.gpg | apt-key add - && \
    echo "deb http://download.mono-project.com/repo/debian wheezy main" > /etc/apt/sources.list.d/xamarin.list && \
    apt-get update && \
    apt-get -y install mono-complete && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN mozroots --import --sync --machine && echo "y\n\y\n" | certmgr -ssl -m https://nuget.org
END

Makefile

cat > Makefile <<END
all: build

build:
        @docker build --tag=$USER/gitlab-ci-runner-mono .
END

Build it

make

Launch the runner

docker run --name gitlab-ci-runner-mono -d \
  -e "CI_SERVER_URL=http://$CI_HOST:11080" -e "REGISTRATION_TOKEN=$CI_TOKEN" \
  -v /opt/gitlab-ci/runners/mono:/home/gitlab_ci_runner/data \
  $USER/gitlab-ci-runner-mono

C++

Let's create another one for C++ development.

docker build -t $USER/gitlab-ci-runner-cpp - <<END
FROM sameersbn/gitlab-ci-runner:latest
MAINTAINER me@localhost

RUN apt-get update && \
    apt-get install -y build-essential libboost-all-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*
END

docker run --name gitlab-ci-runner-cpp -d \
  -e "CI_SERVER_URL=http://$CI_HOST:11080" -e "REGISTRATION_TOKEN=$CI_TOKEN" \
  -v /opt/gitlab-ci/runners/cpp:/home/gitlab_ci_runner/data \
  $USER/gitlab-ci-runner-cpp

TODO

• Launch at boot (docker start gitlab)
• nginx reverse proxy, also ssh

Summary

You now have a working GitLab instance and also a GitLab CI instance. You can remove GitLab CI at any time or add new runners. Everything is persisted in /opt/gitlab and /opt/gitlab-ci.

Troubleshooting

You can see the list of running docker processes with docker ps. If it's not there, it's not running. Use docker ps -a to see recently exited processes.

You can see the logs with docker logs gitlab.

Testing before committing

Instead of -d which daemonizes the process, use -it --rm which will remove the container after exit or Ctrl+C. Since you're persisting data locally, you can safely delete the container and launch it again.

Based on the following guides:

• https://github.com/sameersbn/docker-gitlab
• https://github.com/sameersbn/docker-gitlab-ci
• https://github.com/sameersbn/docker-gitlab-ci-runner

They are extensive and explain all options and use cases. These here are my public notes. You can set up a running GitLab and GitLab CI installation by copy and pasting the commands below.