Scenario:

You want to run Azure pipeline to build your project locally. But you do not want to install all the dependencies (like Maven, Golang, Helm or JDK versions etc) and tools required to build your project. You can still build your project on your laptop using build agent in Docker.

 

Solution:

You have following options-

  1. Use MS hosted build agent.
  2. Start build agent on your local device. This option will require you to install all required dependencies and tools on your local device.
  3. Start build agent in Docker locally on your device. This way you can add dependencies and tools required into docker image instead of installing them locally.

We will see how you can achieve option#3.

1. Assuming that you have installed WSL2 and Desktop-Docker on your Windows 10.

2. We will create Linux container to run the build agent. By default, Docker for Windows is configured to use Linux containers.

3. Open WSL and create folder “dockeragent”

4. Change to folder “dockeragent” and create file Dockerfile with following content.

It has Docker, GoLang and JfrogCli included in it. You can add other dependent tools as well before building the image from this dockerfile.

FROM ubuntu:18.04

ENV GO_VERSION=1.16.7
ENV DOCKER_VERSION=20.10.8

# To make it easier for build and release pipelines to run apt-get,
# configure apt to not require confirmation (assume the -y argument by default)
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes

RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    jq \
    git \
    iputils-ping \
    libcurl4 \
    libicu60 \
    libunwind8 \
    netcat \
    libssl1.0 \
  && rm -rf /var/lib/apt/lists/*

# Install Docker 
RUN curl -o docker.tgz -L https://download.docker.com/linux/static/stable/x86_64/docker-$DOCKER_VERSION.tgz \
     && tar -zxvf docker.tgz --directory /usr/local \
     && rm docker.tgz
ENV PATH=$PATH:/usr/local/docker

# Install JFrog Cli
RUN JFROGDIR=_work/_tool/_jfrog/current; \
     mkdir -p $JFROGDIR; \
     curl -fLsS https://getcli.jfrog.io | bash -s v2; \
     mv jfrog $JFROGDIR

# Install GoLang
RUN curl -o go.tgz -L https://golang.org/dl/go$GO_VERSION.linux-amd64.tar.gz \
     && tar xzvf go.tgz --directory /usr/local \
     && rm go.tgz
ENV PATH=$PATH:/usr/local/go/bin

RUN curl -LsS https://aka.ms/InstallAzureCLIDeb | bash \
  && rm -rf /var/lib/apt/lists/*

ARG TARGETARCH=amd64
ARG AGENT_VERSION=2.185.1

WORKDIR /azp
RUN if [ "$TARGETARCH" = "amd64" ]; then \
      AZP_AGENTPACKAGE_URL=https://vstsagentpackage.azureedge.net/agent/${AGENT_VERSION}/vsts-agent-linux-x64-${AGENT_VERSION}.tar.gz; \
    else \
      AZP_AGENTPACKAGE_URL=https://vstsagentpackage.azureedge.net/agent/${AGENT_VERSION}/vsts-agent-linux-${TARGETARCH}-${AGENT_VERSION}.tar.gz; \
    fi; \
    curl -LsS "$AZP_AGENTPACKAGE_URL" | tar -xz

COPY ./start.sh .
RUN chmod +x start.sh

ENTRYPOINT [ "./start.sh" ]

 

5. Create start.sh with following content

#!/bin/bash
set -e

if [ -z "$AZP_URL" ]; then
  echo 1>&2 "error: missing AZP_URL environment variable"
  exit 1
fi

if [ -z "$AZP_TOKEN_FILE" ]; then
  if [ -z "$AZP_TOKEN" ]; then
    echo 1>&2 "error: missing AZP_TOKEN environment variable"
    exit 1
  fi

  AZP_TOKEN_FILE=/azp/.token
  echo -n $AZP_TOKEN > "$AZP_TOKEN_FILE"
fi

unset AZP_TOKEN

if [ -n "$AZP_WORK" ]; then
  mkdir -p "$AZP_WORK"
fi

export AGENT_ALLOW_RUNASROOT="1"

cleanup() {
  if [ -e config.sh ]; then
    print_header "Cleanup. Removing Azure Pipelines agent..."

    # If the agent has some running jobs, the configuration removal process will fail.
    # So, give it some time to finish the job.
    while true; do
      ./config.sh remove --unattended --auth PAT --token $(cat "$AZP_TOKEN_FILE") && break

      echo "Retrying in 30 seconds..."
      sleep 30
    done
  fi
}

print_header() {
  lightcyan='\033[1;36m'
  nocolor='\033[0m'
  echo -e "${lightcyan}$1${nocolor}"
}

# Let the agent ignore the token env variables
export VSO_AGENT_IGNORE=AZP_TOKEN,AZP_TOKEN_FILE

source ./env.sh

print_header "1. Configuring Azure Pipelines agent..."

./config.sh --unattended \
  --agent "${AZP_AGENT_NAME:-$(hostname)}" \
  --url "$AZP_URL" \
  --auth PAT \
  --token $(cat "$AZP_TOKEN_FILE") \
  --pool "${AZP_POOL:-Default}" \
  --work "${AZP_WORK:-_work}" \
  --replace \
  --acceptTeeEula & wait $!

print_header "2. Running Azure Pipelines agent..."

trap 'cleanup; exit 0' EXIT
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM

# To be aware of TERM and INT signals call run.sh
# Running it with the --once flag at the end will shut down the agent after the build is executed
./run.sh "$@"

 

6. You should have both the files – Dockerfile and start.sh in dockeragent folder as shown-

 

7. Build the agent image now using the below command. I have added tag as 2.185.1 based on the build agent version used in it.

docker build -t dockeragent:2.185.1 .

 

8. Run the docker build agent container using the command.

docker run -e AZP_URL=<Azure DevOps instance> -e AZP_TOKEN=<PAT token> -e AZP_POOL=<Pool Name> -e AZP_AGENT_NAME=<Agent Name> -v  /var/run/docker.sock:/var/run/docker.sock --privileged dockeragent:2.185.1

Example:
docker run -e AZP_URL=https://dev.azure.com/ghamad -e AZP_TOKEN=4yu3sghxqwzuydr -e AZP_POOL=Self-Hosted-Agent -e AZP_AGENT_NAME=mydockeragent -v  /var/run/docker.sock:/var/run/docker.sock --privileged dockeragent:2.185.1

 

9. The Docker build agent will connect to Azure DevOps and register it under the specific agent pool.

 

Reference:

https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops