GITLAB, Jenkins and GKE kubernetes pipeline

We were planning to migrate a project workload to K8s. I was comfortable with K8s and deployments. However, I was not pretty sure of CI/CD pipeline.
With GITLAB we’ve off the shelf connectors and runners to K8s clusters, however, GITLAB setup at our orgranization, wasn’t upgraded to support that feature.

I came across Jenkins which can do this job and started exploring and successfully set it up. Hope this helps someone setting up pipeline.

Prerequisites:

• Setup Jenkins (I used AWS EC2 instance) refer to this link below for the easy setup.

How To Install Jenkins on Ubuntu 20.04 | DigitalOcean

Install Docker (provide permissions to run it as Jenkins user), Kubectl on Ubuntu machine.

• Setup k8s cluster as per your requirement (I used GKE 1.20 1500)
• Create a project on GITLAB

Let’s Automate:

Steps:

1. Setup Jenkins
2. Setup GITLAB
3. Setup Jenkins pipeline in your code
4. Sit-back and enjoy the magic

Setup Jenkins:

1. Install plugins: Jenkins Dashboard -> Manage Jenkins -> Manage plugins.

2. Install plugins listed below:

• GitLab
• Docker pipeline
• Kubernetes plugin
• Google kubernetes plugin
• Pipeline

click “Install without restart”.

3. Manage credentials:Jenkins Dashboard -> Manage Jenkins -> Manage credentials

We need to create 5 credentials.

• GITLAB Jenkins access.

First, we must create a user that Jenkins will use to interact via the GitLab API. You should set this user as global admin or add as a member to related Group/Project. In order to report build status, developer permission required.

First open GitLab and go to Profile Settings

Then, click “Access Token” and create a Jenkins API Token.

Then, save and copy this API Token. Let’s go on with Configure the Jenkins Server.

Configure the Jenkins Server

Go to Jenkins and install Jenkins GitLab Plugin and Jenkins Git Plugin.

Go to Manage Jenkins -> Configure System and scroll down to the ‘GitLab‘ section.

Write a custom connection name, enter the GitLab server URL in the ‘GitLab host URL‘ field and click Add -> Jenkins button.

Then, fill required fields as shown below with the Jenkins Access API Token which we created in GitLab before.

Then, select this API Token credential and finally click “Test” and see the “Success” message.

Create and Configure a Jenkins Job

Create a new job on Jenkins and then click “Configure”.

Give a project name, Click “Discard old builds” checkbox and write “3” for “Max # of builds to keep“. It only keeps last 3 builds and saves disk space.

For “GitLab connection” field, select predefined connection which is done with GitLab plugin above.

For “Gitlab Repository Name” write “<group name>/<repository name>“. You can see this on GitLab projects page.

Then go to Credentials –> System –> “Add domain” and click “Add credentials” as shown below,

Then, add your GitLab username and password as shown below and click OK.

After that, select “Git” as Source Code Management system. Fill “Repository URL“, Select Credentials which is added above, set branch, select repository browser as gitlab, and write your project URL and GitLab version as shown below.

Add Jenkins server public key to GITLAB.

sudo -su jenkins
ssh-keygen
cat /home/jenkins/.ssh/id_rsa.pub

Copy the above result. and Paste it here. Refer the screenshot below.

Paste and give a nice name and save it.

Quickly test if Jenkins is able to access GITLAB. Go to Jenkins Dashboard -> Manage Jenkins -> configure system -> GITLAB section as shown below.

Enter the details and select the API token from the drop-down and hit test connection. You should see Connection successful. BINGO! One part is done.

Connecting to Kubernetes:

1: create a service account in Google cloud project

• Login to your GCP account and create a Project. In this demo, I created a project with the name “catalogue-testbed”.
• Next, navigate to the “IAM & admin -> Service accounts” page and create a new service account. For this demo, I have created a service account with the name “jenkins”. Assign it the “Kubernetes Engine Admin” role.
• Next, create a new JSON key for the service account. Download and save this key, as it will be needed by Jenkins.

Create a credential in jenkins and Upload the file as below

quick check on the creds needed

Let’s ensure the above credentials are present before we execute any pipeline.

Connect to the GKE cluster created on your local machine. Upload the config file as a secret file, as you can see k8s config file in the above screenshot. I use ubuntu, so it will be in .kube/config

Let’s run a pipeline.

Create a simple express project and expose 8080. Below will create a docker for the same.

FROM bitnami/node:9 as builder
ENV NODE_ENV="production"

# Copy app's source code to the /app directory
COPY . /app

# The application's directory will be the working directory
WORKDIR /app

# Install Node.js dependencies defined in '/app/packages.json'
RUN npm install

FROM bitnami/node:9-prod
ENV NODE_ENV="production"
COPY --from=builder /app /app
WORKDIR /app
ENV PORT 5000
EXPOSE 5000

# Start the application
CMD ["npm", "start"]

create a deployment.yaml file in your project

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
  labels:
    app: hello
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: hello
      
  template:
    metadata:
      labels:
        app: hello
        tier: hello
    spec:
      containers:
        - name: hello
          image: <<Your DockerHub Username>>/hello:latest
          imagePullPolicy: Always
          ports:
          - containerPort: 5000
            name: hello


---
apiVersion: v1
kind: Service
metadata:
  name: hello
  labels:
    app: hello
spec:
  ports:
    - port: 5000
      targetPort: 5000
  selector:
    app: hello
    tier: hello
  type: LoadBalancer

Now let’s create a jenkins file.

pipeline {
    agent any
    environment {
        PROJECT_ID = '<<Your GCP Project ID>>'
        CLUSTER_NAME = '<<Your GKE Cluster Name>>'
        LOCATION = '<<Your GKE Cluster Location>>'
        CREDENTIALS_ID = 'multi-k8s'
    }
    stages {
        stage("Checkout code") {
            steps {
                checkout scm
            }
        }
        stage("Build image") {
            steps {
                script {
                    myapp = docker.build("<<Your DockerHub username>>/hello:${env.BUILD_ID}")
                }
            }
        }
        stage("Push image") {
            steps {
                script {
                    docker.withRegistry('https://registry.hub.docker.com', 'dockerID') {
                            myapp.push("latest")
                            myapp.push("${env.BUILD_ID}")
                    }
                }
            }
        }        
        stage('Deploy to GKE') {
            steps{
                sh "sed -i 's/hello:latest/hello:${env.BUILD_ID}/g' deployment.yaml"
                step([$class: 'KubernetesEngineBuilder', projectId: env.PROJECT_ID, clusterName: env.CLUSTER_NAME, location: env.LOCATION, manifestPattern: 'deployment.yaml', credentialsId: env.CREDENTIALS_ID, verifyDeployments: true])
            }
        }
    }    
}

Create a pipeline on jenkins:

in the pipeline section

script path refer the name of the jenkinsfile created.

Now, whenever the code is pushed, jenkins pipeline would be triggered.

All Happy and green :-)

If you’ve found this useful. Please leave a clap and if any questions, please leave them in the comments section, I’ll answer them as and when I get time.

Happy coding! Until next article BBYE!