How to use secrets in Jenkins

photo by Paolo Chiabrando

Why do we need secrets?

Whatever the language we choose to build our applications there is always a need to consume or publish binary artifacts, like packages, container images, etc.:

• Maven repositories in Java
• PyPi servers for Python
• NPM repositories for Java Script
• Docker Hub registry for container images

But those repositories are public, and all of them offer a way to at least download without any credential; things get complicated when we need to authenticate against them or using their private versions in your company. Some of those credential files, as the settings.xml file can be stored on the developer’s file system $HOME/.m2, but what happens when we need to share those credentials with other team members or we need the CI/CD system’s to use it: we need a secret.

A secret in software development is any sensitive data that we want to keep private. Secrets generally refer to digital authentication credentials that grant access to services, systems and data. These are most commonly API keys, usernames and passwords, or security certificates.

As we can see [2], there is a tendency to follow the minimal resistance path and use the the version control system (VCS) to store those secrets, which is a very bad practice, so what can we do to store those secrets, to control the access to them and how to consume them in our CI/CD pipelines? We need a Secrets Manager.

A Secrets Manager helps you protect secrets needed to access your applications, services, and IT resources. The service enables you to manage and retrieve database credentials, API keys, and other secrets throughout their lifecycle. Users and applications retrieve secrets with a call to Secrets Manager APIs, eliminating the need to hardcode sensitive information in plain text.

Here [2] I listed some good candidates to play the role of Secret Manager, but on this post I want to write on how to use the good old Jenkins which is present in many CI/CD systems nowadays.

Using Jenkins as a Secret Manager

In Jenkins there are two features to handle secrets:

• Manage Credentials
• Manage files

Managing Credentials

This is the most common way to handle credentials in Jenkins and it is specially designed to handle passwords, users, or certificates as shown on the following picture:

But the Credentials feature can be also used to store different kinds of secrets:

In our case we will use Secret file and we need to select our settings.xml file to be uploaded to Jenkins and we have to provide an ID and a Description.

Now that we have the settings file stored as a secret, we can use it in our pipelines as a secret without the risks of storing it in Git repositories.

The following is an snippet on how to use it:

def img

pipeline {
...
    environment {
        // The credential ID set on Jenkins configuration
        MAVEN_GLOBAL_SETTINGS = credentials('maven-settings') 
    }

    stages {
        stage('verify') {
            when { environment name: 'gitlabActionType', value: 'MERGE' }
            stages {
                stage('build') {
                    steps {
                        container('maven') {
                            sh 'mvn -gs $MAVEN_GLOBAL_SETTINGS package -DskipTests=true'
                        }
                    }
                }
            }
        }
    }
...
}

As we can see, once the file content is stored on the MAVEN_GLOBAL_SETTINGS, environment variable, the file is injected by Jenkins in the mvn sentence as any other file.

Managing files

Jenkins has another way to manage configuration files, which is by using the Config File Management plugin. As we can see on the following picture it can be used to handle settings.xml, pip.conf or .pypi.rc files or any other like a kubeconfig file to access a Kubernetes cluster.

The following picture shows how to upload a settings.xml file by cut and paste it:

The following is the snippet on how to use it on a pipeline:

def img

pipeline {
...
    environment {
        // The credential ID set on Jenkins configuration
        MAVEN_GLOBAL_SETTINGS = credentials('maven-settings') 
    }

    stages {
        stage('verify') {
            when { environment name: 'gitlabActionType', value: 'MERGE' }
            stages {
                stage('build') {
                    steps {
                        container('maven') {
                            configFileProvider([configFile(fileId: '42d10aee-187e-4b0f-a30e-9af4e3d99f32', variable: 'MAVEN_GLOBAL_SETTINGS')]) {
                                sh 'mvn -gs $MAVEN_GLOBAL_SETTINGS package -DskipTests=true'
                            }
                        }
                    }
                }
            }
        }
    }
...
}