How to access Google Cloud secrets from CI jobs

Accessing secrets securely in CI/CD pipelines is notoriously tricky. This how-to guide provides some reusable code to make the process of retrieving one or more Google Secret Manager secrets easier.

We have a reusable fragment which implements fetching secrets from GCP.

Prerequisites

To use the fragment in your CI jobs you will need to ensure that the following prerequisites are met.

• You'll need to ensure that bash, curl, and jq are installed in your jobs' image and that your job is running in a bash shell.
• The CI runner executing your job must be running in the context of a Google Cloud identity. The identity must either have the secrets.secretAccessor role granted for all secrets to be retrieved, or have the ability to impersonate a service account which does.
• You'll need to ensure that the CI jobs run in a GKE runner:

my-job:
  tags:
    - $GKE_RUNNER_TAG

Fetching secrets

The following steps will retrieve the latest version of the Google Secret Manager secret named my-secret from the Google Cloud project named my-project and export the secret value as an environment variable named MY_SECRET.

In your .gitlab-ci.yml file include the get-gcp-secrets.yml template.

include:
  - project: uis/devops/continuous-delivery/ci-templates
    file: /fragments/get-gcp-secrets.yml
    ref: <specify version here>

Next, define a job with the variables GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest.

my-job:
  stage: build
  image: ubuntu:24.04
  variables:
    GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest

Specify the $GKE_RUNNER_TAG job tag. When working on UIS DevOps projects, you'll usually need to run your CI jobs on one of our Google Kubernetes Engine hosted runners to be able to use this helper. To do that, add the $GKE_RUNNER_TAG variable to the list of tags for your job.

my-job:
  stage: build
  image: ubuntu:24.04
  variables:
    GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest
  tags:
    - $GKE_RUNNER_TAG

Add a reference to the included hidden .get-gcp-secrets job's before_script block in your job's before_script.

my-job:
  stage: build
  image: ubuntu:24.04
  variables:
    GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest
  before_script:
    # Required as this example uses the ubuntu:24.04 base image.
    - apt-get update && apt-get install -y curl jq
    - !reference [.get-gcp-secrets, before_script]

The before_script referenced in .get-gcp-secrets will retrieve the value of the secret in the projects/my-project/secrets/my-secret/versions/latest secret object and export it as an environment variable named MY_SECRET. This variable can then be referenced in the script block of your job in the usual way.

my-job:
  stage: build
  image: ubuntu:24.04
  variables:
    GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest
  before_script:
    # Required as this example uses the ubuntu:24.04 base image.
    - apt-get update && apt-get install -y curl jq
    - !reference [.get-gcp-secrets, before_script]
  script: do-something --secret $MY_SECRET

Specifying secrets

The before_script looks for all variables available to the job which begin with GET_GCP_SECRET_. It expects that each of these variables contains the full id of the secret version to be retrieved e.g. projects/my-project/secrets/my-secret/versions/latest

Each secret version is retrieved and exported to an environment variable named with the variable name minus the GET_GCP_SECRET_ prefix. For example, if you use the variable name GET_GCP_SECRET_API_TOKEN the secret value will be exported to the API_TOKEN environment variable.

Providing your runner has the appropriate permissions, it is possible to define multiple secrets in a single job. For example:

my-job:
  stage: build
  image: ubuntu:24.04
  variables:
    GET_GCP_SECRET_SUPER_SECRET_TOKEN: projects/some-project-1/secrets/some-secret-1/versions/latest
    GET_GCP_SECRET_DATABASE_PASSWORD: projects/some-project-2/secrets/some-secret-2/versions/56
  before_script:
    # Required as this example uses the ubuntu:24.04 base image.
    - apt-get update && apt-get install -y curl jq
    - !reference [.get-gcp-secrets, before_script]
  script: |
    do-something --secret $SUPER_SECRET_TOKEN
    do-something-else --database-password $DATABASE_PASSWORD

Google Service Account impersonation

In some situations, the identity of your CI runner may not have permission to access the required secrets itself but it may be able to impersonate a service account which does. To specify the email address of a service account to be impersonated you can use the GOOGLE_IMPERSONATE_SERVICE_ACCOUNT variable.

my-job:
  stage: build
  image: ubuntu:24.04
  variables:
    GOOGLE_IMPERSONATE_SERVICE_ACCOUNT: my-service-account@project-a.iam.gserviceaccount.com
    GET_GCP_SECRET_SUPER_SECRET_TOKEN: projects/some-project-1/secrets/some-secret-1/versions/latest
    GET_GCP_SECRET_DATABASE_PASSWORD: projects/some-project-2/secrets/some-secret-2/versions/56
  before_script:
    # Required as this example uses the ubuntu:24.04 base image.
    - apt-get update && apt-get install -y curl jq
    - !reference [.get-gcp-secrets, before_script]
  script: |
    do-something --secret $SUPER_SECRET_TOKEN
    do-something-else --database-password $DATABASE_PASSWORD

Summary

In this guide you learned how to fetch secrets stored in Google Cloud into a CI job.