Skip to content

Cloud Platform Deployment

This section explains how we deploy and manage our Google Cloud Platform infrastructure. It focusses on the core cloud components and services which lay the groundwork for our application deployments.

Deployment overview

Our cloud platform deployment is comprised of multiple stages. Each stage is a Terraform configuration which is responsible for a specific section of the deployment. The stages/repositories are as follows.

Note

The following Terraform configurations are designed to be applied in stage order. For example, the gcp-product-factory configuration depends on resources which are created by the previous stage's gcp-infra configuration. This can be very important when adding new products to the configurations etc.

Stage 1 - Google Cloud Infrastructure for Information Services

The Google Cloud Infrastructure for Information Services (gcp-infra) Terraform repository is the initial "bootstrap" stage of our cloud platform deployment. This config is responsible for high-level resources, such as:

  • Top-level and per-institution Google folders
  • High-level IAM bindings (admin users, billing account admins etc.)
  • DevOps-wide service accounts and IAM bindings

Stage 2 - Google Cloud Product Factory

The Google Cloud Product Factory (gcp-product-factory) Terraform repository is responsible for deploying core infrastructure for each of our products. The types of resources it is responsible for are as follows.

  • The product's "meta" project.
  • Per-environment Google projects (e.g. development, staging, production).
  • Google API service enablement for all projects.
  • A product DNS zone.
  • Google DNS domain verification records. (These are not required for new products.)
  • Product-specific service accounts and IAM bindings.
  • A product-wide artifact registry container repository.
  • Logging sinks and monitoring configuration.

Stage 3 - GitLab Project Factory

The GitLab Project Factory (gitlab-project-factory) Terraform repository is responsible for creating and managing groups and projects in our GitLab instance. This closely ties in with the product factory stage and ensures groups and repositories are created in a standardised and repeatable way. Some of the resources this code manages are as follows.

  • Per-product GitLab subgroups.
  • GitLab projects (repositories) including config such as:
    • Default branch protection.
    • Merge request required approvals.
    • A group level access token for use in other automation.
    • Container registry expiration policies.

Stage 4 - GitLab GKE CI Runner Infrastructure

The GitLab GKE CI Runner Infrastructure (gitlab-runner-infrastructure) Terraform repository is responsible for deploying and managing our Google Kubernetes Engine (GKE) hosted GitLab runners. These runners are configured on a per-product basis and run in the context of a dedicated service account to ease the burden of authentication in CI/CD pipelines.

Some of the resources this code manages are as follows:

  • GKE clusters, node pools, and Google IAM service accounts.
  • Per-product Kubernetes namespaces, Kubernetes service accounts, and Workload Identity IAM bindings.
  • Per-product GitLab runner helm chart deployments.

For more information on the configuration of these runners see here.

Further information

The rest of the pages in this section go into more detail on the specifics of our cloud platform environment. However, the how-to and tutorial sections are often a better place to start as they contain short, focussed, articles that are more relevant to day-to-day tasks.