Skip to content

Deployments

This section of the guidebook discusses how we deploy applications to Google Cloud. It does not discuss legacy deployments.

It is intended as a guide for those new to the division or those interested in how we do things generally.

Not all deployments will use all of the techniques covered in this section but all the techniques here are embodied in at least one live service. Where possible, links to techniques being used in a live service are provided. Some of those links may be restricted to current DevOps members or members of the University.

Quick start

The following high-level steps describe how to deploy a standard product to Google Cloud. This consists of the core Google Cloud resources (Folders, Projects, DNS Records etc.), the webapp container image, and the application Terraform config.

  1. Create an issue against the gcp-product-factory project requesting the deployment of core product infrastructure resources to GCP for the new product. The DevOps Cloud Team are responsible for actioning these requests.

    • The following example issue template can be used and will help speed up this process. If you're unsure of any of the values in the example you should discuss further with a member of the Cloud Team.

      Example Issue

      This issue is to request that a new GCP product be deployed with the following details:

      • Product Name: A human-friendly name for the product
      • Product Slug: A short DNS friendly name for product
      • Budget: *Budget available for the product
      • Cost Centre: University Finance System cost centre for the product
      • Team Name: Name of the team responsible for this product

  2. Create a new webapp repository using the webapp-boilerplate copier template and push the main branch to a new Gitlab project.

    • Additional information on the Webapp Developer Environment can be found here.
    • Pushing the main branch to Gitlab will ensure that the default AutoDevops CI/CD pipeline runs which builds the container image and stores it in the Gitlab container registry. This is required by the gcp-deploy-boilerplate.

  3. Create a new infrastructure repository using the gcp-deploy-boilerplate copier template.

    • This template generates the Terraform config required to deploy the webapp and related
    • infrastructure to Google Cloud.
    • Additional information on our Terraform deployments can be found here.

The 10,000 foot view

Wherever possible, we attempt to automate our deployments either through GitLab CI pipelines or terraform configurations.

Warning

Our deployments vary greatly in how well they meet this ideal. In particular, some of our older services have rough edges in their deployments. Usually these are documented in dedicated service documentation.

To help us meet this ideal, we strive that our deployments have the following characteristics:

  • They use infrastructure as code to provide a machine readable and runnable description of our deployments.
  • They are generic. We try to avoid as many hard-coded globally unique identifiers as possible by generating identifiers dynamically. This makes it easier to spin up dedicated testing or development environments.
  • They avoid clickops when possible and carefully document it when not.
  • They are consistent. We are a small team relative to the number of services we run. Keeping our deployments consistent with each other where possible makes it easier for Engineers to move between them.

Our standard environments

By keeping our deployments generic we can easily stand up parallel deployments which we term "environments". All products should have a production and staging environment. Ideally these should be as similar to each other as technology and budgets allow.

We also find it is useful to keep a dedicated development environment provisioned for testing changes. This is only an optimisation; it should be a relatively straightforward and automated process to tear down the dev environment and re-build it from scratch.

Our standard deployment

We have a standard deployment boilerplate (University members only) based on terraform. Pages in this section of the guidebook will often refer to specific parts of the boilerplate to provide a concrete example of a technique.

Per-product terraform configurations are usually found in a project named "infrastructure" or, historically, "deploy" within the project-specific group in GitLab.

Where feasible we try to keep these visible to signed in University members to encourage cross-team collaboration and learning.

Deploy on day one

Through hard won experience we have learned that spending several weeks developing a product and then trying to deploy it means that it is all to easy to design in architectures which make it hard to deploy, maintain or upgrade the product.

To combat this we have a "deploy on day one" policy. When we start work on a new product we:

As such we have a "production" environment on day one and are strongly motivated to make sure that adding features allows progressive enhancement of the existing deployment. We are also strongly motivated to build in audience limiting features from the start if it is necessary to restrict access to the service as it is being developed.