GCP Projects and Folders¶
This page describes how we make use of Google Cloud projects and folders to structure our deployments.
Centrally managed infrastructure¶
We use the gcp-infra project (University members only) to configure and provision Google Cloud for the wider University. This repository contains terraform configuration for the scheme described below.
We use a series of Google Folders within Google Cloud to reflect a rough institution/group hierarchy similar to the hierarchy which has developed on the Developer Hub.
At the bottom of this hierarchy are products. At the Google Cloud management level we care only about products and each product is given its own product folder.
For example, the "Information Services" folder contains sub-folders for groups within the institution and each group has a sub-folder for each product:
Each product is associated with a finance cost code and a team name. These are added to projects within the product folder as a label. These labels are intended to be used to automate finance reporting.
In addition to the labels, products are allocated a budget and a budget alert is configured to notify us when a product is likely to go over budget for a month.
Individual user accounts can be designated as "product owners". These accounts will be given "Owner" rights on projects under the product folder. More information can be found on the permissions and roles page.
The product folder and meta project¶
When a new product is provisioned in Google Cloud, we take the budget, finance code and list of owners and add them to the gcp-infra project. The terraform will create a product folder and a meta project for the product.
The product folder contains all Google Cloud projects which relate to a given product. We generally use completely separate Google Cloud projects for each of our production, test and development environments. Occasionally we'll provision additional projects within the product folder to develop larger features which are likely to require multiple iterations before they can be merged into the product as a whole.
The meta project is a project within the product folder which contains resources which are specific to the product but which are shared by all environments.
For example, here are the projects within the "Applicant Portal" product folder:
You can see how the meta project is tagged with the team name and cost-centre code. A running total of costs can also be seen.
Examples of resources which belong within the meta project are:
- A Cloud DNS zone for the product. Usually of the form
[product].gcp.uis.cam.ac.uk. Projects within the product folder will generally contain environment-specific zones such as
- Product-wide secrets such as GitLab access tokens or credentials for external services which are shared by all environments.
- A Cloud Storage bucket which hosts the terraform state.
The personal accounts of each product owner are added to the product folder with the "owner" role. This allows all product owners to examine and modify configuration within the Google Cloud console. More information can be found on the permissions and roles page.
In addition a service account is added to the meta project which is also given owner rights over the product folder and the right to associate projects within the product folder with the University's Billing Account.
This service account is called the product admin service account.
Credentials for the product admin service account are added to a Google Secret Manager secret within the meta project. Anyone who can read this secret can, in effect, act as the product admin.
The gcp-infra configuration will create a generic "configuration" Cloud Storage bucket in the meta project which can be used for terraform state. It will place two objects in that bucket: a human-readable HTML configuration "cheat sheet" and a machine-readable JSON document with the same content. Both are only readable by Product Owners.
An example of the configuration cheat sheet is as below:
The cheat sheet contains the Product Folder id, the meta project id, the billing account id, the configuration bucket name, the DNS zone name and the product admin service account name along with the location of the secret which contains credentials for the service account.
It is intended that the information on the cheat sheet should be enough for someone to bring their own deployment tooling to GCP. In DevOps, we have a boilerplate deployment. The cheat sheet links to the boilerplate and provides values which can be directly pasted into the boilerplate parameters.
Our boilerplate also makes use of the generated JSON document to automatically fetch many of these parameters meaning a) we can update them and have deployments use the new values automatically and b) we don't need to copy-paste them into the deployment. This is an example of the Don't Repeat Yourself (DRY) principle.
- We manage Google Cloud as a collection of products and each product has a dedicated product folder and meta project created for it in Google Cloud.
- Within the meta project we create:
- A product admin service account which has rights to create projects within the product folder and to associate them with the University billing account.
- Credentials for the product admin service account which are stored in a Google Secret.
- A general configuration Cloud Storage bucket which can be used to store product-wide configuration.
- A cheat sheet and configuration JSON document providing human-readable and machine-readable information on the Google Cloud resources provided for the product.
- A managed DNS zone of the form
[product].gcp.uis.cam.ac.ukwhich is pre-wired in to the University DNS.
- Each environment, such as "production", "staging" and "development" should be a separate Google Cloud project created within the product folder.
- The individual accounts for each product owner are added as owners on the product folder and so they can read the product admin service account credentials or create new ones.