Serverless Web Applications¶

This page documents how we use the Cloud Run service to deploy web applications in a serverless manner.

How can there be no server?¶

Serverless is a term of art which refers to a method of executing code in the Cloud without having to directly manage the servers which the code runs on. Clearly, there are servers but their management is delegated to the Cloud provider who specifies a common interface which code running on the platform should support.

Even before the term was coined, "serverless" computing had been around since at least the 1980s. Nowadays, rather that having a broad, complex set of supported libraries and runtimes, we define a thin contract between the hosting platform and a container which packages the application code.

Where possible we architect our applications to follow the serverless computing contract. This ensures that we can easily port our applications between any hosting platform which supports the Knative specification. The serverless computing contract is also a good target to aim for even when deploying applications in kubernetes clusters as it provides a clean, orthogonal and well-specified interface between the application and the hosting environment.

Cloud Run¶

Cloud Run is a managed serverless platform with some extensions which make things convenient for our needs. Aside from simply hosting a containerised application on the web with auto-scaling, Cloud Run also supports:

Automatic exposure of a Cloud SQL instance to the container via a Unix socket.
Associating the workload with a Cloud IAM identity allowing the use of default application credentials within the container.
Wrapping the service in a Cloud Load Balancer allowing for custom TLS certificates, HTTP to HTTPS redirect and content caching. See traffic ingress for more details.

Our boilerplate¶

We have a copier template for deploying a web application within our boilerplate (Developer Hub users only). This makes use of our standard Cloud Run application terraform module.

The boilerplate follows a GitOps model: the exact version of the application container image to deploy is specified in the terraform configuration itself, per workspace, in the local.container_images map. The terraform is run by GitLab CI and "releasing" involves merging a change to master which changes the container image tag. We use Renovate to keep the deployed image tag fresh automatically.

Our boilerplate splits web-application configuration into two parts: the service and the application settings. Load balancer resources are not configured in the boilerplate directly; they are managed by the Cloud Run terraform module itself when the enable_load_balancer variable is set.

The Cloud Run module creates a dedicated service account identity for the application. This service account will be used by Google API libraries which use application-default credentials.

This service account is granted the following permissions:

connecting to the SQL instance (the roles/cloudsql.client role, granted by the module when grant_sql_client_role_to_webapp_sa is true),
reading the sensitive settings secret, and
reading the non-sensitive settings storage object.

If a load balancer is in use, a DNS A record is created for the application within the project's DNS zone pointing at the load balancer's static IP. A Google-managed TLS certificate is provisioned for that domain and for any custom domain specified in local.webapp_custom_dns_name, but records for custom domains are not created. Usually this local is used to host the "friendly" .cam.ac.uk domain for the service and records under .cam.ac.uk must be created by other means.

Service configuration¶

The Cloud Run service itself is configured in webapp.tf. This file configures:

a Google Secret Manager secret to hold sensitive configuration,
a Google Cloud Storage object to hold non-sensitive configuration,
a database user and password within the SQL database instance,
the Cloud Run service itself, via the module "webapp" block, and
a DNS record for the application if it is behind a load balancer.

The module's containers variable configures a single container named webapp whose image is resolved from Artifact Registry using the tag pinned in local.container_images. If the application is behind a load balancer, the module by default configures the Cloud Run service to be "internal and load balancer only" so it is not directly accessible from the public Internet.

Scaling is configured via the module's scaling variable. We start with a default of scaling.max_instance_count = 5 which is suitable for a lightly-used web application. The minimum instance count defaults to zero which allows the web-application to use no web hosting resources when it is not being used. Generally we would increase max_instance_count as the application gets more use and would set min_instance_count above zero if we are seeing latency spikes due to application startup delays.

Important

Even if the web application uses database connection pooling there is a minimum of one connection per server process. As such one needs to make sure that scaling.max_instance_count multiplied by the number of server processes in the container is less than the maximum connection count of the SQL instance. For our webapp boilerplate, there are usually four server processes per container instance, and the default db-f1-micro PostgreSQL instance allows 25 connections.

Database connectivity¶

The Cloud SQL instance is exposed to the container via the module's mount_cloudsql_instance variable, which mounts the instance as a Unix socket at /cloudsql/<instance connection name> inside the container. The database connection in the application settings uses this socket path as its host.

Optionally, the boilerplate can be generated with a database which has no public IP address (the database_private_network_only copier question). In this configuration the database is only reachable over the project's VPC network and the Cloud Run service is configured with direct VPC egress (the module's vpc_access variable, with egress limited to private ranges) so that it can reach the database's private IP.

The module also supports an optional "pre-deploy" Cloud Run job (the enable_pre_deploy_job variable) which runs a configurable command, such as database migrations, before new revisions of the service are deployed. The boilerplate does not enable this by default; see how to run database migrations when deploying for details.

Application configuration¶

To avoid secrets appearing in the environment, we have re-architected our applications to load some configuration at runtime. For our Django projects, we make use of the externalsettings Python module and code in our settings modules which load settings from YAML-formatted documents. These documents are located at a set of comma-separated URLs passed in the EXTRA_SETTINGS_URLS environment variable. The URLs can use use any schemes supported by our geddit library.

Our boilerplate passes two URLs: a gs://... URL pointing to non-sensitive settings stored in a Cloud Storage object and a sm://... URL pointing to sensitive settings stored in a Secret Manager secret.

Note that the Cloud Storage object is non-public; it can only be read by the web application's service account. Despite this, it is not suitable for storing sensitive values since they will be visible to anyone browsing the bucket in the Google Cloud console.

When would we ever use the Cloud Storage object?

Secret manager secrets can support a maximum of 64KiB of content. For most of our applications the sensitive and non-sensitive configuration fits well within this limit and we put all settings within the secret for convenience. The Cloud Storage object is there to provide an "overflow" for non-sensitive values if we breach the 64KiB limit.

The settings themselves are encoded in a YAML document specified in webapp_settings.tf. We use terraform's yamlencode function to let us interpolate values without worrying about character escaping problems. Common secrets such as database credentials and Django secret keys are managed entirely by terraform using the random_password resource.

Note

Cloud Run itself supports exposing Secret Manager secrets directly as environment variables or mounted files, and the module's containers variable supports this via each environment variable's value_source key. Our standard approach nevertheless remains loading settings via EXTRA_SETTINGS_URLS as described above.

Third-party applications¶

When deploying third-party applications it is usually non-trivial to modify them to load configuration from Secret Manager secrets. In this case we make use of a tool called berglas. This tool wraps the third-party applications and detects environment variables which contain sm://... formatted URLs. These URLs are fetched and then, depending on their format, the content is used to either replace the environment variable or is written to a file on disk.

The UIS Technical Design Authority have an example on their site of how to use berglas with a third-party application.

Cloud Load Balancer¶

In our boilerplate, if local.webapp_use_cloud_load_balancer is true (the default for new deployments), the application will be hosted behind a Cloud Load Balancer.

Using a Cloud Load balancer has the following advantages:

We can have a static ingress IP which is occasionally useful if we need to have long-lived DNS records or if it is non-trivial to have dynamic records. (For example, the IP register database only refreshes the live configuration once per hour.)
We can make use of Cloud Armor [sic] rules to provide dynamic protection for the application.
Using Cloud CDN allows us to cache application static assets in Google's Content delivery network.
We can bring our own TLS certificates if we cannot make use of Google's auto-provisioning or there is a requirement to support EV/OV certificates.

Future work

While we don't make use of the feature yet, Cloud Load Balancer allows us to weight incoming traffic and direct it to multiple backends which aids with smoothly moving load between services when using Blue-green deployment strategies.

The load balancer resources are created by the Cloud Run terraform module itself when enable_load_balancer is true; the module makes use of Google's serverless_negs terraform module internally. The boilerplate creates a DNS record for the application if load balancing is enabled. See traffic ingress for a fuller discussion of ingress options.

Monitoring and alerting¶

In the production workspace, the boilerplate enables the Cloud Run module's built-in monitoring (the enable_monitoring variable) which configures an uptime check against the application's /healthy endpoint and an SSL certificate expiry alert. In addition, the boilerplate's monitoring_default.tf deploys the ucam-minimal-gcp-monitoring module to the production workspace, which provides a recommended baseline of project-level alert policies including a Cloud Load Balancer 5XX response alert. Non-production workspaces have monitoring disabled by default as alert policies and related resources incur costs.

Multiple web-applications¶

Our boilerplate assumes there is a single web application named "webapp". For some products this will be fine. For others we will need multiple applications. Currently we support multiple applications by copying and renaming the various webapp*.tf files and duplicating the local.webapp_... settings.

Example

An example of this can be see in the identity platform infrastructure (DevOps only) where two applications are configured: card and photo.

For the moment products with multiple web-applications are rare and the overhead associated with manual copy-and-paste is manageable. In future we'd like to provide a cleaner solution for this, possibly by means of a custom terraform module.

Summary¶

In summary,

We use Cloud Run to host our web applications where possible.
Our standard boilerplate contains example terraform configuration to:
- create the Cloud Run service via our standard Cloud Run terraform module,
- place application configuration in a Secret Manager secret,
- connect the application to a SQL database via a mounted Unix socket,
- place it behind a Cloud Load Balancer, and
- provision TLS certificates.
Deployment follows a GitOps model: the container image tag is pinned in the terraform configuration and updated via merge requests, with Renovate keeping it fresh.
We use a serverless platform to allow for "scale to zero" workloads where we can tune the number of active instances, and thus the cost, automatically with demand.
Third-party applications who cannot load their configuration directly from a Secret Manager secret are wrapped with the berglas tool.
Baseline monitoring and alerting is enabled in the production workspace.
Creating multiple web-applications within a single product is currently a process of copy and pasting configuration.