Skip to content

Container image cleanup policies

Our CI pipelines and build jobs push a lot of container images to both GitLab container registry and Google Cloud Artifact Registry. Cleanup policies delete old images from these registries automatically. This page explains the policies we apply, what each one deletes and keeps, and how to change it.

Why we clean up images

Projects built from our boilerplates push a new image every time CI runs, on every branch. Renovate merge requests add to this churn, so a busy product can push dozens of images a week that are never deployed again.

Before we had cleanup policies these images were kept forever. In Artifact Registry alone they added up to several terabytes across all products, most of it more than 90 days old. That storage costs money, and the clutter makes tagged releases harder to find.

Artifact Registry

The GCP product factory applies a cleanup policy to each product's Docker repositories in Artifact Registry. Every product gets a default Docker repository in its meta project, and optionally a public one. For background on how products and their meta projects are created, read service infrastructure.

What the policy does

The policy has three rules. Artifact Registry evaluates them in the background, not at push time, so an image can stay for a while after it becomes eligible for deletion.

The first rule deletes any image version older than 180 days, whether it is tagged or not.

The second rule keeps any version with a tag that looks like a release. By default that means semantic version tags such as 1.2.3 or v1.2.3, and latest.

The third rule keeps the five most recent versions of each image, however old they are. This protects a service that is still running an image which has not been rebuilt in a long time. Without it, Cloud Run could be left pointing at an image that no longer exists.

Artifact Registry always gives keep rules precedence over delete rules. An image version is only deleted if it is older than 180 days and matches neither keep rule.

How release tags are matched

Artifact Registry matches tags by prefix and has no regular expression support. We approximate semantic versions with the prefixes 0. to 99., plus v and latest. The trailing dot keeps commit SHAs out, because a SHA never contains a dot. A bare 1 prefix would match a large share of SHA tags and keep them forever.

Warning

The default prefixes only cover major versions 0 to 99. A release tagged 100.0.0 is not protected and becomes eligible for deletion after 180 days, although v100.0.0 is still kept by the v prefix. If you expect your product to release a major version above 99, add the extra prefixes to keep_tag_prefixes in your product's policy, as described in changing the policy for a product.

Any other tag starting with v or latest is kept too, for example latest-dev. That mistake keeps images that could have gone, which is the safer way to be wrong.

Changing the policy for a product

Each product can tune the policy through the docker_repository_cleanup_policy variable in its .tfvars file in the gcp-product-factory. The same settings apply to both the default and public repositories. For example, a product that needs a longer history could keep images for a year.

docker_repository_cleanup_policy = {
  delete_older_than_days = 365
}

The variable also lets a product change how many recent versions are kept, replace the list of tag prefixes, switch the policy to dry run, or disable it entirely. Setting keep_tag_prefixes replaces the default list rather than adding to it, so include the semantic version prefixes if you still want them. The variable description lists every field and its default.

In dry run, Artifact Registry writes what it would delete to the project's audit logs and deletes nothing. Use it to check the effect of a change before you enforce it.

GitLab container registry

We are working on a cleanup policy for GitLab container registry. We will announce it once it has been through the Tech Lead Forum.

Summary

Product Docker repositories in Artifact Registry delete image versions older than 180 days by default. Versions with a release tag and the five most recent versions of each image are always kept. Products can tune or disable the policy through docker_repository_cleanup_policy in the product factory. A policy for GitLab container registry is in progress.

See also