Data Backup Service¶
This page gives an overview of the Data Backup Service, describing its current status, where and
how it's developed and deployed, and who is responsible for maintaining it.
Service Description¶
Data Backup Service performs data exports of products hosted in GCP. It currently supports the exporting of data from Cloud SQL databases to a GCP bucket.
Service Status¶
The Data Backup Service is currently beta.
The service is currently running in a live-like state and runs in parallel with the existing Scheduled Database Exports.
Once the service has been fully tested and is ready for production, the existing Scheduled Database Exports will be decommissioned and the Data Backup Service will become the primary method for database exports.
Contact¶
Technical queries and support should be directed to cloud@uis.cam.ac.uk and will be picked up by a member of the team working on the service. To ensure that you receive a response, always direct requests to cloud@uis.cam.ac.uk` rather than reaching out to team members directly.
Issues discovered in the service or new feature requests should be opened as GitLab issues in the application repository.
Environments¶
The Data Backup Service is currently deployed to the following environments:
| Name | Cloud Scheduler | Backup Bucket |
|---|---|---|
| Production | sql-backup | data-backup-prod-63bdb80a |
| Development | sql-backup | data-backup-devel-37d7c4f2 |
Source code¶
The source code for the Data Backup Service is spread over the following repositories:
| Repository | Description |
|---|---|
| Application Server | The source code for the main application server |
| Infrastructure Deployment | The Terraform infrastructure code for deploying the application server to GCP |
Technologies used¶
The following gives an overview of the technologies the Data Backup Service is built on.
| Category | Language | Framework(s) |
|---|---|---|
| Server | Python 3.11 | FastAPI 0.110.3 |
| GCP deployment | Terraform 1.7 | Google Cloud Platform |
Operational documentation¶
The following gives an overview of how the Data Backup Service is deployed and maintained.
Which databases are exported¶
All databases discovered in any child GCP projects of the configured folders are exported. Exlusion patterns can be configured to exclude project databases from being exported, such as in development projects.
The configuration is controlled by the scheduled job in the Data Backup Service Terraform
infrastructure and can be found in the following file:
How and where the Data Backup Service is deployed¶
Deployment is via our standard terraform deployment CI pipeline.
Deploying a new release¶
Making a new release of the application is done via release automation. In short: merged commits are collected together into a draft "next release" Merge Request. When merged, a new release tag is pushed to the repository along with a Docker image being pushed to Google's Artefact registry.
Deployment is done by:
- Updating the deployment project's repository with any changes, including bumping the deployed web application release.
- Using the "play" buttons in the CI pipeline to deploy to production when happy. (Deployments to
staging happen automatically on merges to
main.)
Monitoring¶
Monitoring is configured as per our standard Google Cloud Run application module.
Additional documentation¶
Information about the overall architecture of the new Data Backup System can be found in:
Service Management and tech lead¶
The service owner for the Data Backup Service is Abraham Martin.
The service manager for the Data Backup Service is Adam Deacon.
The tech lead for the Data Backup Service is Roy Harrington.
The following engineers have operational experience with the Data Backup Service and are able to
respond to support requests or incidents: