Unified DevOps Platform

The Unified DevOps Platform is an approach to software development which will be used for all new project development or architecturally significant changes made within the DevOps Function.

Platform composition

The Platform isn’t simply a collection of technologies that DevOps will make use of, but a set of templates, tools and boilerplates that enable members of DevOps to operate efficiently, providing sensible defaults as well as an ongoing upgrade path. For example, the solution provided for Web Application Development is the DevOps Boilerplate Web Application, which includes the latest version of Django, a frontend module using TypeScript and React, and CI / CD configuration for running the application locally using Docker Compose and building a deployable Docker image. Crucially, the Boilerplate Web Application is managed via Copier, which allows any improvements that are made to the Boilerplate to be applied to applications which have already been created using the Boilerplate. Additionally, clear documentation regarding use of the Boilerplate Web Application is included within this guidebook.

Technology selection

The technologies included in the Platform were selected based on:

• Being used in multiple different applications currently operated by DevOps, and proven to be successfully deployed and operated at the scale expected by the University.
• Where there is a high level of existing expertise in DevOps with the technology.
• Where high-quality documentation or training material already exists for the technology.
• Being MIT licensed or where the University has an existing commercial agreement with the provider (e.g. Google Cloud Platform).
• Being actively maintained by an open-source community or commercial organisation.

Defining the Platform in this way does mean that DevOps is intentionally deprecating the use of other technologies. This allows the Function to invest in solutions that are a part of the Platform and enhances the ability for shared learning and developer mobility across the Function.

Platform adoption

The Platform has been designed so that it can be incrementally adopted, as well as used for greenfield development. For example, existing Java-based web applications can, and currently do, make use of the Continuous Integration and Continuous Deployment solutions from the Platform, despite not making use of the Web App solution. It is not expected that all applications maintained by DevOps will move to the Platform at pace, but instead an approach of incremental adoption will be taken - introducing elements of the platform as architecturally significant changes are made to existing applications.

Use of other technologies

There is no desire to stifle innovation, or force teams in DevOps to use unsuitable technologies to solve unique problems. Team members in DevOps are encouraged to try new technologies and identify where a non-platform technology should be used to meet specific functional or non-functional requirements. In these cases the DevOps Technical Lead Forum will review and agree on the approach to take. Technologies or approaches that have proven value and ability for re-use may be added to the Platform with agreement from the Technical Lead Forum.

Support within the DevOps Function

Line managers across DevOps are actively supporting those with less knowledge of Platform technologies by enabling team members to work closely with those who have more familiarity with the Platform as well as making time for specific personal development. Cross-DevOps support is being provided in the form of a dedicated Pluralsight channel which focuses on Platform technologies, fortnightly Show-and-Tell sessions, and ongoing improvements to this guidebook's "How To" guides.

Benefits of the Unified DevOps Platform

The introduction of the Unified DevOps Platform is intended to both improve the efficiency of the Function, as well as enhance the culture and working environment.

Efficiency improvements:

• The solutions provided under the Unified DevOps Platform for multi-user systems (Web Apps and Websites) will allow compliance with the Systems Management Policy and related technical standards by default, avoiding the need for security mechanisms and protections to be built into systems individually.
• The Platform includes a solution for deploying web applications into a production-ready environment in Google Cloud Platform, including all the necessary SecOps tools with automated scaling, budget protection, monitoring and alerting provided by default, again avoiding this tooling being introduced on a project-by-project basis.
• The solutions provided under the Unified DevOps Platform are linked to template responses to the TDA Non Functional Requirements (NFRs) - allowing teams to easily demonstrate how the NFRs have been met when using the Platform.
• All elements of the Platform include an automated upgrade-path enabled by Renovate Bot, allowing improvements made at the boilerplate level to be rolled out to all applications quickly.

Working environment improvements:

• By ensuring that all members of the Function have some level of expertise with the Platform, DevOps enhances its developer mobility across the Function, increasing the potential for shared learning and cross-team collaboration.
• DevOps can be very clear on the expected tech stack used when hiring, allowing the Function to better target candidates with skills in technologies that are a part of the Platform.
• When outsourcing work, DevOps can require that agency or external staff use the Platform directly, or at least use Platform technologies, to ensure that teams within DevOps are not asked to maintain code which has been outsourced for development and delivered in an unknown technology stack.

Platform components

AREA	SEGMENTS	SOLUTIONS	PLATFORM TECHNOLOGY COMPOSITION	LINKS
CODE VERSION CONTROL		GitLab Project Factory GitLab Runner Infrastructure	GitLab	🦊 Gitlab Project Factory 📖 Bootstrap an Application
WEB APPS	BACK-END DEVELOPMENT FRONT-END DEVELOPMENT API DEVELOPMENT	Boilerplate Web Application	Docker Docker Compose Python Pydantic Pytest Django Django Rest Framework Poe Poetry Tox Typescript React MaterialUI DevOps CI Templates Precommit Flake8 Black isort prettier eslint	🦊 WebApp Boilerplate 📖 Bootstrap Django App
WEBSITES		Cambridge Web Platform	Drupal Pantheon MariaDB Twig Tailwind CSS Typescript RollUp.JS DDEV Storybook Playwright Behat Precommit eslint prettier Google Cloud DNS Hostmaster / On Prem DNS Google Cloud Storage	🦊 uis/devops/webcms
CONTINUOUS INTEGRATION	SECURITY TESTING BUILD	DevOps CI Templates Base Docker Images	GitLab Ultimate SAST GitLab Ultimate DAST GitLab Ultimate Dependency Scanning GitLab Ultimate Vulnerability Scanning GitLab Ultimate Container Scanning GitLab Ultimate Secret Detection Renovatebot	🦊 CI Templates 📖 Add Common CI Jobs
	AUTOMATED APPLICATION TESTING		Pytest Browserstack Selenium Cypress	🦊 WebApp Boilerplate 📖 Bootstrap Django App
	RELEASE		Semantic versioning release-it	🦊 CI Templates 📖 GitLab release automation
	CI RUNNER PLATFORM		GitLab	🦊 uis/devops
	PACKAGE & CONTAINER IMAGE PUBLICATION		PyPi	🦊 CI Templates 📖 How to publish a Python package
CONTINUOUS DEPLOYMENT		Google Cloud Product Factory	Terraform Google Cloud	🦊 GCP Product Factory 📖 Create Google Cloud Projects
		Boilerplate Google Cloud Deployment	Terraform tflint Trivy Google Cloud Google Cloud Artifact Registry Google Cloud Load Balancer Google Cloud DNS Google Cloud issued TLS Certificates Google Cloud Storage for Blob data storage Google Cloud Observability for Logs, Trace & Metrics Google Cloud SQL for PostgreSQL	🦊 GCP Deploy Boilerplate 📖 App Deployment
API PUBLICATION		API Gateway	Apigee	🛡️ API Gateway 🔌 https://developer.api.apps.cam.ac.uk/your-api-here
DOCUMENTATION		MKDocs	MKDocs
CRON JOBS SMALL FUNCTIONS	EVENTS WEBHOOKS	ucam-faas	Python Terraform Google Cloud Google Cloud Cloud Run Google Cloud Scheduler Google Cloud Pub/Sub Precommit Flake8 Black isort	🦊 ucam-faas
EMAIL SENDING	EMAIL SENDING	Sendgrid	Sendgrid	📖 SendGrid
	EMAIL TESTING	Mailtrap	Mailtrap	📖 MailTrap
COMMAND LINE SCRIPTS		Click	Python Precommit Flake8 Black isort