GitLab

We "eat our own dogfood" and use the GitLab instance on the Developers' Hub to manage our work. All team members should be part of the DevOps group on GitLab.

For particular products there may be more members added to the product-specific group.

Golden rules

  1. GitLab holds the entire state of the product development.
  2. Nothing should only be in your head.
  3. External resources such as spreadsheets, documents or websites should always be linked to in either an issue or on a wiki page.
  4. It is your responsibility to move things forward.

What do I do day-to-day?

If you are at a loose end and want to move the product forward, look at the sprint board. There are three ways to move forward:

  • Pick up an issue which is "urgent" from the backlog, assign yourself and
  • start work.
  • Pick up an issue which is "review required" and review it.
  • Pick up an issue which was assigned to you which is currently in "rework".
  • Pick up an issue which is "sprint ready" from the backlog, assign yourself and start work.

Note

When scheduling a sprint we tend to order issues in rough order of priority. In general try to pick issues which are near the top or, for exceptional reasons, have been pre-assigned to you.

You should also keep an eye on issues which are in the "needs refinement" state as you may be able to offer some insight in the discussion on the issue.

If none of these options are possible, contact the scrummaster and they’ll try to unblock you.

Note

A more detailed description of day-to-day work is contained in the common flows document.

Team responsibilities

Although the scrummaster ensures that the process in this document is followed, it is the responsibility of every team member to do their part.

Be proactive in monitoring issues and taking part in discussions.

Subscribe to notifications on the uis/devops group.

Do not wait until a refinement meeting to look at issues which "need refinement".

GitLab concepts

There are a number of GitLab concepts which can have non-obvious names like "issue", "epic", "milestone", "label", etc. This section provides an overview of the terms.

Issues

An issue is the smallest unit of work. It may form part of a larger epic. Each issue belongs to zero or one sprint which is represented by its milestone.

An issue cannot be assigned to a sprint until it is "sprint ready". The bar for being "sprint ready" is relatively low:

  • It is sufficiently well specified that at least one team member can read the discussion on the issue, determine what needs to be done to make progress and can start work doing it and that another team member knows how to determine if the work has been done.
  • It has a "size".
  • It is not blocked by any issues which are not part of the same sprint.

Merge Requests

A Merge Request represents a request to merge some code changes into the master branch. Merge requests are covered in more detail in a dedicated section of the guidebook.

Milestones

Milestones are collections of issues with a "start" and "due" date. We use milestones to document which sprint an issue is part of. Milestones are created within the uis/devops/uga group.

Until an issue is in a sprint, it is "floating". It is a common pattern to create an issue when a work item appears and "pencil it in" to the next sprint but label it as "needs refinement" or "needs sizing" as appropriate. When we come to plan the next sprint, this ensures these issues are visible and triaged appropriately as being urgent enough to schedule in the upcoming sprint, move the sprint after next or to keep "floating".

All work associated with a sprint should be added to the sprint milestone. When addressing a GitLab issue make sure to add any related Merge Requests to the milestone.

Warning

Merge Requests are not listed on the board. If you want someone to pick up and review your Merge Request in good time, make sure it has a related issue which appears on the board.

Sprint boards

A sprint board, for example https://gitlab.developers.cam.ac.uk/groups/uis/devops/uga/-/boards/630, provides an overview of the current progress of a sprint. It contains all the issues which form part of the sprint organised into columns showing where the issue is in its lifecycle.

We create a new board for each sprint and set the "scope" of the board to be the sprint's milestone. Each board will show the state of each issue in the sprint and the progress through the issue lifecycle.

Issues can be re-ordered within columns and dragged from column to column. In particular we find that it is useful to order the "sprint ready" column in rough priority order so that issues toward the top are higher priority than issues near the bottom.

The issue lifecycle

Issues are born, are refined, are done and die. The progress of an issue is called the "issue lifecycle". Diagrammatically, the issue lifecycle is represented by the following graph:

The issue lifecycle

Start

An issue needs to be created. Many things can prompt an issue creation from an ad hoc comment in a conversation to a formal specification process. Creating issues is supposed to be a lightweight, common process.

The minimum required to create an issue is a title and a description. Usually one of the "workflow::needs refinement" or "workflow::needs sizing" labels will be applied at creation time. Urgent issues are distinguished by applying the "urgent" label.

Some issues will naturally have deadlines associated with them. In such cases a deadline is added to the issue in GitLab which will make sure to email the project in advance of the deadline if the issue is not closed.

Needs refinement

An issue starts as a description and title. Sometimes this is enough in that the issue is sufficiently small that it can be tackled immediately. It is more often the case that the work required to move the issue forward is unclear.

In this case the "workflow::needs refinement" label is added to the issue.

Discussion takes place on the issue until a way forward becomes clear. It can be that the discussion results in the original issue being closed and split into smaller sub-issued.

Once the work required to move the issue forward is clearly specified either in the description or in the discussion, the issue is re-labelled as "workflow::need sizing". Anyone on the team can apply this label if they feel the issue is refined.

Needs sizing

There is a weekly opportunity for team members to help sizing issues. We size issues in two ways:

  • Estimated time taken to perform the required work and to review it. This is measured in "days" with a granularity of half a day.
  • Estimated complexity measured in dimensionless units:
  • I know how to do this
  • Someone else on the team knows how to do this
  • Someone in the world knows how to do this
  • No-one knows how to do this

When sizing an issue, each team member scores the issue according to the above and reveals the scores simultaneously.

The final "size" is the approximate mean of the product of each member’s scores.

The "size" has dimension days since it is the product of "day" and "dimensionless" but it is not a deadline.

Once sized and refined, an issue gets the "workflow::sprint ready" label.

Note

Issues with the "spike" label are timeboxed. They are issues which could expand without bound and so we set a hard maximum of time that can be spent on them before the person doing the spike reports back in the issue.

Sprint ready

Once "sprint ready", an issue is eligible for scheduling in a sprint. This is usually done by the scrummaster depending on priorities at the time.

An issue is scheduled into a sprint by adding it to the appropriate sprint milestone.

Doing

When an issue is picked up by a team member, they:

  • Assign themselves to the issue
  • Add the "workflow::doing" label

Ordinarily we do not "pre-assign" issues except where there is a clear need. Pre-assignment is an early indicator of single points of failure. Reasons for pre-assignment should be clear and it should be the exception rather than the rule.

Rasons for pre-assignment can include selecting educational issues for junior or mentee team members. In that case care should be taken when sizing to account for the time that a senior or more experienced team member will spend helping the assignee.

Review required and Needs testing

When you’ve completed a work item, add the "workflow::review required" label.

All issues must be reviewed. Even non-technical issues will benefit from another team member seeing what has been done. Review is not just used to ensure correctness, it is used to ensure that at least two people are aware of what happened.

An issue in review may be reviewed by any team member but the team member who completed the work may "@-tag" a particular person if they feel that there is someone who would be particularly familiar with the work.

The reviewer will take a look at the issue, any description of the work completed and any associated merge requests.

If the reviewer has questions or concerns, they should write them in the issue and label it as "workflow::rework". Even if the reviewer only has a question, not any concerns, they should label the issue.

Review requires that the reviewer test the functionality. Ordinarily this is done as part of the "review required" stage and the issue can then move directly to "rework" or be closed. If the work requires that someone other than the reviewer test it and confirm that the work has been completed, the reviewer adds the "workflow::needs testing" label and explains in the issue what testing needs doing.

If the reviewer understands what has been done and is confident that what has been done matches the issue description, they close the issue. For issues with merge requests, it is usually the case that merging the merge request will automatically close the issue.

Review is a corner-stone of our process. It is a serious process and shouldn't be "eyeballed". Reviews allow team members learn from each other. As such, it is important that if you have improvements to suggest, the code is not understandable, or you detect something wrong, you mention this in the merge request or the issue. Peer-review is the major mechanism by which team members to improve their skills day-to-day.

As review is as more about learning than "checking work", it is important to take reviews as a positive process, not negative criticism. This is be especially important for junior team members or mentees.

Things to review when looking at issues with associated merge requests include: functionality, design, code, code style, tests, CI/CD pipelines, security advisories from GitLab CI and code quality check from GitLab CI.

Sometimes the issue will also need a separate testing step. For example, a non-trivial code change may require that a reviewer spin up a local copy of deploy the code change to the development instance. In this case the reviewer will indicate that they are testing the change by moving the issue to "workflow::needs testing". They may also do this if they are happy with the code but are blocked from testing it because, for example, the development instance is being used by another developer.

Rework

You should keep an eye on the board for issues which are in rework which are assigned to you. Go into the issue and review the comments.

If there is simply a question, answer it and add the "workflow::review required" label.

If there is more work to do, add the "workflow::doing" label and continue work on the issue. Once the work is finished, move the issue forward as discussed above in "Doing".

Closing an issue

Occasionally the work described by an issue may either become unnecessary, be replaced by another issue or become merged with another issue. In those circumstances is is fine to just close the issue with a note explaining why it wasn’t necessary any more or where the work has moved to.

Labels

The "workflow:..." labels are used to manage the issue lifecycle but we also use other labels to help categorising labels.

A good rule of thumb for deciding if something should be a label is "would I want to search for issues with it". For example, it is desirable to search for issues which relate to a particular role within a product. It would not be necessary to label issues with, for example, the epic they are part of because one could just list all issues in that epic directly.

Role labels

We have some labels which describe which role(s) within the product team especially relate to the issue.

Important

Remember, a golden rule is "it is your responsibility to move things forward". Just because an issue is labelled with a particular role, it doesn't mean that your input on the issue is not valued.

The following role labels are defined in the uis/devops group:

  • Business Analysis. Issues which require analysing business practices of the University.
  • Development. Issues which require active software or infrastructure-as-code development work.
  • Documentation. Issues which require writing documentation for the team or for end-users.
  • Operations. Issues which relate to deployment or packaging of products.
  • Support. Issues relating to an end-user support request.
  • Testing. Issues relating to testing tools, procedures, technologies or practices.
  • UI Design. Issues which require UI design work.
  • UX Research. Issues relating to user experience research.

Chores

The work required to address some issues is "obvious" or "well rehearsed". We call these issues "chores" since they reflect repetitive work which many team members could do. Examples of chores include: fixing typographic or grammatical errors in documentation, sending an email requesting something from the rest of the business or performing some manual configuration of a service.

We use the "chore" label, defined in the uis/devops group to label such issues.

A "chore" may be sufficiently small that it can be addressed and closed without progressing to review. Usually, however, it is good practice for even small configuration changes to be put into "review required" as a sanity check and to allow at least one other team member to see what work had been done.

Spikes

Some issues have the potential to be open-ended. These are usually issues which require some form of research into a particular technology or business practice. It can be hard to estimate such issues as they could expand to fill the available time.

To make it possible to estimate work from a point of knowledge, an issue may be labelled as a "spike". A spike is an issue which is time-boxed, usually to one day. The team member picking up the spike does as much work as they can within the spike's time window and reports back in the issue.

The report can be used to more accurately gauge what work is required to close an issue and how long it will take. It may also result in the team deciding to close or de-sprint the issue if the work/reward ratio is not favourable.

Usually any code arising from a spike is considered "prototype" or "throwaway". Occasionally the refinement of the code into a more polished initial version may be added as a new issue.

The "spike" label is defined in the uis/devops group.

Priorities

Although one usually reflects priorities within a sprint board by directly ordering the issues in the "sprint ready" column, it may be useful for the product manager to have a wider view of project priorities.

We're still experimenting with using GitLab to provider extra-sprint prioritisation. We could use the "weight" feature from issues or add priority labels.

As part of the experiment, "Priority {1,2,3}" labels have been added to the uis/devops group to see if they help the product manager gain visibility into issue priority.

Groups and projects

Each product has its own group on GitLab. These are usually sub-groups of the DevOps group.

Within each product's group, there are usually one or more projects. Projects can be roughly grouped into the following categories:

  • Support and documentation. Projects which have repository hosting disabled and exist purely as wiki and/or issue tracking projects. These can be the "public" side of private repos if we want to offer public developer focussed documentation or a public issue tracker.
  • Deployment. Projects which contain code to deploy a particular product. These usually only contain deployment configuration and secrets. They are almost always private as they contain deep details of how a product is deployed.
  • Implementation. Projects which contain the implementation of products. We try to structure our products with the idea that they may be deployed in multiple places or by multiple people. As such the implementation of a product should not assume it will be deployed in a particular place or at a particular URL.

When creating new projects in GitLab we try to follow the conventions below.

Open by default

Our software projects are Open Source by default. As such the GitLab project should be made public except where:

  1. It contains internal details of deployments. (These are usally the "deploy" repos.)
  2. It is a project entirely devoted to tracking operational issues with a particular deployment where tier 3 issues need to be escalated.
  3. If contains grandfathered code which has not been audited.

We run our projects as Open Source projects and so the code should be general, understandable and written with the assumption it may be deployed in multiple places. Try to avoid Cambridge-specific hard-coded values although Cambridge-specific defaults are fine.

License

Our default license is the MIT-style license.

Where

Usually a project will be created within a sub-group of the DevOps group.

Naming

Naming is hard. When providing the "short name" for a project, consider that it will appear in URLS. As such it is better to have projects named like uis/devops/raven/deploy rather than uis/devops/raven/raven-deploy.

The "long name" should make sense in isolation so, for the example above, naming the project "Raven Deployment Configuration" makes sense.

Settings

New projects should have the following settings changed:

  • Require 1 approver for all Merge Requests
  • Default to "delete branch after merging" for Merge Requests

Content

Create the project with a README. This provides an initial commit which forms the master branch. The contents of the repository should be opened as the first merge request for review.

Default issue template

Under Settings > General > Default issue template, paste the following:

**DO NOT POST PERSONAL OR CONFIDENTIAL DATA IN THIS PROJECT**

This project is hosting an Open Source project. By design the development
process is open to be inspected. Issues should be related to the code itself and
bugs which affect all deployments. Do not open issues relating only to a single
deployment.

It is useful if you follow [Simon Tatham's guide for reporting
bugs](https://www.chiark.greenend.org.uk/~sgtatham/bugs.html). At a minimum we
would need:

* Browser and Operating System vendor and version numbers.
* A description of what you were *trying* to achieve.
* A description of what you did in order to achieve it.
* A description of what you expected to happen.
* A description of what actually happened.

If we don't have enough information to reproduce your issue, we will probably
not be able to address it.

GitLab "netiquette"

This section lists some "netiquette" rules for GitLab we've found useful over time.

Create issues liberally

If a task appears organically in a conversation, feel free to create a quick issue and tag it as "needs refinement". A common convention is to "pencil an issue in" to the next sprint so that it is visible for the next refinement session.

Consider people’s inboxes

Whenever you create an issue, it emails people watching the project. Try to make sure that initial email will be useful. A great issue will have the following when "Save" is pressed:

  • A descriptive title
  • A clear body
  • Any labels it needs to

The comments section in an issue supports two forms of reply: creating a new discussion thread or responding to an existing one. If responding to a particular thread, please try to do so within the thread and don’t start a new discussion.

Create group-wide labels

It is rare that a label applies only to issues within a particular project. If in doubt, create them within uis/devops.

Be concise

Think of discussion threads as a chat client rather than email. As such, cuddling your comments with "Hi, X" and "Thanks" are superfluous.

Quote when necessary

When replying to one part of a long comment, quote the part you are replying to.

Assume you're talking in a public forum

Always be mindful that GitLab supports open projects as well as closed ones. Never assume that anything said in an issue is confidential. If in doubt, make an issue confidential before replying.

Don't trap words or screenshots in a Word document

Paste the contents of a document into a discussion thread or wiki page. Include a screenshot inline.

Use attachments lightly

Rather than attaching, for example, a spreadsheet to an issue, consider uploading it to a Google Shared Drive for the product and pasting a link in.

Use @-tags appropriately

If you @-tag someone, they will get an email drawing their attention to a thread. This is useful if you want to bring someone into a discussion. Don’t over use it!

Don’t be precious

GitLab and the state within it should be seen as being owned by the team collectively. Never be afraid to add a label, move an issue forward or comment on a discussion. Respect the rights of others to do the same.

Explain your actions

A GitLab issue should be readable from top-to bottom. If you change the state of an issue, move it, re-label it, etc provide a one-line summary of what you're doing and why to help make the issue readable.

Don't assume everyone is familiar with the process

We have a fairly light process but a new team member may still not be familiar with it. In particular, they may not have read this document from end-to-end. If you spot an issue with the wrong label, etc. feel free to fix it but respect that the original author may be unaware of convention.