How to set-up pre-commit hooks on a project¶

We add pre-commit hooks in order to run various linting and auto-formatting tools over the source before they are committed and pushed to GitLab. This how-to guide covers how to add pre-commit hooks to an existing project.

Info

If you are using one of our boilerplates, an appropriate set of hooks will already be present.

Projects should usually have a standard set of pre-commit hooks and some additional language-specific hooks

Standard hooks to add to all projects¶

Add the following hooks to all projects, by placing them in .pre-commit-config.yaml at the root of the project:

.pre-commit-config.yaml

# See https://pre-commit.com/hooks.html for more hooks
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
        args:
          - --unsafe
      - id: check-json
      - id: check-toml
      - id: check-xml
      - id: check-added-large-files
      - id: check-executables-have-shebangs
      - id: check-merge-conflict
      - id: check-symlinks
      - id: detect-private-key
      - id: mixed-line-ending
      - id: pretty-format-json
        args:
          - --autofix
          - --no-sort-keys
      - id: debug-statements
  - repo: https://github.com/DavidAnson/markdownlint-cli2
    rev: v0.14.0
    hooks:
      - id: markdownlint-cli2
        args: ["--fix"]
        language_version: 22.10.0

Warning

As the rev parameter above is hard-coded, you should also set-up renovatebot to keep this and your project's other dependencies up-to-date.

Python-specific hooks¶

Additionally, the following pre-commit plugins should be added to .pre-commit-config.yaml for Python projects (replacing the rev parameters with latest releases of the plugins):

  - repo: https://github.com/python-poetry/poetry
    rev: 1.5.1
    hooks:
      - id: poetry-check

  - repo: https://github.com/editorconfig-checker/editorconfig-checker.python
    rev: 2.7.2
    hooks:
      - id: editorconfig-checker
        args: ["-disable-indent-size"]

  - repo: https://github.com/psf/black
    rev: 23.3.0
    hooks:
      - id: black

  - repo: https://github.com/timothycrosley/isort
    rev: 5.12.0
    hooks:
      - id: isort

  - repo: https://github.com/pycqa/flake8
    rev: 6.0.0
    hooks:
      - id: flake8

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.4.1
    hooks:
      - id: mypy
        additional_dependencies: [types-requests, types-oauthlib]
        args: [--explicit-package-bases]

Java-specific hooks¶

Additionally, we use the google-java-format hook to ensure that Java source is formatted consistently. Add the following to pre-commit-config.yaml for Java projects:

    - repo: https://gitlab.developers.cam.ac.uk/uis/devops/continuous-delivery/precommit-google-java-format
      rev: v1.2.0
      hooks:
        - id: google-java-format
          files: "\\.java$"
          args: ["-i"]

This plugin formats your code to match the Google Java Style Guide on git commit. If your code already meets the required code format the plugin will pass and allow your commit to continue. If your code requires formatting it will format the code automatically for you but fail pre-commit. This allows you to review the changes and commit any updates to continue.

To run locally, the current v1.2.0 version of the pre-commit formatter requires Java 11 in your path.

Warning

Minor differences in Java source formatting have been seen when running the google-java-format pre-commit hook locally with Java versions > 11 compared to the results when the hook runs as part of a GitLab pipeline (which is currently running the formatter with Java 11). To avoid inconsistencies it is recommended that you must run the formatter with Java 11 in your path.

If your Java project requires a different version of Java to compile, you can use sdkman to easily switch between different versions at your command line.

Installing pre-commit hooks¶

After adding a .pre-commit-config.yaml or modifying it, the following should be run in order to set-up the hooks:

pre-commit-install

The above command will also need to be run after the repository has been newly-cloned in order to set-up the hooks on your local copy.

Using pre-commit¶

Having been set-up, pre-commit will run against staged files whenever you run git commit in this repository. If files do not conform to the checks, then where possible the pre-commit hooks will modify them accordingly. The commit will be aborted.

The modified files can then be re-staged with git add and the commit re-tried.

Resolving existing compliance failures¶

The behaviour above will likely not be sufficient when adding pre-commit to an existing project. For example:

you may desire to re-format all existing source code to conform to a relevant style guide
when a repository is using our standard CI templates, the CI pipeline pre-commit job executes the checks against all files in the repository. This will cause the pipeline to fail if there is any existing code which does not conform to the pre-commit checks.

You can execute the hooks separately from a git commit against all files in your repository by running:

pre-commit run --all-files

Configuring Git to ignore bulk formatting commits¶¶

Adding pre-commit hooks to an existing project can result in formatting a large number of files. If you use git blame to show changes from previous commits, a single commit with a large number of formatting changes can reduce the usefulness.

This will often produce a large set of formatting changes which do not affect the behaviour or design of the code. Once committed, these would limit the usefulness of git blame, since many lines of code would be caught up in this bulk-formatting-change commit.

To avoid this, having applied the commit its SHA can be added to a .git-blame-ignore-revs file in the root of the repository. For example:

# initial pre-commit formatting changes
c998bb1ed4b3285398c9c7797135d3f060243c6a

You can then configure your local copy of the repository by running:

git config blame.ignoreRevsFile .git-blame-ignore-revs

and commit the file for the benefit of your colleagues:

git add .git-blame-ignore-revs
git commit -m "chore: add .git-blame-ignore-revs"

It should be noted that git config changes do not get pushed or persisted, so these commits will still show in the blame view on GitLab and for other users unless/until they set blame.ignoreRevsFile in their local copy of the repository.

Summary¶

In this how-to you learned how to set-up a standard set of pre-commit hooks on a project.

Further information on using pre-commit: