poe
Tests Pipeline - Currently Wilson Team POC¶
The poe
test pipeline is currently in POC state and being used and developed by the Wilson team.
It is a alternative way to run all types of software tests for the unified DevOps platform,
although currently it is primarily being used for tests that don't currently have existing off
the shelf test runners.
Background¶
Primary Goals¶
- Support non-python (and python) based tests e.g. TS, cypress, selenium, etc.
- Easily add multiple test types within a single repo, e.g. python and TS tests
- Cheaply add docker based dependencies, e.g. pubsub emulator
- Greater consistency between CI and local development - ability to run the same command
Secondary Goals¶
- Able to test built docker images, e.g. make API call on built docker image
- Reduction of sources of truth, e.g. a single file that defines the database version tests run against
- Concise declaration of test commands
- Pipeline level parallelism
Guide¶
In order to use the poe
based test pipeline the first step is to include the pipeline template.
- project: "uis/devops/continuous-delivery/ci-templates"
file:
"/auto-devops/poe-tests.gitlab-ci.yml"
#ref: poe-tests-poc # at the time of writing the code is not yet merged
To run a test command you need to add an entry that starts with test-
or tests-
to the list of
poe
commands in your repos pyproject.toml
file. Non-python projects need not have any other
entry in pyproject.toml
than commands or you can use the slimed down poe_tasks.yaml/json/toml
which follow slightly different syntax, see
without_poetry.
That is all that is needed for a child pipeline to execute.
E.g.
[tool.poe.tasks."test-hello-world"]
cmd = "echo hello"
An example test command using docker compose run
.
[tool.poe.tasks."test-cypress"]
cmd = "docker compose run cypress"
However, you will need to make sure that your test commands only use the tools listed in the guidebook as the minimum developers tools. Otherwise the pipeline is likely to fail with missing dependencies. The test commands should bring their own dependencies beyond the minimal set of tooling defined here: prepare-your-system (at time of writing this link has an open MR that brings these values up to date).
This generally means running tests through docker compose. This also includes when the tests do only need tooling available in the pipeline docker image (e.g. python) as the tests should be incorporating pinning of tooling/libraries versions used with the tests. The availability of python is only intended for tooling/scripting that helps to run the tests and should not to be used run the test cases themselves. I.e. make sure python tests run in a container defined in the docker compose file.
The main pipeline template uses dynamically generated child pipelines in order to run each command in parallel. This means that in the gitlab UI there are multiple pipeline entries and nested child pipelines. E.g. an expanded view with multiple tests will look something like:
In this example one of the two poe
test commands has failed (test-cypress-last-name-disable
)
and the other has passed. The job poe-tests-generator
job should generally always pass as this
is only is responsible for generating the child pipelines template. poe-tests-child-trigger
will
general reflect fail if any child pipelines fail and pass otherwise.