Skip to content

How to add tasks to an existing webapp

In some cases, it is desirable to have task-style asynchronous or scheduled workloads tightly coupled to a web application. This could be due to code-sharing, or a need to keep versions of tasks and the applications tightly in-sync.

Adding task functions

The first step is to add some Python code for the tasks.

To create new ucam-faas task functions the ucam-faas python library should be installed into the poetry dependencies. It is recommended to do this in a dedicated poetry group:

poetry add -G tasks ucam-faas

Other dependencies needed only by the tasks function code should be added into this dependency group.

Next we add a dedicated module at the root level of the repository to store the task function code:

mkdir task_functions
touch task_functions/__init__.py task_functions/main.py
mkdir task_functions/tests
touch task_functions/tests/__init__.py task_functions/tests/test_main.py

And then some task-running code:

# task_functions/main.py
from ucam_faas import raw_event

@raw_event
def my_task(event: bytes) -> None:
    # Do whatever work required.
    pass

Note: the raw_event handler is suitable mostly for tasks that are triggered on schedules, or can ignore the message contents. For more information on parsing the message that triggered the function and using it in the function code see the development guide.

Adding tests is omitted from this guide, but is expected.

Adding a new Docker target

It is expected that (from the standard webapp boilerplate) a Dockerfile exists in the webapp repository, with an existing installed-deps target.

To add a task image target:

...
# The installed-deps target should be above the new target.
FROM base AS installed-deps
...

###############################################################################
# Example task target
FROM installed-deps AS my-task

RUN set -e; \
  poetry export --only=tasks --format=requirements.txt --output=.tmp-requirements.txt; \
  pip install --no-cache-dir -r .tmp-requirements.txt; \
  rm .tmp-requirements.txt

COPY . .

RUN chown -R webapp .
USER webapp

ENTRYPOINT ["ucam-faas", "--target"]
CMD ["my_task", "--source", "task_functions/main.py"]

# The new target should not be last in the file.
...

In order to deploy the task container it will also need to be built in the CI pipeline and pushed out to the relevant GCP meta project. This can be done using the multi-target docker builds functionality. In short, adding to the .gitlab/webapp.gitlab-ci.yml:

# .gitlab/webapp.gitlab-ci.yml
include:
  ...
  # Include the multi-target project, with the target specified.
  - project: "uis/devops/continuous-delivery/ci-templates"
    file: "/auto-devops/multi-target-docker-images.gitlab-ci.yml"
    ref: v7.29.0
    inputs:
      docker_build_targets: ["my-task"]

variables:
  ...

  # If you have configured your repository from the boilerplate, you likely have the following
  # variable set. If you do, it must be *removed* for multi-target docker builds to function.
  CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE

  # Override variables from the multi-target template, this preserves the default behaviour of the
  # common templates of building, scanning, and pushing the default/final docker image target.
  BUILD_DISABLED: ""
  CONTAINER_SCANNING_DISABLED: ""
  DISABLE_ARTIFACT_REGISTRY_PUSH: ""

Warning

Overriding the CI_APPLICATION_REPOSITORY variable will prevent the multi-target docker build pushing to the artefact registry. Ensure it is removed if this override is present.

Now the task container which runs the my_task function when triggered will be built and pushed to the artefact registry for deployment.

Summary

In this guide you learned how to add a task function and container build to an existing webapp.

Next Steps