Skip to content

How to synchronise secrets from 1password to Google Cloud

In this guide you will learn how to synchronise a secret from 1password to a Google Secret Manager secret. Usually this is done when 1password is the canonical source for a secret. Examples include API keys issued by services which cannot be automated or sensitive configuration which is manually maintained.

Prefer automatic generation of credentials when possible

It is preferable to have terraform create and manage secrets fully. For example, if one is generating a private key for a certificate, use the tls_private_key resource rather than creating a key manually via openssl.

We use a tool named sanctuary to synchronise secrets. This tool ships as part of our logan tool since it is usually used as part of a deployment to Google Cloud.

Ensure prerequisites are present

To synchronise a secret you must have:

  • an item in 1password which you want to be the source of a secret, and
  • a Google Secret Manager secret which you want to be the destination of a secret.

For the purposes of this how to, an example secret has been created in an example Google Cloud project and an example 1password item has been created in the "DevOps Division Vault".

I can't see one or both of those, how can I get access?

If you can't see the Google Secret Manager secret, firstly check that you appear in the "devops" team in our team data repository and that you have signed in to Google with the correct account. If you do not appear in that repository, talk to the tech lead on your project.

All DevOps division members should have access to the DevOps Division 1password vault. If that's not the case, again ask your tech lead who can pass the request on to one of our 1password admins.

You will also need to install the sanctuary tool. Installation instructions can be found in the logan README.

Make sure Google Secret Manager secrets are created

The sanctuary tool manages Google Secret Manager secret versions but does not create the secrets themselves. It is expected that there is some terraform deployment which creates the secret and makes use of its contents. For example, the deployment may use our standard terraform module to create a blank secret:

module "secret" {
  source      = "git::https://gitlab.developers.cam.ac.uk/uis/devops/infra/terraform/gcp-secret-manager.git?ref=v3"
  project     = local.project
  region      = "europe-west1"
  secret_id   = "test-secret"
}
What if my secret needs some value before the first synchronisation?

Depending on the nature of the deployment, the secret may need to start with some value in order for the initial deployment to succeed even if that value is blank or some placeholder.

A workaround for this "chicken and egg" problem is to create an initial version of the secret and tell terraform to expect the secret to be changed subsequently. For example:

# A secret used for configuration data.
resource "google_secret_manager_secret" "secret" {
  secret_id = "secret"

  replication {
    user_managed {
      replicas {
        location = "europe-west2"
      }
    }
  }
}

# An initial value for the secret which is an empty JSON document. Subsequent versions
# are managed via sanctuary.
resource "google_secret_manager_secret_version" "initial_secret" {
  secret = google_secret_manager_secret.secret.id

  secret_data = "{}"

  lifecycle {
    ignore_changes = [secret_data, enabled]
  }
}

Check that you can copy 1password item identifiers

1password associates a Universally Unique Identifier (UUID) with each item. You'll need this item identifier in order to configure sanctuary.

1password desktop

If you are running the 1password desktop client, you should enable the "Show debugging tools" setting which adds the ability to copy an item UUID:

Enabling "Show debugging tools" in 1password.

This adds a "Copy item UUID" option to the "..." menu in 1password:

Copying an item's UUID.

1password browser extension

If you are using the 1password browser extension, the item id can be obtained from the item link. Firstly, copy the link:

Copying an item's link in the browser extension.

The link will have the form https://start.1password.com/open/i?a={...}&v={...}&i={...}&h={...}. The item identifier can be obtained from the i={...} portion of the link.

Add sanctuary configuration to .logan.yaml

The sanctuary tool can be configured in a wide variety of ways which are covered in the tool's documentation. We consider the simple case of copying fields from a 1password item into a JSON document stored in a Google Secret. Update or create a .logan.yaml file with a sanctuary: configuration block:

# .logan.yaml
version: "1.3"

sanctuary:
  secrets:
    some-name:
      from:
        op-cli-item:
          item-id: 3gf2o33iyz4grffycrxcm6g5pu
          fields: [username, credential, custom-field]
          use_field_labels: true
      to:
        google-secret:
          project: example-devel-40c81aca
          name: sanctuary-example

The item id is the one copied from the 1password desktop app. Fields to be copied from the item must be listed explicitly so that it is always clear which secrets are being synchronised.

Synchronise 1password items to Google Secret Manager secrets

Run the sanctuary sync command from the directory containing the .logan.yaml file. A --verbose flag can be used to print information on what is being done:

$ sanctuary sync --verbose
[info     ] Processing secret              secret_name=some-name
[info     ] Using application default Google credentials.
[info     ] Ensuring that secret exists and is ready for update. secret=SecretSpec(google_secret=GoogleSecret(project='example-devel-40c81aca', name='sanctuary-example', version='latest', destroy_previous_versions=True), op_cli_item=None, op_cli_document=None)
[info     ] Updating secret                from_secret=SecretSpec(google_secret=None, op_cli_item=OnePasswordCLIItem(item_id='3gf2o33iyz4grffycrxcm6g5pu', fields=['username', 'credential', 'custom-field'], field=None, use_field_labels=True), op_cli_document=None) to_secret=SecretSpec(google_secret=GoogleSecret(project='example-devel-40c81aca', name='sanctuary-example', version='latest', destroy_previous_versions=True), op_cli_item=None, op_cli_document=None)
[info     ] Destroying previous versions of secret name=projects/example-devel-40c81aca/secrets/sanctuary-example

Hint

The --dry-run flag can be used to display what sanctuary would do without actually doing it. This is useful if you want to check what will happen before making changes.

Once sanctuary sync is run, the Google Secret Manager secret is updated with the contents of the 1password item:

The contents of the Google Secret Manager secret after synchronisation.

Summary

In this how to, you learned what the sanctuary tool is, how to configure it and how to synchronise a secret from 1password to a Google Secret Manager secret.

Next steps