DataKitchen DataOps Documention

Vault & Secrets

Vaults securely store sensitive infrastructure and toolchain credentials as Secrets, which are encrypted and stored remotely. Vaults can be configured in a number of ways to match your organization's operational and security needs. Vaults are ideal for associating kitchens with environments like production, staging, and development.

A view of the global and kitchen-level vault interface.

A view of the global and kitchen-level vault interface.

A global vault with a custom, private connection; and a kitchen vault with a private connection inherited from the parent kitchen.

A global vault with a custom, private connection; and a kitchen vault with a private connection inherited from the parent kitchen.

Global Vault

The Global Vault contains secrets that are available for use by recipes in any kitchen. It is enabled by default. The global vault can be used alone, disabled in favor of kitchen vaults, or used in conjunction with kitchen vaults. All users can view secrets (but not their underlying values), as well as edit, create, or delete secrets contained in the global vault, as long as it is not configured as a private, custom vault.

Disable the Global Vault

To disable the global vault and use only kitchen vaults, perform these steps for the UI or command line interface.

  • UI: Click the Configure Vault link on the Secrets page within any kitchen to open the Configure Global Vault dialog, then select the None option and click Update.
Disabling the Global Vault.

Disabling the Global Vault.

  • Command Line: Use the vault-info command with the --global option to see information about the global vault.
~ $ dk vault-info --global



Current context is: default
YYYY-MM-DD HH:MM:SS - Getting the vault info

---- global config ----
 prefix                 vault_prefix
 private                False
 service                custom
 token                  ******
 url                    http://12.345.678.9:1234

Disable the global-vault using the vault-config command with the --disable option.

~ $ dk vault-config --disable



Current context is: default
You are configuring the global vault!. Are you sure you want to proceed? [yes/No]yes
YYYY-MM-DD HH:MM:SS - Setting the vault info
Done.

Reenable the Global Vault

  • UI: On the Secrets page, the click the Configure Vault link to open the Configure Global Vault dialog, then select the Default option and click Update.
  • Command Line: Use the vault-config command to configure a new default connection setting.
~ $ dk vault-config --url http://12.345.678.9:1234 --token sample-token



Current context is: default
You are configuring the global vault!. Are you sure you want to proceed? [yes/No]yes
YYYY-MM-DD HH:MM:SS - Setting the vault info
Done.

Kitchen Vaults

Kitchen Vault connection settings are defined at the kitchen level.

When order runs process, they first look to kitchen vaults for secret values. If a secret value is not found in a kitchen-level vault, the order runs next looks to the global vault for the secret value. If a secret is found in the kitchen vault, the global vault's corresponding value will be ignored.

The Redshift password secret value from the kitchen vault takes precedent over the value in the global vault.

The Redshift password secret value from the kitchen vault takes precedent over the value in the global vault.

The Redshift password secret value from the global vault is ignored.

The Redshift password secret value from the global vault is ignored.

Vault Connections & Management

Each kitchen may be connected to only one kitchen vault at any given time. Only users included in the kitchen's staff can manage kitchen vault connection settings.

Users who are not in the kitchen staff cannot enter that kitchen to view or manage its vault settings.

Users who are not in the kitchen staff cannot enter that kitchen to view or manage its vault settings.

Creating Kitchen Vault Connections

  • Via Kitchen Wizard: The kitchen creation process prompts you to configure a Vault Service connection.
The connection settings for a custom kitchen vault.

The connection settings for a custom kitchen vault.

  • Via Secrets Page: Vault connection settings may be edited at any time after they are created.

Note that a custom connection provides more options, but requires more configuration.

Configuring a kitchen vault connection from the Secrets page.

Configuring a kitchen vault connection from the Secrets page.

Kitchen Vault Connection Inheritance

Custom kitchen vault connection settings may be configured as inheritable by child kitchens from the parent kitchen in which the connections are defined. The inheritance of these settings applies across generations of child kitchens.

Default kitchen vault connection settings may not be inherited.

Editing vault connection settings
Users can modify inheritable, custom settings only from within the kitchen where the connection was initially configured. They cannot make edits to these settings from a child kitchen that inherited them.

Deleting a kitchen with vault connection inheritance
If a user attempts to delete a kitchen that has child kitchens with inherited vault connections, the platform generates a warning to first delete relevant child kitchens.

The platform prevents a user from deleting a kitchen if there is a custom kitchen vault connection inherited by child kitchens.

The platform prevents a user from deleting a kitchen if there is a custom kitchen vault connection inherited by child kitchens.

Turning off inheritance
Users can set a custom kitchen vault to not be inheritable by child kitchens.

  1. From the Kitchens screen, enter the kitchen by clicking its name.
  2. Click the Secrets tab in the menu bar at the top of the Recipes page.
  3. On the Secrets screen in the Kitchen Secrets section, click the Configure Vault link.
  4. In the Configure Kitchen Vault dialog, the "Custom" option should already be selected.
  5. Clear the Inheritable Vault Service checkbox.

If a user changes a kitchen vault's connection setting to non-inheritance, and child kitchens exist that previously inherited this connection, the child kitchens will lose their inherited connection.

If the user turns on inheritability again for the kitchen vault, the child kitchens that had inherited the connection settings will re-inherit the connection automatically.

Default vs. Custom Vaults

Both global and kitchen vaults may be configured as either default or custom vault types.

Default Vault

By default, DataKitchen customers have access to a secure vault hosted by DataKitchen. Default vaults do not require the configuration of connection settings.

No connection setting configuration is required for Default Vault settings.

No connection setting configuration is required for Default Vault settings.

Custom Vault

Customers may opt to set up their own, externally-hosted vault services. Custom vaults offer a number of configuration options.

A custom Vault's connection settings.

A custom Vault's connection settings.

  • Private: Setting a vault service as private disables the viewing, editing, creation, and deletion of Secrets via the UI and command line. Secret values may still be leveraged by recipes in private vault service configurations as long as the vault service is accessible on the environment where OrderRuns are executed (the Mesos Agent).
  • Vault Service URL: This field is required. This URL must be accessible by DataKitchen if the custom vault configuration is NOT private. Set to a blank value for the Global-Vault if you wish to disable this vault's connection. An example vault service URL is http://34.234.109.5:8200.
  • Vault Service Token: If this token is set to periodically expire, you must either renew the token or update its value at the configuration screen shown above. The token value is obscured from the user once it has been saved. Set to a blank value for the global vault if you wish to disable this vault's connection.
  • Vault Path Prefix: This is an optional value denoting a path prefix, allowing you to isolate DataKitchen-related secrets from other secrets in your vault(s). A valid example entry for this field is "datakitchen".
    Vault paths follow the structure secret/[some path prefix]/[path]. Here, you need only denote the [some path prefix] value when defining the path prefix. The path prefix need not be included in the secret path. For example, the secret path vault://dockerhub/password is valid whether or not a path prefix is configured.

Managing Secrets

Secret Management is Disabled for Private, Custom Vault Services

Note that if you have selected to configure a Vault service as private you will be unable to create, delete, edit, or view Secrets via DataKitchen's UI or command line tool.

View Secrets

Global vault secrets and kitchen vault secrets are listed on the Secrets page of the DataKitchen UI. If a vault has a custom connection set to private, that vault's secrets will not be visible via the DataKitchen application, though the secret values may still be called by order runs in process.

Viewing secrets in the UI for both global and kitchen-level vaults.

Viewing secrets in the UI for both global and kitchen-level vaults.

Create Secrets

Secret values may either be saved as string or files containing keys.

  • String Values: Vault entries for a common service can be grouped under a common path. For example, the following values can be inputted to create secrets related to Docker Hub.
    • dockerhub/email
    • dockerhub/namespace
    • dockerhub/password
    • dockerhub/username
Adding a vault secret with a string value.

Adding a vault secret with a string value.

  • Key Files: For example, you may upload a .pem file:
Adding a Vault Secret with a value set equal to a file.

Adding a Vault Secret with a value set equal to a file.

Editing Secrets

Viewing Vault Secret Values

After a secret value is saved, you cannot view its underlying value. You may, however, overwrite the saved value.

When editing a pre-existing secret, the user cannot view the existing value, only the new value that is being entered.

When editing a pre-existing secret, the user cannot view the existing value, only the new value that is being entered.

Deleting Secrets

Deleting a vault secret in the UI.

Deleting a vault secret in the UI.

Using Secrets

See Secrets Usage for best practices for referencing secrets in recipes.

Updated a day ago


Vault & Secrets


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.