Background

Pipeline configuration files can reference other files for pipeline configuration, environment variables, task parameters, etc.

Since files may contain sensitive information, permission checks are needed for file references to prevent information leakage.

Reference Methods

CNB supports four types of configuration file references:

• Pipeline template reference: include
• Environment variable reference: imports
• Built-in task parameter reference: optionsFrom
• Plugin task parameter reference: settingsFrom

Note:

For flexible pipeline task parameter configuration, file paths declared in imports, optionsFrom and settingsFrom support environment variables.

You can declare environment variables in the pipeline first, then reference them in tasks, like:

main:
  push:
    - env:
        CNB_CONFIG_URL: https://cnb.build/<your-repo-slug>/-/blob/main/xxx
        CNB_CONFIG_FILE: account.yml
      imports:
        # Assume env1.yml declares variable CNB_ENV_FILE_URL
        - https://cnb.build/<your-repo-slug>/-/blob/main/xxx/env1.yml
        # File paths after imports can use variables declared in previous files
        - ${CNB_ENV_FILE_URL}/env2.yml
      stages:
        - name: echo
          script: echo 1
        - name: Built-in task
          type: some-type
          optionsFrom:
            - ${CNB_CONFIG_URL}/${CNB_CONFIG_FILE}
        - name: Plugin task
          image: some-image
          settingsFrom:
            - ${CNB_CONFIG_URL}/${CNB_CONFIG_FILE}

Additionally, optionsFrom and settingsFrom support reading local files. Even if ./path/to/file doesn't exist in the repository, you can write it in the pipeline for loading during execution, like:

main:
  push:
    - env:
        CNB_CONFIG_URL: https://xxx.com/p1/xxx
        CNB_CONFIG_FILE: account.yml
      stages:
        - name: echo
          script: echo some-content > ./path/to/file-not-in-git
        - name: Built-in task
          type: some-type
          optionsFrom:
            # Reference file generated by pipeline but not in git
            - ./path/to/file-not-in-git
        - name: Plugin task
          image: some-image
          settingsFrom:
            # Reference file existing in git
            - ./path/to/file-in-git

File Types

In CNB, three types of configuration files are supported for reference:

YAML Files

Files with .yml or .yaml extension, like:

account.yml

SOME_ACCOUNT: some-account
SOME_PASSWORD: some-password

JSON Files

Files with .json extension, like:

account.json

{
  "SOME_ACCOUNT": "some-account",
  "SOME_PASSWORD": "some-password"
}

Text Files

Files not of the above two types are parsed as text files, like:

account.txt

SOME_ACCOUNT=some-account
SOME_PASSWORD=some-password

The variables declared in a text file do not support objects. It is recommended to use the yaml or json format instead.

Permission Checks

For configuration files in public or same-origin repositories, pipelines can reference them directly.

For private and non-origin repositories, you can declare four fields in the configuration file: allow_slugs, allow_events, allow_branches, and allow_images to control the access scope. All four fields are in glob pattern strings (multiple expressions separated by , or |) or string arrays.

For example:

# String format, multiple expressions separated by ',' or `;`
allow_slugs: "group/project,some/another-project/feature-*"
# String array
allow_events:
  - push
  - tag_push
  - "tag_deploy.*"
# Allow all images
allow_images: "**"
# Allow branches starting with main or feature/
allow_branches:
  - main
  - "feature/*"

It is recommended to declare multiple expressions using string arrays for better readability. If the configuration file is in a text format that does not support declaring arrays, you can use the string format.

If the above four fields are not declared, it will check if the triggerer of the pipeline is a developer or a role higher than that in the repository of the configuration file. If they are, access is granted; otherwise, access is denied.

Where:

• allow_slugs specifies which repositories' pipelines are allowed to read the current file.
• allow_events specifies under which events pipelines can read the current file. Configurable event names refer to Trigger Events.
• allow_branches specifies under which branches pipelines can read the current file.
• allow_images specifies which image plugins are allowed to read the current file.

The permission check flow for whether a pipeline can load target files is shown below:

Overall flow:

Flow explanation:

• Same source: Pipeline and target file belong to same repository
• Allow check: Declared allow fields checked sequentially, all must pass
• File read permission: See Role Permissions

Allow necessity check explanation:

• "Pull request" events include pull_request and pull_request.update events
• In PRs, especially from forked repos, source branch code is untrusted without admin review. For security, config files must have allow_events for pull_request pipelines
• Pipelines can use third-party plugins. For security, config files must declare allow_images for plugin tasks

Branch checked in allow_branches is same as CNB_BRANCH.

Image check notes:

• Plugin task parameters can be set via settingsFrom or imported as env vars via imports. Both count as plugin task file references
• When image name has no tag, both name and name:latest are matched against allow_images. Match either passes
• Non-plugin task image is empty string, can't match any glob pattern. Config files with allow_images can't be referenced by non-plugin tasks
• Note: Jobs with both image and script are script tasks using image as execution environment
• Syntactically pipelines can declare image as build environment but not as plugin task. Config files with allow_images can't be referenced at pipeline level

Examples

Pipeline Secret File Reference

# Configuration file secret.yml in the keystore repository
allow_slugs: "p1/**"
allow_events: push
allow_branches: main
# DockerHub username and password
DOCKER_USER: docker-user
DOCKER_PWD: docker-pwd

# Pipeline configuration file .cnb.yml
main:
  push:
    - services:
        - docker
      imports: https://cnb.build/<your-repo-slug>/-/blob/main/xxx/secret.yml
      stages:
        - name: docker login
          script: |
            # Log in to DockerHub
            docker login -u ${DOCKER_USER} -p "${DOCKER_PWD}"
            # docker build xxx
            # docker push xxx

The pipeline references the configuration file secret.yml from another keystore repository, but the pipeline triggerer does not have read access to this file.

The referenced file secret.yml declares allow_slugs, allow_events, and allow_branches.

If a repository under the p1 organization triggers a push event on the main branch, the pipeline can reference this secret file and use the sensitive information within it.

This ensures that only the owner or administrator of the keystore repository can view or modify the sensitive content in secret.yml, while still allowing it to be referenced by pipelines that meet the specified conditions.

Plugin Task Parameter Reference

# Configuration file image-settings.yml in the keystore repository
allow_images: "registry.com/image1/**"
allow_slugs: "p1/**"
arg1: arg1
arg2: arg2

# Pipeline configuration file .cnb.yml configuring a plugin task
name: image job
image: registry.com/image1/print:latest
settingsFrom:
  - https://cnb.build/<your-repo-slug>/-/blob/main/xxx/image-settings.yml

If the image is created by a third party, there may be a risk of information leakage. This can be mitigated by using allow_images to restrict the configuration file to only be used by plugin tasks with specific image names that are deemed safe.

The image registry.com/image1/print:latest matches registry.com/image1/**, so the image check passes.

allow_slugs specifies that the configuration file can only be referenced by pipelines of repositories under the p1 organization, further controlling the scope of the configuration file's usage.