PyPI Artifact Registry

About 1183 wordsAbout 4 min

PyPI Artifact Registry is a repository for storing and distributing Python packages. Developers can upload, manage, and install various Python third-party libraries and tools through it.

Local Development

Configure Pull Credentials

Option 1: Add pip source configuration in ~/.pip/pip.conf (MacOS/Linux) or %HOME%\pip\pip.ini (Windows)

[global]
# TOKEN: Access token password; REPO_URL: Artifact Registry address
# Example: index-url = https://cnb:29bGg84xqRf3ZweNpYlL92e7tDv@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
index-url = https://cnb:<TOKEN>@<REPO_URL>

Option 2: You can also use command-line setup

# TOKEN: Access token password; REPO_URL: Artifact Registry address
# Example: pip config set global.index-url https://cnb:29bGg84xqRf3ZweNpYlL92e7tDv@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
pip config set global.index-url https://cnb:<TOKEN>@<REPO_URL>

Configure Push Credentials

twine configuration

Add the following configuration in ~/.pypirc file

[distutils]
index-servers =
 coding-pypi
[coding-pypi]
# Artifact Registry address
repository: <REPO_URL>
username: cnb
# Access token password
password: <TOKEN>

Example:

[distutils]
index-servers =
 coding-pypi
[coding-pypi]
repository: https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
username: cnb
password: "29bGg84xqRf3ZweNpYlL92e7tDv"

poetry configuration

Run the following commands

poetry config http-basic.cnb-pypi cnb <TOKEN> # Access token password
poetry config repositories.cnb-pypi <REPO_URL> # Artifact Registry address

Example:

poetry config http-basic.cnb-pypi cnb 29bGg84xqRf3ZweNpYlL92e7tDv
poetry config repositories.cnb-pypi https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple

Pull Artifacts

#Example 1: pip install requests
#Example 2: Pull a specific version pip install requests==2.30.1
pip install <PACKAGE_NAME>

# Example, pull from a temporary source pip install -i https://cnb:29bGg84xqRf3ZweNpYlL92e7tDv@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple requests
pip install -i https://cnb:<TOKEN>@<REPO_URL> <PACKAGE_NAME>

Build and Push Artifacts

twine

python -m build  
twine upload -r cnb-pypi dist/*

poetry

poetry build
poetry publish -r cnb-pypi

Cloud Native Build

Pull Artifacts

Option 1: Use built-in CNB_TOKEN environment variable:

main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest 
      stages:
        - name: Pull artifacts
          script:
            # Artifact Registry address
            # Example: pip config set global.index-url https://cnb:${CNB_TOKEN}@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
            - pip config set global.index-url https://cnb:${CNB_TOKEN}@<REPO_URL>
            # Example: pip install requests
            - pip install <PACKAGE_NAME>

Option 2: Use secret registry file:

main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        # Import secret repository configuration file
        - <SECRET_REPO_URL> 
      stages:
        - name: Pull artifacts
          script:
            - pip config set global.index-url https://<USER_NAME>:<PASS_WORD>@<REPO_URL>
            - pip install <PACKAGE_NAME>

Example:

main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        - https://cnb.cool/cnb-demo/env-demo/-/blob/main/envs/env.yml 
      stages:
        - name: Pull artifacts
          script:
            # Import secret registry file and inject USER_NAME and PASS_WORD as environment variables
            - pip config set global.index-url https://${USER_NAME}:${PASS_WORD}@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
            - pip install requests

Push Artifacts

twine configuration

Option 1: Use built-in CNB_TOKEN environment variable:

main: 
  push: 
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      stages:
        - name: Build & Upload to repository
          script:
            - export TWINE_USERNAME="cnb" 
            - export TWINE_PASSWORD=${CNB_TOKEN}
            # Artifact Registry address
            # Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple 
            - export TWINE_REPOSITORY_URL=<REPO_URL>
            - python -m build
            - twine upload dist/*

Option 2: Use the secret repository file

main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        # Secret repository file address
        # Example: https://cnb.cool/cnb-demo/env-demo/-/blob/main/envs/env.yml
        - <SECRET_REPO_URL> 
      stages:
        - name: Package & Upload to Registry
          script:
            # Secret repository file USER_NAME configuration
            - export TWINE_USERNAME=${USER_NAME}
            # Secret repository file PASS_WORD configuration
            - export TWINE_PASSWORD=${PASS_WORD}
            # Artifact Registry address
            # Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
            - export TWINE_REPOSITORY_URL=<REPO_URL>
            - python -m build
            - twine upload dist/*

poetry configuration

Option 1: Use built-in CNB_TOKEN environment variable:

main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest 
      stages:
        - name: Build & Upload to repository
          script:
            - export POETRY_HTTP_BASIC_CNB_PYPI_USERNAME="cnb"
            - export POETRY_HTTP_BASIC_CNB_PYPI_PASSWORD=${CNB_TOKEN}
            # Artifact Registry address
            # Example: poetry config repositories.cnb-pypi https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
            - poetry config repositories.cnb-pypi <REPO_URL>
            - poetry build
            - poetry publish -r cnb-pypi

Option 2: Use the secret repository file:

main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        # Secret registry file address, example https://pypi.cnb.build/cnb-demo/env-demo/-/blob/main/envs/env.yml 
        - <secret_repo_url>
      stages:
        - name: Build & Upload to repository
          script:
            # Secret repository file USER_NAME configuration
            - export POETRY_HTTP_BASIC_CNB_PYPI_USERNAME=${USER_NAME}
            # Secret repository file PASS_WORD configuration
            - export POETRY_HTTP_BASIC_CNB_PYPI_PASSWORD=${PASS_WORD}
            # Artifact Registry address
            # Example: poetry config repositories.cnb-pypi https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
            - poetry config repositories.cnb-pypi <REPO_URL>
            - poetry build
            - poetry publish -r cnb-pypi

Workspaces

Prerequisites

Add the following configuration in .cnb.yml file

  $:
    vscode:
      - docker:
          image: docker.cnb.cool/examples/language/python-3:latest
        services:
          - vscode
          - docker
        # If using the secret repository file, fill in the following configuration  
        imports:
          # Example: https://cnb.cool/cnb-demo/env-demo/-/blob/main/envs/env.yml
          - <SECRET_REPO_URL>

Configure Pull Credentials

Option 1: Use built-in CNB_TOKEN environment variable:

# Example: pip config set global.index-url https://cnb:${CNB_TOKEN}@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
pip config set global.index-url https://cnb:${CNB_TOKEN}@<REPO_URL>

Option 2: Use the secret repository file:

# Example: pip config set global.index-url https://${USER_NAME}:${PASS_WORD}@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
pip config set global.index-url https://${USER_NAME}:${PASS_WORD}@<REPO_URL>

Configure Push Credentials

Enter the following commands in the Workspaces TERMINAL

twine

Option 1: Use built-in CNB_TOKEN environment variable:

export TWINE_USERNAME="cnb" 
export TWINE_PASSWORD="${CNB_TOKEN}"
# Artifact Registry address
# Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple 
export TWINE_REPOSITORY_URL=<REPO_URL>

Option 2: Use the secret repository file:

# Secret repository file USER_NAME configuration
export TWINE_USERNAME="${USER_NAME}" 
# Secret repository file PASS_WORD configuration
export TWINE_PASSWORD="${PASS_WORD}"
# Artifact Registry address
# Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple   
export TWINE_REPOSITORY_URL=<REPO_URL>

poetry

Option 1: Use the built-in CNB_TOKEN environment variable:

export POETRY_HTTP_BASIC_CNB_PYPI_USERNAME="cnb" 
export POETRY_HTTP_BASIC_CNB_PYPI_PASSWORD="${CNB_TOKEN}"
# Artifact Registry address
# Example: poetry config repositories.cnb-pypi https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple 
poetry config repositories.cnb-pypi <REPO_URL>

Option 2: Use the secret repository file:

# Secret repository file USER_NAME configuration
export POETRY_HTTP_BASIC_CNB_PYPI_USERNAME="${USER_NAME}"
# Secret repository file PASS_WORD configuration
export POETRY_HTTP_BASIC_CNB_PYPI_PASSWORD="${PASS_WORD}"
# Artifact Registry address 
# Example poetry config repositories.cnb-pypi https://pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple
poetry config repositories.cnb-pypi <REPO_URL>

Pull Artifacts

#Example 1: pip install requests
#Example 2: Pull a specific version pip install requests==2.30.1
pip install <PACKAGE_NAME>

# Example: Pull from a temporary source pip install -i https://cnb:${CNB_TOKEN}@pypi.cnb.build/cnb-demo/pypi-demo/-/packages/simple requests
pip install -i https://cnb:${CNB_TOKEN}@<REPO_URL> <PACKAGE_NAME>

Build and Push Artifacts

twine

python -m build  
twine upload -r cnb-pypi dist/*

poetry

poetry build
poetry publish -r cnb-pypi

A: The package name and version you are uploading already exist. Please set the Version Overwrite Strategy of your Artifact Registry to Allow overwriting of all existing versions, or change the current package version before uploading.

Q: Why do I get a 400 error when uploading?

A: This usually means your artifact filename does not comply with PyPI artifact specifications. Please check and fix it before uploading.

Q: What PyPI artifact formats does CNB PyPI Artifact Registry currently support?

A: We currently only support wheel and sdist package formats. Older formats like egg or exe will be rejected.

More Usage

For more PyPI usage, refer to the official documentation.

PyPI Artifact Registry

Prerequisites

Local Development

Configure Pull Credentials

Configure Push Credentials

Pull Artifacts

Build and Push Artifacts

Cloud Native Build

Pull Artifacts

Push Artifacts

Workspaces

Prerequisites

Configure Pull Credentials

Configure Push Credentials

Pull Artifacts

Build and Push Artifacts

FAQ

Q: Why do I get a 409 error when uploading?

Q: Why do I get a 400 error when uploading?

Q: What PyPI artifact formats does CNB PyPI Artifact Registry currently support?

More Usage