knowledge:update
About 1313 wordsAbout 4 min
knowledge:update
Update knowledge base, used to build Issues, repository documents, etc. into the knowledge base.
When to use it: use it to build a knowledge base from your repository content for retrieval or for RAG (Retrieval-Augmented Generation) applications - for example splitting and vectorizing repository documents and Issues into the knowledge base, syncing automatically as code is pushed or Issues change. Chunking, vectorizing and incremental syncing are costly to script, so knowledge:update folds this into a single built-in task.
Applicable Events
Different events take different execution paths, see How it works.
How it works
After the task is triggered, it first chooses an execution path based on the event type:
| Event | Path | Description |
|---|---|---|
issue.* (Issue events) | In-place update | No Runner or plugin image required, calls platform APIs to sync Issues incrementally |
Other events (e.g. push, tag_push) | Plugin build | Starts the cnbcool/knowledge-base plugin image and builds repository documents from the Git commit diff |
The common flow of both paths is:
Fetch content -> chunking -> vectorization -> into knowledge base
Issues are converted to Markdown in the following structure before being indexed: basic info (state, author, assignees, priority, labels, created/updated time), description, and all comments (including author and time).
Validation and rebuild
On every run, the task compares the current options with the existing knowledge base configuration. If any of the following changes, a rebuild is required: the knowledge base is deleted and recreated, then a full build is performed.
includeorexcludechangedembeddingModelchangedchunkSizeorchunkOverlapchangedissueSyncEnabledchangedissueState,issueLabels, orissueExcludeLabelschanged- The knowledge base has no build record yet (both commit and Issue sync records are empty)
forceRebuildistrue
Tip
A rebuild wipes the existing knowledge base and rebuilds everything from scratch, so it takes longer. Day-to-day Issue changes go through incremental update instead.
Incremental update
When triggered by issue.* events (such as opening an Issue or adding a comment), the task updates the knowledge base incrementally rather than rebuilding it:
- Based on the last sync time recorded in the knowledge base, only Issues updated after that time are fetched
- Before writing, each Issue's old document is deleted and the latest content is written, so additions, edits, and deletions of the title/description/comments are all synced
- After processing, the last sync time is advanced, so the next run only syncs new changes
The first run, or a rebuild, falls back to a full sync.
Cleanup when filters change
In incremental mode, if issueState, issueLabels, or issueExcludeLabels is configured, the task also compares filters: Issue documents that no longer match the filters are deleted from the knowledge base so they do not remain stale.
Sync time and failure handling
The last sync time is advanced only when:
- All Issues are processed successfully
- Some Issues fail but
ignoreProcessFailures: trueis set (the failure list is printed in the pipeline log)
If some Issues fail and failures are not ignored, the task fails and the sync time is not advanced, so the same batch is retried on the next run. Issues without permission are skipped (and logged), which does not block the sync time from advancing.
Parameters
The list below groups the parameters by purpose (documents / issues) for a quick overview.
- include: Files to include in the knowledge base, matched by
Glob - exclude: Files to exclude, matched by
Glob - chunkSize: Text chunk size
- chunkOverlap: Overlapping tokens between adjacent chunks
- embeddingModel: Embedding model (only
hunyuanis supported) - issueSyncEnabled: Whether to sync issues into the knowledge base
- issueState: Sync only issues of a given state (
open/closed) - issueLabels: Sync only issues with given labels
- issueExcludeLabels: Exclude issues with given labels (takes precedence)
- forceRebuild: Whether to drop the knowledge base and rebuild it
- ignoreProcessFailures: Update the knowledge base even if some files fail to process
Detailed descriptions for each parameter are provided below.
include
- type:
String|Array<String> - required:
false - default:
**/**.md
Specify which files to include, using Glob pattern matching. Defaults to **/**.md (all Markdown files). Supports arrays or comma-separated values. Supported file types: .md, .mdx, .docx, .txt, .pdf.
exclude
- type:
String|Array<String> - required:
false
Specify files to exclude using Glob patterns. By default, nothing is excluded. Supports arrays or comma-separated values.
chunkSize
- type:
Number - required:
false - default:
1500
Specify the text chunk size.
chunkOverlap
- type:
Number - required:
false - default:
0
Specify the overlapping token count between two adjacent chunks.
embeddingModel
- type:
String - required:
false - default:
hunyuan
Embedding model. Currently, only hunyuan is supported.
issueSyncEnabled
- type:
Boolean - required:
false - default:
true
Whether to enable Issue sync. If enabled, Issues from the repository will be automatically pulled and added to the knowledge base.
issueState
- type:
String - required:
false
Only sync Issues with the specified state. Defaults to all. Optional values: open | closed.
issueLabels
- type:
String|Array<String> - required:
false
Only sync Issues with the specified labels. Defaults to all. Supports arrays or comma-separated values.
issueExcludeLabels
- type:
String|Array<String> - required:
false
Exclude Issues with the specified labels. By default, no Issues are excluded. Supports arrays or comma-separated values. When overlapping with issueLabels, exclude takes priority.
forceRebuild
- type:
Boolean - required:
false - default:
false
Whether to delete and rebuild the knowledge base. When set to true, the knowledge base is deleted and created afresh.
ignoreProcessFailures
- type:
Boolean - required:
false - default:
false
Whether to ignore document processing failures. If set to true, the knowledge base will update even if some documents failed to process.
Configuration Examples
Rebuild the knowledge base on push to main
Build repository documents from the Git commit diff.
main:
push:
- stages:
- name: build knowledge base
type: knowledge:update
options:
include: "**/**.md"Append an Issue on creation
Issue event pipelines must be anchored under $. The knowledge base is updated as soon as a new issue is created in the repository.
$:
issue.open:
- stages:
- name: build knowledge base
type: knowledge:update
options:
# Only sync issues in the open state
issueState: openSync an Issue on update
Triggered when an issue's title, description, assignee, labels or priority changes.
$:
issue.update:
- stages:
- name: build knowledge base
type: knowledge:update
options:
# Only sync issues labeled "docs", excluding those labeled "no-index"
issueLabels:
- docs
issueExcludeLabels:
- no-indexBuild documents only on Issue close
If closed issues should no longer stay in the knowledge base, rebuild only the repository documents on close.
$:
issue.close:
- stages:
- name: build knowledge base
type: knowledge:update
options:
# Disable issue sync and build repository documents only
issueSyncEnabled: false