Skip to main content

​
Overview

The Knowledge Stack workflow system lets you build automated ETL processes that use AI to extract, process, and output content. Each workflow is defined by four parameters — called ABCD — that control what data goes in, how it is processed, and where results are written.

​
ABCD Parameters

Every workflow is configured with these four parameters:

​
Parameter Rules

  • Each parameter’s items must be strictly disjoint — no item can appear in more than one group
  • No item can be an ancestor or descendant of another item within or across groups
  • You need read access for A, B, and D parameters, and write access for C

​
Creating a Workflow

​
Define a Workflow

Provide a name, description, runner configuration, and your ABCD parameters. The system validates all parameters when you save the definition.

​
Workflow Definitions API

​
Running a Workflow

​
Invoke a Workflow

When you invoke a workflow:
  1. The system validates all ABCD parameters (paths exist, permissions are correct, source folders contain documents)
  2. A workflow run is created with status PENDING
  3. A scoped JWT is minted for the runner
  4. The runner is notified to begin execution
Only one run can be active (RUNNING) per workflow definition at a time. Additional invocations are queued as PENDING and dispatched automatically when the active run completes.

​
Run Status

​
Workflow Runs API

​
Idempotency

You can include an Idempotency-Key header when invoking a workflow. If a run with the same key already exists in PENDING or RUNNING state, the API returns 200 OK with the existing run instead of creating a duplicate.

​
Self-Hosted Runners

For the current version, workflows use self-hosted runners. When you create a workflow definition, you configure a runner URL where the workflow engine sends execution requests.

​
How Runners Work

  1. Knowledge Stack POSTs the ABCD parameters and a scoped JWT to your runner URL
  2. Your runner processes the data according to the instructions
  3. When finished, your runner calls back to Knowledge Stack with the result

​
Runner Security

  • Scoped JWT: Runners receive a purpose-built token that is bound to the specific run, with a TTL matching the maximum run duration
  • SSRF prevention: Runner URLs are validated against blocked IP ranges (private networks, loopback). HTTPS is enforced in production.
  • Timeout protection: A background process monitors for stuck runs and automatically fails them after the configured timeout plus a grace period

​
Callback Endpoint

Your runner reports completion via:
The callback token is validated to ensure it matches the specific run.

​
Scheduling Workflows

Knowledge Stack does not include a built-in scheduler. You can trigger workflows on a schedule using any external scheduler (cron, cloud schedulers, CI/CD pipelines):

​
Authorization