Pipeline runtime guide
Use Kedro-inspired DAGs and pluggable runners for assistant data processing.
Purpose
Use Kedro-inspired DAGs and pluggable runners for assistant data processing.
This guide is part of the Agentic Assistants documentation set. The project is a local-first assistant framework with a CLI, FastAPI and WebSocket server, MCP bridge, Next.js control panel, indexing, scoped retrieval, knowledge bases, pipelines, discovery, and training workflows. Use this page when you need enough context to understand the design before changing code, manifests, configuration, or operational runbooks.
Where it fits
Repository: agentic_assistants
Audience: developers who want private coding assistants, repository intelligence, local RAG, and repeatable assistant deployments.
Main technologies: Python, Poetry, Click, FastAPI, WebSockets, MCP, LanceDB, Chroma, DuckDB, Polars, PyArrow, CrewAI, LangChain, LangGraph, Ollama, MLflow, OpenTelemetry, Next.js, Docusaurus.
The implementation should be read as a set of boundaries rather than a pile of tools. Configuration declares what should exist, runtime services perform the work, persistence layers keep durable state, and observability explains what happened after the fact.
Implementation pattern
- Identify the durable resource: table, topic, deployment, agent spec, bot spec, manifest, or content page.
- Find the wrapper or runtime responsible for it instead of bypassing the project contract.
- Add configuration in the narrowest location that can express the new behavior.
- Add a smoke test or runbook step that proves the change can be recovered.
- Document the operational owner and the failure mode that matters most.
Operational checks
| Check | Why it matters |
|---|---|
| Configuration is explicit | Rebuilds and restarts should not depend on memory. |
| State has one owner | Competing writers make recovery hard. |
| Logs and progress are visible | Long-running workflows need user trust. |
| Backups or rebuild steps exist | A local-first system still needs disaster recovery. |
| Links to related docs are current | The portal is the public index for the workspace. |
Extension notes
When extending this area, prefer the project's existing factory, registry, wrapper, or runtime pattern. That keeps new work compatible with the surrounding code and avoids a second hidden implementation of the same idea.