Flink session jobs
Package and operate PyFlink and Java jobs with the Flink operator.
Purpose
Package and operate PyFlink and Java jobs with the Flink operator.
This guide is part of the RPi Kubernetes documentation set. The project is a hybrid k3s homelab with an Ubuntu control plane, four Raspberry Pi 5 workers, Cloudflare Tunnel, and a data platform made from Kafka, Flink, Redis Stack, MinIO, DataHub, Airbyte, Polaris, and observability services. Use this page when you need enough context to understand the design before changing code, manifests, configuration, or operational runbooks.
Where it fits
Repository: rpi_kubernetes
Audience: operators who want a small but real Kubernetes platform with enough services to learn production habits.
Main technologies: k3s, Kustomize, Helm, Strimzi Kafka, Flink Operator, Redis Stack, RAGFlow, DataHub, Airbyte, Polaris, MinIO, Prometheus, Grafana, Loki, OpenTelemetry, Cloudflare Tunnel, FastAPI, Next.js.
The implementation should be read as a set of boundaries rather than a pile of tools. Configuration declares what should exist, runtime services perform the work, persistence layers keep durable state, and observability explains what happened after the fact.
Implementation pattern
- Identify the durable resource: table, topic, deployment, agent spec, bot spec, manifest, or content page.
- Find the wrapper or runtime responsible for it instead of bypassing the project contract.
- Add configuration in the narrowest location that can express the new behavior.
- Add a smoke test or runbook step that proves the change can be recovered.
- Document the operational owner and the failure mode that matters most.
Operational checks
| Check | Why it matters |
|---|---|
| Configuration is explicit | Rebuilds and restarts should not depend on memory. |
| State has one owner | Competing writers make recovery hard. |
| Logs and progress are visible | Long-running workflows need user trust. |
| Backups or rebuild steps exist | A local-first system still needs disaster recovery. |
| Links to related docs are current | The portal is the public index for the workspace. |
Extension notes
When extending this area, prefer the project's existing factory, registry, wrapper, or runtime pattern. That keeps new work compatible with the surrounding code and avoids a second hidden implementation of the same idea.