¶ 2026-02-20 Gitea Disk Pressure Cleanup and Runner Storage Guardrails
- Node
drg-k8s.auth.home.lan entered DiskPressure and tainted NoSchedule.
gitea and other workloads were repeatedly evicted.- Root cause was local disk growth under
/var/lib/gitea-act-runner/docker-lib-rootless (~33G).
gitea pods were evicted and unschedulable during pressure window.- CI runner pods were unstable.
- Scaled
gitea-act-runner deployment to 0 to stop churn.
- Cleared runner docker cache directory:
/var/lib/gitea-act-runner/docker-lib-rootless/*
- Confirmed node recovered:
DiskPressure=False- taint cleared
- Cleaned stale failed/evicted pods in affected namespaces.
- Applied runner guardrails:
docker-lib changed to emptyDir with sizeLimit: 12Gi- container resource requests/limits include
ephemeral-storage
dind: request 2Gi, limit 12Girunner: request 512Mi, limit 2Gi
- switched dind image to
docker:27-dind for compatibility with bounded emptyDir.
- Scaled
gitea-act-runner back to 1 and verified rollout.
/ filesystem reduced from ~83% to ~40% used.gitea pod running (1/1).gitea-act-runner pod running (2/2).- Node conditions:
Ready=True, DiskPressure=False.
- Move runner manifests to a clearly tracked repository path (single source of truth).
- Add periodic cleanup policy for CI images/build cache where feasible.
- Keep explicit
ephemeral-storage requests/limits on CI workloads.