cc74ad0bd0
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
- .woodpecker.yaml: image paths -> library/autojanet-{agent,dispatcher}
- .woodpecker.yaml: secret names RS_HARBOR_USER / RS_HARBOR_PASS (global)
- container/Dockerfile: restore COPY skills/, skills/ populated from opencode config
- skills/: 84 opencode skills bundled into image
- k8s/manifests: update image refs to library/
209 lines
5 KiB
Markdown
209 lines
5 KiB
Markdown
---
|
|
name: securing-k8s-service
|
|
description: Use when finishing a new Kubernetes service deployment or auditing an existing one for security hardening on homelab k3s or cloud EKS clusters.
|
|
---
|
|
|
|
# Securing a Kubernetes Service
|
|
|
|
## Overview
|
|
|
|
Work through each section in order. If you can only do some: follow the **priority order** at the bottom — the top items give the most security improvement per minute of effort.
|
|
|
|
---
|
|
|
|
## 1. ServiceAccount (RBAC)
|
|
|
|
Every app gets its own ServiceAccount. Never use `default`.
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: <app>
|
|
namespace: <app>
|
|
automountServiceAccountToken: false # disable unless app calls k8s API
|
|
```
|
|
|
|
If app needs k8s API access, create a minimal Role and bind it:
|
|
|
|
```yaml
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: Role
|
|
metadata:
|
|
name: <app>-role
|
|
namespace: <app>
|
|
rules:
|
|
- apiGroups: [""]
|
|
resources: ["configmaps"]
|
|
verbs: ["get", "list", "watch"]
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: RoleBinding
|
|
metadata:
|
|
name: <app>-rolebinding
|
|
namespace: <app>
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: <app>
|
|
roleRef:
|
|
kind: Role
|
|
name: <app>-role
|
|
apiGroup: rbac.authorization.k8s.io
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Pod Security Context
|
|
|
|
Set on `spec.template.spec` (pod-level) AND each container.
|
|
|
|
```yaml
|
|
# Pod-level (spec.template.spec.securityContext)
|
|
securityContext:
|
|
runAsNonRoot: true
|
|
runAsUser: 1000 # check image docs for correct UID
|
|
runAsGroup: 1000
|
|
fsGroup: 1000
|
|
seccompProfile:
|
|
type: RuntimeDefault
|
|
|
|
# Container-level (each container's securityContext)
|
|
securityContext:
|
|
allowPrivilegeEscalation: false
|
|
readOnlyRootFilesystem: true
|
|
capabilities:
|
|
drop: ["ALL"]
|
|
add: [] # only add if explicitly required (e.g., NET_BIND_SERVICE for :80)
|
|
```
|
|
|
|
`readOnlyRootFilesystem: true` — if the app writes to disk, mount writable paths via `emptyDir`:
|
|
|
|
```yaml
|
|
volumes:
|
|
- name: tmp
|
|
emptyDir: {}
|
|
- name: cache
|
|
emptyDir: {}
|
|
volumeMounts:
|
|
- name: tmp
|
|
mountPath: /tmp
|
|
- name: cache
|
|
mountPath: /var/cache
|
|
```
|
|
|
|
Common paths that need emptyDir: `/tmp`, `/var/cache`, `/var/run`, `/home/<user>/.config`
|
|
|
|
---
|
|
|
|
## 3. Resource Limits
|
|
|
|
Always set both requests AND limits.
|
|
|
|
```yaml
|
|
resources:
|
|
requests:
|
|
cpu: "100m"
|
|
memory: "128Mi"
|
|
limits:
|
|
cpu: "500m" # throttles (not kills) — set generously or omit if unsure
|
|
memory: "512Mi" # OOMKill if exceeded — set to ~2-3x steady-state usage
|
|
```
|
|
|
|
Check actual usage after deploy:
|
|
```bash
|
|
kubectl top pods -n <namespace>
|
|
```
|
|
|
|
---
|
|
|
|
## 4. NetworkPolicy (Cilium)
|
|
|
|
Default deny all, then add exceptions.
|
|
|
|
```yaml
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: <app>-netpol
|
|
namespace: <app>
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app.kubernetes.io/name: <app>
|
|
policyTypes: [Ingress, Egress]
|
|
ingress:
|
|
- from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: traefik
|
|
ports:
|
|
- port: <app-port>
|
|
egress:
|
|
# DNS — almost always needed
|
|
- to: []
|
|
ports:
|
|
- port: 53
|
|
protocol: UDP
|
|
# External HTTPS — add if app calls external APIs
|
|
- to: []
|
|
ports:
|
|
- port: 443
|
|
# Specific namespace (e.g., database)
|
|
- to:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: <db-namespace>
|
|
ports:
|
|
- port: 5432
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Secret Handling
|
|
|
|
- Never put secrets in `values.yaml` or ConfigMaps — use ExternalSecret → OpenBao
|
|
- Set `argocd.argoproj.io/sync-wave: "-1"` on ExternalSecrets so they exist before the Deployment
|
|
- Prefer `secretKeyRef` in env vars over mounted files (unless app requires file format)
|
|
|
|
---
|
|
|
|
## 6. Image Security
|
|
|
|
- Pin image tags: `image:v1.2.3` not `image:latest`
|
|
- Check registry.ctz.fyi Harbor scans for vulnerability reports
|
|
- For custom images: use minimal base (distroless, alpine, scratch)
|
|
- Set `imagePullPolicy: IfNotPresent` (not `Always`, unless actively testing)
|
|
|
|
---
|
|
|
|
## Quick Audit (Existing Deployments)
|
|
|
|
```bash
|
|
# Security contexts
|
|
kubectl get deploy <name> -n <ns> -o jsonpath='{.spec.template.spec.securityContext}'
|
|
kubectl get deploy <name> -n <ns> -o jsonpath='{.spec.template.spec.containers[0].securityContext}'
|
|
|
|
# Resource limits
|
|
kubectl get deploy <name> -n <ns> -o jsonpath='{.spec.template.spec.containers[0].resources}'
|
|
|
|
# ServiceAccount
|
|
kubectl get deploy <name> -n <ns> -o jsonpath='{.spec.template.spec.serviceAccountName}'
|
|
|
|
# Default SA automount (should be false or unset)
|
|
kubectl get sa default -n <ns> -o jsonpath='{.automountServiceAccountToken}'
|
|
|
|
# NetworkPolicies
|
|
kubectl get networkpolicy -n <ns>
|
|
```
|
|
|
|
---
|
|
|
|
## Priority Order
|
|
|
|
If you can only do some of these, start here:
|
|
|
|
1. **Non-root + no privilege escalation** — most impactful, easy to add
|
|
2. **Resource limits** — prevents noisy-neighbor and OOM cascade
|
|
3. **Dedicated ServiceAccount + no automount** — limits blast radius
|
|
4. **NetworkPolicy** — isolates a compromised pod
|
|
5. **readOnlyRootFilesystem** — hardens against post-compromise persistence
|