Skip to main content

Kubernetes Mutating Webhook

⬅️ Back to Features Catalog

What It Does​

The proxy includes a Kubernetes Mutating Webhook endpoint. When registered in a cluster, this webhook automatically injects the LLM-Shield-Proxy container as a sidecar into any Pod matching specific labels.

How It Works​

For teams managing hundreds of deployments, manually editing YAML manifests to add the proxy sidecar is error-prone. The admission webhook automates this.

  1. Admission Interception: When a developer deploys a Pod, the Kubernetes API Server intercepts the request and sends the manifest to the proxy's webhook endpoint (/v1/k8s/mutate).
  2. Label Trigger: The proxy checks if the Pod has the label llm-shield.io/inject: "true".
  3. Sidecar Patch: If the label is present (and the proxy isn't already there), the webhook returns a JSON Patch that appends the llm-shield-proxy container to the Pod specification.
  4. Result: Kubernetes applies the patch and starts the Pod with both the application and the proxy sidecar running together.

Performance Profile​

  • Overhead: The API Server must wait for the webhook to respond before persisting the Pod, adding a few milliseconds to deployment times. It has no impact on application runtime traffic.

Configuration Flags​

Environment VariableDescription
ENABLE_K8S_WEBHOOKServe /v1/k8s/mutate. Off by default; the Helm chart in deploy/helm sets it when webhook.enabled is true.
K8S_SIDECAR_IMAGEThe proxy container image digest to inject.
K8S_SIDECAR_SECRET_NAMESecret each injected sidecar loads its environment from (envFrom): VALID_VIRTUAL_KEYS and a provider key. A pod's llm-shield.io/keys-secret annotation overrides it.
K8S_WEBHOOK_AUTH_TOKENOptional bearer token to authenticate incoming admission requests from the API Server.

Implementation Details & Edge Cases​

  • No Automatic Routing: The webhook only injects the proxy container. It does not automatically rewrite the application's environment variables (e.g., OPENAI_BASE_URL) or configure iptables rules. You must still configure your application to send traffic to localhost:8000.
  • Keys for the sidecar: The injected container loads its keys from the Secret named by K8S_SIDECAR_SECRET_NAME or the pod's llm-shield.io/keys-secret annotation. With neither, it rejects every request with a 401, and the webhook logs a warning. The Secret must exist in the pod's namespace.
  • Idempotency: If the Pod already contains a container named llm-shield-proxy, the webhook skips injection to prevent conflicts.

FAQ​

Q: Do I have to use this webhook feature? A: No. You can manage sidecar injection manually via Helm, Kustomize, or Terraform, or rely on a standard service mesh (like Istio) instead.

Practical Effect​

This feature automates the placement of the proxy sidecar across a large fleet of Kubernetes workloads based on simple namespace or pod labels.

Tests: tests/test_k8s_webhook.py.