Tag: Pod Security Standards

Applying Kubernetes Pod Security Standards: A 2026 Production Architecture Case Study

If you’re running Kubernetes at scale in 2026, enforcing Pod Security Standards (PSS) is essential—but making them work across multi-tenant clusters, legacy apps, and compliance audits is where teams struggle. This post details how a SaaS provider implemented, enforced, and audited PSS using production-grade YAML, CLI commands, exception protocols, and operational lessons learned. You’ll see what works, what doesn’t, and how to avoid common pitfalls with real-world Kubernetes security policy enforcement.

Key Takeaways:

• How to architect Pod Security Standards (PSS) enforcement for multi-team, multi-tenant clusters
• Production YAML and CLI for applying, auditing, and troubleshooting PSS in Kubernetes 1.29+
• Strategies for handling non-compliant legacy workloads without weakening your overall security posture
• Lessons learned from real compliance audits and incident response in 2026
• Comparison of PSS enforcement strategies for regulated, high-security, and mixed-trust environments

Architecture Overview: Security Policy Enforcement in Production

This case study centers on a SaaS provider with over 80 Kubernetes clusters (Kubernetes 1.27+), deployed across AWS, GCP, and Azure. Production clusters support:

• Critical SaaS application workloads (multi-region, HA)
• Shared and isolated namespaces for multiple engineering teams
• Platform services: CI/CD runners, log aggregation, monitoring exporters, and custom infrastructure controllers

Security and compliance teams required mandatory enforcement of the Kubernetes Pod Security Standards at the restricted level for all production namespaces. Only legacy or infrastructure workloads, after review, could use the baseline profile—never privileged in production. All exceptions were documented and reviewed quarterly.

The enforcement stack included:

• Native Pod Security Admission (PSA) for namespace-based policy
• GitOps automation (ArgoCD) for policy-as-code and daily drift reconciliation
• Custom admission webhooks for limited, documented exceptions
• Centralized SIEM/log aggregation for auditing and compliance reporting

This layered approach enforced strict policy, enabled developer agility in lower environments, and satisfied regulatory requirements. For details on integrating centralized logging, see our production log aggregation comparison.

Step-by-Step: Designing and Applying Pod Security Standards

Pod Security Standards Levels

• Privileged: Completely unrestricted. This policy is purposely open, allowing full host and kernel access. Only use for trusted, infrastructure-level workloads managed by cluster admins. Not recommended for regular application namespaces.
• Baseline: Minimally restrictive. Blocks known privilege escalations but allows most common workload configurations. Intended for legacy or non-critical workloads that cannot fully comply with hardening best practices.
• Restricted: Heavily restricted. Enforces industry best practices—no host networking, no privileged containers, restricted capabilities, seccomp required. Designed for production and sensitive workloads. (source)

For the most current requirements for each profile, refer to the official Kubernetes documentation.

Namespace Labeling for Policy Enforcement

1. Namespaces are labeled with the desired policy profile and mode (enforce, warn, or audit), plus the version to target.
2. The Pod Security Admission controller validates all pod creates/updates against these labels.

Example: Enforcing restricted policy in a production namespace

kubectl label namespace prod-app \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=latest

YAML for GitOps-managed namespace:

apiVersion: v1
kind: Namespace
metadata:
  name: prod-app
  labels:
    pod-security.kubernetes.io/enforce: "restricted"
    pod-security.kubernetes.io/enforce-version: "latest"

All namespace changes are submitted via Pull Request and reviewed for compliance, preventing accidental privilege escalation or label drift.

Testing Enforcement with PSA

Teams validate enforcement by attempting to deploy a pod that intentionally violates the restricted profile:

kubectl run test-pod --image=nginx \
  --overrides='{ "spec": { "hostPID": true } }' \
  --namespace=prod-app

Result (Kubernetes 1.29+ with restricted policy):

Error from server: pods "test-pod" is forbidden: violates PodSecurity "restricted:latest": hostPID != false (container "nginx" must not set hostPID=true)

This immediate feedback helps developers and platform teams see exactly what’s blocked by policy. For runtime protection, see our container security defense guide.

Managing Modes and Policy Versions

• enforce: Blocks non-compliant pods at admission
• warn: Logs violations, but does not block
• audit: Used for continuous monitoring only

Use enforce-version=latest unless you have workloads pinned to a specific Kubernetes release for compatibility. During major upgrades, start with warn mode cluster-wide, then promote to enforce after confirming compatibility.

Automating Policy with GitOps

1. Namespace creation is done via PR with required labels
2. Automated checks verify correct policy labeling and compliance
3. Security team reviews and merges; ArgoCD syncs to the cluster
4. Nightly reconciliation jobs compare live cluster labels to Git, alerting on drift

This workflow ensures all changes are tracked, reviewed, and auditable for compliance.

Handling Exceptions and Legacy Workloads

No large enterprise has 100% policy-compliant workloads from day one. Legacy applications, third-party images, or vendor agents often need exceptions. In this case, the SaaS provider established a multi-layer exception strategy that avoids overuse of permissive policies:

1. Segregated Namespaces with Lower Policies

• Legacy and infrastructure workloads are isolated in namespaces labeled with pod-security.kubernetes.io/enforce=baseline. The privileged profile is supported by the Pod Security Admission controller, but its use as an enforce label is not recommended in production and is intended only for system/infrastructure-level workloads under direct cluster admin control. (source)
• Each exception namespace is registered with the security team and reviewed quarterly.

kubectl label namespace monitoring-agents \
  pod-security.kubernetes.io/enforce=baseline \
  pod-security.kubernetes.io/enforce-version=latest

Exception namespaces are monitored to prevent unauthorized use for new application deployments.

2. Custom Admission Webhooks for Granular Exceptions

• A custom webhook validates pods against an approved exception list, including justification, owner, and expiration date.
• All exceptions are logged and reviewed by security/compliance staff regularly.
• Expired exceptions trigger alerts—no permanent exceptions without explicit high-level approval.

3. Automated Auditing and Drift Detection

• Nightly jobs compare live namespace labels and pod specs with the GitOps source of truth
• Detect and alert on:
  • Namespaces missing required policy labels
  • Pods running with forbidden fields (e.g., hostNetwork: true, privileged: true)
• Audit findings are tracked in compliance tickets with enforced remediation SLAs

4. Developer Self-Service with Guardrails

• Developers request exceptions via an automated portal tied to the GitOps repo
• Platform team can approve time-limited exceptions for dev/test; only security can approve production exceptions

Monitoring, Auditing, and Incident Response

Continuous monitoring and auditing ensure you catch violations before they become incidents or audit failures.

1. Real-Time Admission Event Streaming

• All denied pod admissions generate events sent to a central SIEM (Splunk, Elastic, or Loki)
• Security dashboards track policy violation trends and exception usage

2. Automated Compliance Reporting

• Nightly checks for policy drift:

kubectl get ns --show-labels | grep -v "enforce=restricted"

• Pod spec auditing for forbidden fields:

kubectl get pods --all-namespaces -o json | jq '.items[] | select(.spec.securityContext.privileged == true)'

• Findings are logged in compliance tickets and resolved within defined SLAs (typically 24–72 hours)

3. Incident Response Playbooks

• On violation, responsible teams are alerted (PagerDuty)
• Violating pods are quarantined via taint or network policy
• Security reviews, remediates, and determines if escalation is needed

Incident lessons feed back into automation and guardrail improvements.

4. Third-Party and Compliance Audit Support

• Policy enforcement and exceptions are exported monthly for external audit (SOC2, ISO 27001, etc.)
• Auditors receive read-only access to policy events and exception documentation

This workflow passed multiple external audits with zero major findings in 2026.

Pitfalls and Pro Tips from the Field

Common Pitfalls

• Developer friction: Rolling out enforce mode abruptly breaks CI/CD for teams not prepared for new policies. Start with warn mode and thorough developer training, then promote to enforce after validation.
• Exception sprawl: Overuse of baseline (or, worse, privileged) namespaces erodes your security baseline. Review exceptions at least quarterly and expire them by default.
• Platform agents and vendor tooling: Some vendor agents require elevated privileges. Where possible, work with vendors for compliant images or run them in tightly controlled exception namespaces.
• Manual changes outside GitOps: Labels set by hand are quickly forgotten, leading to policy drift. Reconcile cluster state against GitOps daily and alert on mismatches.
• Audit gaps: Not streaming admission events to SIEM leaves blind spots. Full visibility is non-negotiable for production and compliance.

Pro Tips

• Use warn and audit modes for early detection and developer education in pre-prod and dev clusters.
• Automate namespace creation with required policy labels—never allow unlabeled namespaces in production.
• Include policy violation and exception counts in Grafana or similar dashboards for security monitoring.
• Default to time-limited exceptions. Permanent exceptions require director/CISO approval and regular review.
• Pair Pod Security Standards with network policies and continuous image scanning. For details, see our container security guide.
• Continuously educate teams on common “gotchas”—e.g., why hostNetwork: true and privileged containers are rarely needed in modern apps.

Next Steps: Operationalizing Pod Security in Your Environment

Enforcing Kubernetes Pod Security Standards is a continuous, auditable process. Start with audit mode in dev, automate policy labeling, educate engineers, and enforce restricted in production. Make exceptions visible, time-limited, and subject to regular review. Integrate all policy events with central SIEM and compliance tooling.

For advanced multi-tenant patterns, see our DNS architecture case study. For a deep dive on runtime security, check out our container security strategy. For the latest on policy requirements, always refer to the official Kubernetes Pod Security Standards documentation. For comprehensive log and monitoring options, review our log aggregation comparison.

Pod security is a journey, not a checkbox. Make it real, make it visible, and make it reliable—and you’ll avoid the most expensive Kubernetes security failures.