Kubernetes Pod Security Policy

This story is about one of the many ways to secure kubernetes cluster. By default users and service accounts are authorized to create pods, which is good in the testing environment but in a production environment, it can cause huge problems. To restrict users and service accounts access, pod security will be used.

Overview

This section provides basic guidelines before hands-on with pod security policy on minikube cluster.

Pre-Requisites

Readers must understand these concepts to better understand Pod Security Policy. List of pre-requisites are given below:

• Minikube
• Namespace
• Pod
• Replica Set
• Service Account
• Cluster Role
• Cluster Role Binding

Pod Security Policy

Pod Security Policies (admission controller) enables authorization policies for pod creation and update, for users and service accounts. It is a cluster-level resource that controls security sensitive aspects of the pod specification.

Admission Controller

An admission controller intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized. It validates the requests for appropriateness like whether a controller is authorized to create pods in privileged mode to access to resources like hostNetwork etc.

Policy

It is a YAML file containing authorization rules for resources.

Policy Definition Guidelines

Policies are of two types restrictive(to restrict resources access) and permissive(to enable resources access).

Pod Security Policy should be implemented in this pattern:

• Create a restrictive policy for references/resources whose access should be restricted in the cluster.
• Create a permissive policy for reference(s)/resource(s) whose access needs to be enabled for the users or service accounts based on the requirements.

Example

This section will provide guidelines on how to enable and implement pod security policy.

1. Enabling Pod Security Policy

PSP is not enabled by default. PSP can be enabled by appending PodSecurityPolicy keyword in the --enable-admission-plugins element of spec.containers.command list, provided in /etc/kubernetes/manifests/kube-apiserver.yaml file.

Example

— enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodSecurityPolicy

Keypoint

Although Pod Security Policy is enabled in the cluster but without any policies it will cause pod creation(including re-creating pods from a scheduling event) to fail.

Testing

Use the deployment manifest given below to check whether a pod can be created with the absence of PSP policy.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: default
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.4

Use this command to create the deployment:

$ sudo kubectl deploy -f <file-name>.yaml

Use this command to check whether the pod is created or not

kubectl get pods,replicasets,deployments

Output

$ sudo kubectl get po,rs,deploy

State of the k8s cluster after the creation of nginx-deployment

It can be seen that deployment and replica sets controller is running but the pod is not created due to the absence of PSP policy.

2. Defining Policies

We will define two policies.

1. Restrictive policy

It is the default restrictive policy to restrict resources access:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restrictive
spec:
  privileged: false
  hostNetwork: false
  allowPrivilegeEscalation: false
  defaultAllowPrivilegeEscalation: false
  hostPID: false
  hostIPC: false
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - 'configMap'
  - 'downwardAPI'
  - 'emptyDir'
  - 'persistentVolumeClaim'
  - 'secret'
  - 'projected'
  allowedCapabilities:
  - '*'

2. Permissive Policy

It is a permissive policy that enables access to resources. Multiple permissive policies can be defined to enable access to multiple resources and later use that policy with service accounts based on the use case.

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: permissive
spec:
  privileged: true
  hostNetwork: true
  hostIPC: true
  hostPID: true
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  hostPorts:
  - min: 0
    max: 65535
  volumes:
  - '*'

Above two policies can be applied using the commands given below:

$ sudo kubectl apply -f <default-restrictive-policy>.yaml
$ sudo kubectl apply -f <permissive-policy>.yaml

To check whether policies are created or not.

$ sudo kubectl get psp

Kubernetes Pod Security Policies

To apply policies in our cluster we will create cluster role and cluster role binding.

Cluster Roles

Restrictive cluster role for the restrictive policy.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp-restrictive
rules:
- apiGroups:
  - extensions
  resources:
  - podsecuritypolicies
  resourceNames:
  - restrictive
  verbs:
  - use

Permissive cluster role for the permissive policy.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp-permissive
rules:
- apiGroups:
  - extensions
  resources:
  - podsecuritypolicies
  resourceNames:
  - permissive
  verbs:
  - use

Create the above cluster roles using the command given below:

$ sudo kubectl apply -f <cluster-role-for-restrictive-access>.yaml
$ sudo kubectl apply -f <cluster-role-for-permissive-access>.yaml

Cluster Role Bindings

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp-default
subjects:
- kind: Group
  name: system:serviceaccounts
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: psp-restrictive
  apiGroup: rbac.authorization.k8s.io

The above role binding will bind the restrictive cluster role with all system service accounts.

To apply this binding use the command given below:

$ sudo kubectl apply -f <restrictive-cluster-role-binding>.yaml

Testing

As the policies have been defined and attached with service accounts, we will try to recreate the previous deployment to check whether pods will be created or not.

First of all, delete the previous deployment:

$ sudo kubectl delete deploy nginx-deployment
$ sudo kubectl get po,rs,deploy

State of K8s cluster after the deletion of nginx-deployment

Recreate the deployment:

$ sudo kubectl apply -f nginx-deployment.yaml
$ sudo kubectl get po,rs,deploy

Keypoint

It can be seen that pod has been created because policies have been defined.

3. Breaking Policy Rules

In this section, we will create a deployment that will use a privileged resource “hostNetwork” that has been restricted by restrictive policy.

First of all, delete the previous deployment:

$ sudo kubectl delete deploy nginx-deployment
$ sudo kubectl get po,rs,deploy

State of K8s cluster after the deletion of nginx-deployment

Create a new deployment using the manifest and command given below:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-hostnetwork-deployment
  namespace: kube-system
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.4
      hostNetwork: true
$ sudo kubectl apply -f nginx-host-network-deployment.yaml
$ sudo kubectl get po,rs,deploy

State of K8s cluster after the creation of nginx-deployment

It can be seen that the replica set controller wasn’t able to create a new pod because:

Error creating: pods “nginx-hostnetwork-deployment-fd75d78b-” is forbidden: unable to validate against any pod security policy: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]

Keypoint

It means that when we tried to create a new deployment that tried to access the restricted resource, it wasn’t allowed to be created because of the restrictive policy.

4. Creating a deployment with privileged access

In this section, we will provide privileged rights to some controllers (daemon-set-controller, replicaset-controller, job-controller) by assigning permissive policy to controller’s service account.

First, we will create role binding

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: psp-permissive
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp-permissive
subjects:
- kind: ServiceAccount
  name: daemon-set-controller
  namespace: kube-system
- kind: ServiceAccount
  name: replicaset-controller
  namespace: kube-system
- kind: ServiceAccount
  name: job-controller
  namespace: kube-system

To apply the above role binding use the command given below:

$ sudo kubectl apply -f <permissive-role-binding>.yaml

We will recreate the deployment that will try to use the resource that is restricted by restrictive policy. The only difference between current and previous deployment is that it is being deployed in kube-system namespace instead of default namespace because we have applied permissive policy on the kube-system namespace.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-hostnetwork-deployment
  namespace: kube-system
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.4
      hostNetwork: true

To create the above deployment we will use the command given below:

$ sudo kubectl apply -f <nignx-deployment>.yaml

To check whether pods are created or not use the command given below:

$ sudo kubectl get po,rs,deploy -n kube-system

State of K8s cluster after the creation of nginx-deployment

Keypoint

It can been seen that nginx-hostnetwork-deployment pod has been deployed in the kube-system namespace becuase of the privilaged access given to controllers in the kube-system namespace.

Summary

Above tutorial depicts that Pod Security Policy is a way to secure the pod creation process in k8s cluster by using the PSP authorization policies.

References

https://octetz.com/posts/setting-up-psps