Kubernetes is an open-source container orchestration platform that enables the deployment and management of containerized applications at scale. As such, it provides many features for managing containerized workloads, including storage and security.
Kubernetes provides robust storage and security features that can help to manage containerized workloads in a cluster efficiently and securely.
Kubernetes Storage
In Kubernetes, storage is used to provide persistent data storage for containers running in a cluster. Kubernetes provides several types of storage options, including local storage, network-attached storage (NAS), and cloud storage. Storage volumes can be mounted to a pod and accessed from multiple containers or pods within the same or different nodes. Kubernetes also provides built-in support for storage orchestration, which automates the creation, deletion, and resizing of storage volumes, and dynamic provisioning, which allows for the automatic creation of new storage volumes when needed. This makes it easier to manage persistent data in a containerized environment.
Persistent Volume (PV)
Persistent Volumes (PVs) in Kubernetes are a way to provide persistent storage to containers running in a Kubernetes cluster. PVs are networked storage volumes that can be attached to a container running in a Kubernetes cluster. They provide durable storage and are not tied to any particular node in the cluster. PVs are managed by the Kubernetes control plane and can be shared between multiple containers.
PVs are created by the cluster administrator and are available to all users of the cluster. Users can claim a portion of a PV for their application by creating a Persistent Volume Claim (PVC). Once a PVC is created, the Kubernetes control plane will find a suitable PV to satisfy the request.
Example of Creating a Persistent Volume
To create a PV, you need to define a PV manifest file in YAML format. Here is an example of a PV manifest file:
apiVersion: v1
kind: PersistentVolume
metadata:
name: myebsvol
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
awsElasticBlockStore:
volumeID: # here put the volume id
fsType: ext4
To create the PV, save the manifest file to a file called pv.yaml, and then run the following command:
kubectl create -f pv.yaml
This will create the PV and make it available to the cluster.
Persistent Volume Claim (PVC)
A Persistent Volume Claim (PVC) is a Kubernetes resource that requests a certain amount of storage from a Persistent Volume (PV) resource. PVCs are used to provide a layer of abstraction between the storage requested by a pod and the storage actually provided by the underlying storage system. By using PVCs, a pod can request storage without having to know where that storage is coming from or how it is provisioned.
A PVC is created by a user of the Kubernetes cluster and specifies the desired storage capacity, access mode, and storage class. The Kubernetes control plane matches the PVC with an available PV that meets the PVC's requirements. Once the PVC is bound to a PV, the pod can use the PVC to access the storage.
Example of a Persistent Volume Claim
Here is an example of a PVC manifest file:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myebsvolclaim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
The following manifest file can be used to attach the aforementioned Persistent Volume to a pod.
apiVersion: apps/v1
kind: Deployment
metadata:
name: pvdeploy
spec:
replicas: 1
selector: # tells the controller which pods to watch/belong to
matchLabels:
app: mypv
template:
metadata:
labels:
app: mypv
spec:
containers:
- name: shell
image: centos
command: ["bin/bash", "-c", "sleep 10000"]
volumeMounts:
- name: mypd
mountPath: "/tmp/persistent"
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myebsvolclaim
Storage Class
In Kubernetes, a StorageClass is an abstraction layer that allows users to define different classes of storage with different performance characteristics and then request storage resources using those classes. A StorageClass is a way to provision storage dynamically and automatically, without requiring manual intervention from the cluster administrator.
When a user requests storage by creating a PersistentVolumeClaim (PVC), they can specify the desired StorageClass. The Kubernetes control plane uses the StorageClass to provision a matching PersistentVolume (PV) with the requested capacity and other characteristics.
Here's an example of a StorageClass manifest file:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
In this example, we're defining a StorageClass called "fast" that uses the "kubernetes.io/aws-ebs" provisioner to provision storage on Amazon Elastic Block Store (EBS). The type parameter specifies that the storage should use the gp2 volume type, which provides fast and consistent performance.
Save the StorageClass manifest file to a file called storageclass.yaml, and then run the following command to create the StorageClass:
kubectl apply -f storageclass.yaml
Once this StorageClass is defined, a user can create a PVC that requests storage from this StorageClass, like so:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: fast
In this example, the storageClassName field is set to "fast", which tells Kubernetes to use the StorageClass we defined earlier to provision the requested storage.
Using StorageClasses makes it easy to dynamically provision storage resources that meet different performance and capacity requirements, and allows users to easily request and use the appropriate storage resources
Save the PVC manifest file to a file called pvc.yaml, and then run the following command to create the PVC:
kubectl apply -f pvc.yaml
This will create the PVC and bind it to a suitable PV in the cluster.
Now you can create a deployment that mounts the PVC as a volume in a container. Here's an example deployment manifest file that uses the PVC we created earlier:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
volumeMounts:
- name: nginx-persistent-storage
mountPath: /usr/share/nginx/html
volumes:
- name: nginx-persistent-storage
persistentVolumeClaim:
claimName: my-pvc
In this example, we're creating a deployment that runs a single replica of the nginx container image. We're also specifying a volumeMount that mounts the PVC at /usr/share/nginx/html inside the container.
To create the deployment, save this manifest file to a file called deployment.yaml, and then run the following command:
kubectl apply -f deployment.yaml
This will create the deployment and start the container. You can verify that the container is running by running the following command:
kubectl get pods
Statefullset
In Kubernetes, a StatefulSet is a higher-level abstraction over a Deployment that is designed to manage stateful applications, such as databases or other clustered applications that require stable network identifiers and persistent storage. A StatefulSet provides guarantees around the ordering and uniqueness of pod creation, scaling, and deletion.
Let's walk through an example of how to create a StatefulSet in Kubernetes.
Assuming you have a Kubernetes cluster up and running, you can follow these steps:
- Create a ConfigMap
Before creating a StatefulSet, you may want to create a ConfigMap to store configuration data for your application. Here's an example ConfigMap manifest file:
apiVersion: v1
kind: ConfigMap
metadata:
name: myapp-config
data:
app.properties: |
database.host=mydb
database.port=5432
In this example, we're creating a ConfigMap called "myapp-config" that contains some configuration data in the app.properties key.
To create this ConfigMap, save the manifest file to a file called configmap.yaml, and then run the following command:
kubectl apply -f configmap.yaml
- Create a StatefulSet
Next, you can create a StatefulSet to manage your application. Here's an example StatefulSet manifest file:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: myapp
spec:
serviceName: myapp
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8080
envFrom:
- configMapRef:
name: myapp-config
volumeMounts:
- name: data
mountPath: /data
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: fast
resources:
requests:
storage: 1Gi
In this example, we're creating a StatefulSet called "myapp" that uses the ConfigMap we created earlier. The StatefulSet specifies a replica count of 3, and defines a volumeClaimTemplate that creates a persistent volume for each replica.
To create this StatefulSet, save the manifest file to a file called statefulset.yaml, and then run the following command:
kubectl apply -f statefulset.yaml
- Verify the StatefulSet
You can verify that the StatefulSet has been created and is running correctly by running the following commands:
kubectl get statefulsets
kubectl get pods
kubectl describe statefulset myapp
The get statefulsets command will show you the StatefulSet that was created, the get pods command will show you the pods that were created by the StatefulSet, and the describe statefulset myapp command will give you detailed information about the StatefulSet.
That's it! Now you have a StatefulSet that is managing your application and providing stable network identifiers and persistent storage.
Kubernetes Security
Kubernetes security involves a range of practices and configurations aimed at securing the Kubernetes environment and the applications running on it. Here are some key components of Kubernetes security:
Authentication and Authorization: Kubernetes supports various authentication mechanisms, such as client certificates, bearer tokens, and OpenID Connect tokens. Once authenticated, users are then authorized to access Kubernetes resources based on their RBAC (Role-Based Access Control) permissions.
Network Security: Kubernetes provides various network security mechanisms, such as network policies, pod security policies, and encryption of network traffic using TLS.
Secrets Management: Kubernetes provides a secrets API that allows sensitive information, such as passwords and access keys, to be stored securely and accessed only by authorized users and pods.
Container Image Security: Container images used in Kubernetes should be scanned for vulnerabilities and signed with cryptographic keys to ensure their integrity. Kubernetes also provides admission controllers to enforce image security policies.
Kubernetes API Security: The Kubernetes API can be secured using SSL/TLS, RBAC permissions, and API server authentication.
Logging and Monitoring: Kubernetes provides various logging and monitoring mechanisms to help detect and respond to security incidents, such as audit logging, Prometheus metrics, and Kubernetes events.
Compliance: Kubernetes supports compliance with various industry standards and regulations, such as HIPAA, PCI DSS, and GDPR.
Overall, Kubernetes security requires a comprehensive and layered approach, including best practices in network security, authentication and authorization, secrets management, container image security, Kubernetes API security, logging and monitoring, and compliance.
Role-Based Access Control (RBAC)
Role-Based Access Control (RBAC) is a method of restricting access to resources in a Kubernetes cluster based on roles assigned to individual users or groups. RBAC enables fine-grained control over what actions can be performed on Kubernetes resources, such as pods, services, and deployments.
Here's an example of how to use RBAC to control access to resources in a Kubernetes cluster:
- Create a Role
To create a role, you need to define a set of rules that specify what actions can be performed on which resources. Here's an example of a Role manifest file:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-reader
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
In this example, we're creating a role called "pod-reader" that grants read access to pods.
To create this role, save the manifest file to a file called role.yaml, and then run the following command:
kubectl apply -f role.yaml
- Create a RoleBinding
A RoleBinding is used to associate a role with a user or a group. Here's an example of a RoleBinding manifest file:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: pod-reader-binding
subjects:
- kind: User
name: alice
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
In this example, we're creating a RoleBinding called "pod-reader-binding" that associates the "pod-reader" role with the user "alice".
To create this RoleBinding, save the manifest file to a file called rolebinding.yaml, and then run the following command:
kubectl apply -f rolebinding.yaml
- Verify the Role and RoleBinding
You can verify that the Role and RoleBinding have been created correctly by running the following commands:
kubectl get roles
kubectl get rolebindings
kubectl describe role pod-reader
kubectl describe rolebinding pod-reader-binding
The get roles command will show you the roles that were created, the get rolebindings command will show you the rolebindings that were created, and the describe role and describe rolebinding commands will give you detailed information about the role and rolebinding.
That's it! Now the user "alice" has read access to pods in the Kubernetes cluster.
Pod Security Policies (PSPs)
Pod Security Policies (PSPs) is a Kubernetes feature that allows cluster administrators to define and enforce security policies for pods. PSPs help ensure that pods are created with a minimum set of security requirements, such as running as non-root users, using read-only file systems, and using specific security contexts.
Here's an example of how to use Pod Security Policies to enforce security policies for pods in a Kubernetes cluster:
- Create a Pod Security Policy
To create a Pod Security Policy, you need to define a set of rules that specify what security requirements pods must meet. Here's an example of a Pod Security Policy manifest file:
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
privileged: false
allowPrivilegeEscalation: false
seLinux:
rule: RunAsAny
runAsUser:
rule: MustRunAsNonRoot
fsGroup:
rule: MustRunAs
ranges:
- min: 1
max: 65535
volumes:
- configMap
- secret
- emptyDir
- downwardAPI
- projected
In this example, we're creating a Pod Security Policy called "restricted" that requires pods to run as non-root users, disallows privilege escalation, and allows only specific types of volumes to be used.
To create this policy, save the manifest file to a file called podsecuritypolicy.yaml, and then run the following command:
kubectl apply -f podsecuritypolicy.yaml
- Create a RoleBinding for the PSP
To associate the Pod Security Policy with a specific user or group, you need to create a RoleBinding. Here's an example of a RoleBinding manifest file:
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp-restricted
subjects:
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: PodSecurityPolicy
name: restricted
apiGroup: policy
In this example, we're creating a RoleBinding called "psp-restricted" that associates the "restricted" Pod Security Policy with the group "system:authenticated".
To create this RoleBinding, save the manifest file to a file called rolebinding.yaml, and then run the following command:
kubectl apply -f rolebinding.yaml
- Verify the Pod Security Policy
You can verify that the Pod Security Policy has been created correctly by running the following command:
kubectl get psp
This command will show you a list of Pod Security Policies in the cluster, including the "restricted" policy that we just created.
- Create a Pod that Meets the Policy
Now that we have a Pod Security Policy in place, let's create a pod that meets the policy requirements. Here's an example of a Pod manifest file that meets the "restricted" policy requirements:
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx:latest
securityContext:
runAsUser: 1000
runAsNonRoot: true
volumeMounts:
- name: nginx-config
mountPath: /etc/nginx/nginx.conf
readOnly: true
volumes:
- name: nginx-config
configMap:
name: nginx-config
In this example, we're creating a pod called "nginx" that runs as a non-root user with UID 1000,
Secrets
In Kubernetes, a secret is an object that contains sensitive information, such as passwords, API keys, or certificates. Secrets are stored in the Kubernetes API server and are used to pass sensitive information to a container running in a pod.
Secrets can be created and managed using the kubectl command-line tool or through manifest files.
Let's explore three types of secrets along with examples:
Generic Secrets:
Generic secrets are the most basic type in Kubernetes, allowing you to store key-value pairs.
Example: Let's say you have an application that requires a database password. You can create a generic secret named
db-secretwith the password as the value:
kubectl create secret generic mysecret --from-literal=username=Gopa123 --from-literal=password=MySecurePassword123
kubectl get secrets
Once the secret is created, you can use it in a pod by referencing it in the pod's manifest file. For example, here's a pod manifest file that uses the my-secret secret:
# if we want to access specific variable from secret then we can use.
apiVersion: v1
kind: Pod
metadata:
name: secret-demo-1
spec:
containers:
- name: demo-container
image: nginx
env:
- name: Username
valueFrom:
secretKeyRef:
name: db-secret
key: username
or
# if we want to access all variable from the secret then we can use.
apiVersion: v1
kind: Pod
metadata:
name: secret-demo-1
spec:
containers:
- name: demo-container
image: nginx
envFrom:
- secretRef:
name: docker-secret
kubectl apply -f my-secret-pod.yml
kubectl exec secret-demo-1 -it -- printenv
Docker-Registry Secrets:
Docker-registry secrets are used when pulling images from private Docker registries that require authentication.
Example: If your application uses a private Docker registry, you can create a Docker-registry secret named
docker-secretwith the registry credentials:
kubectl create secret docker-registry docker-secret --docker-email=example@gmail.com --docker-username=dev --docker-password=pass1234 --docker-server=my-registry.example:5000
apiVersion: v1
kind: Pod
metadata:
name: secret-demo-2
spec:
containers:
- name: demo-container
image: nginx
envFrom:
- secretRef:
name: docker-secret
kubectl apply -f <file-name.yaml>
kubectl exec secret-demo-2 -it -- printenv
TLS Secrets:
TLS secrets are used to secure communication over HTTPS or other TLS-enabled protocols.
Example: Suppose you want to secure communication for your application with an SSL/TLS certificate. You can create a TLS secret named
tls-secretby providing the certificate and key files:
kubectl create secret tls my-tls-secret --cert=/root/data/serverca.crt --key=/root/data/servercakey.pem
apiVersion: v1
kind: Pod
metadata:
name: secret-demo-3
spec:
containers:
- name: demo-container
image: nginx
volumeMounts:
- name: data
mountPath: /etc/cert-data
volumes:
- name: data
secret:
secretName: my-tls-secret
kubectl apply -f <file-name.yaml>
kubectl exec -it <pod-name> -- bash
cd /etc/cert-data
ls
Network policies
Here's an example manifest file to create two pods, one with the label app: web and another without the label:
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80
kubectl get svc,pod
kubectl run busybox --rm -ti --image=busybox:1.28 -- /bin/sh
wget --spider --timeout=1 nginx
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: access-nginx
spec:
podSelector:
matchLabels:
app: nginx
ingress:
- from:
- podSelector:
matchLabels:
access: "true"
kubectl apply -f network-policy.yaml
kubectl run busybox --rm -ti --image=busybox:1.28 -- /bin/sh
wget --spider --timeout=1 nginx
This creates a new Pod running the BusyBox image. Once inside the shell of the Pod, run the wget command to access the nginx service. Since the Pod does not have the required access: true label, the output should indicate a download timeout.
kubectl run busybox --rm -it --labels="access=true" --image=busybox:1.28 -- /bin/sh
wget --spider --timeout=1 nginx
This creates a new Pod running the BusyBox image with the required access: true label, and tests the connection to the nginx service again. The output should confirm a successful connection to the service.
TLS(Transport Layer Security)
Transport Layer Security (TLS) is a protocol used to secure communication over a network. It is commonly used to secure web traffic, email, and other types of network communication. In Kubernetes, TLS can be used to secure communication between clients and servers, as well as between different components of a cluster.
Here's an example of how to use TLS in Kubernetes:
- Create a TLS Certificate
The first step in using TLS in Kubernetes is to create a TLS certificate. You can use the openssl command to create a self-signed certificate for testing purposes. For example, to create a certificate with a key length of 2048 bits, you can run the following command:
kubectl create secret tls tls-secret --key tls.key --cert tls.crt
- Use the Kubernetes Secret
Once you have created the Kubernetes secret, you can use it in your Kubernetes manifests to enable TLS. For example, here's a sample deployment manifest that uses the tls-secret to secure communication between the container and the client:
apiVersion: apps/v1
kind: Deployment
metadata:
name: tls-deployment
spec:
replicas: 1
selector:
matchLabels:
app: tls
template:
metadata:
labels:
app: tls
spec:
containers:
- name: tls-container
image: nginx
ports:
- containerPort: 443
volumeMounts:
- name: tls-secret
mountPath: /etc/tls
readOnly: true
volumes:
- name: tls-secret
secret:
secretName: tls-secret
In this example, we're creating a deployment with one replica and a container named tls-container. We're using the nginx image as an example. We're also creating a volume named tls-secret and mounting it to the container at the path /etc/tls. This will make the certificate and key available to the container.
- Test the TLS Connection
To test the TLS connection, you can use the curl command to send a request to the container. For example, to send a request to the container at the IP address 10.0.0.1 over HTTPS, you can run the following command:
curl --cacert tls.crt https://10.0.0.1
In this example, we're using the --cacert option to specify the certificate authority file, which is the same as the certificate file in our case since it's self-signed. The https:// prefix is used to indicate that we want to use HTTPS, which is the protocol that uses TLS to secure the communication.
Overall, using TLS in Kubernetes is an important security measure to protect communication between different components of a cluster, and between clients and servers in the cluster.