Image Source: Unsplash
In this blog, we will go step by step on how to create a custom kubeconfig file with limited access to kubernetes cluster using service account, secret token and RBAC
These are the mulitple ways to authenticate with Kubernetes:
- X509 Client Certificates
- Bearer Tokens
- Authentication Proxy
- HTTP Basic Authentication
- OpenID Connect Tokens
- Service Account Tokens - We will be focusing on this method
- Webhook Token Authentication
Note: For this demo I am using AWS EKS cluster, but the same applies to Kubernetes running anywhere. Moreover, this lab assumes that you have access to create, manage and view ClusterRole, Role, ClusterRoleBinding, RoleBinding, ServiceAccount and Secret API resources.
Let's start with creating custom RBAC role for our users.
There are two categories of it:
- ClusterRole
- Role
ClusterRole API resource as the name says is used to provide access at Cluster level where as with Role you can restrict access at namespace level.
We will create both ClusterRole and Role in this demo for our users Adam and Bill.
Adam is our Senior DevOps Engineer and is working on mulitple projects. Hence, will create ClusterRole for him which will not restricting his permission at namespace level but at resource level.
Bill is a Junior DevOps Engineer whi has recently joined the team and is working on Brown Fox project hence, will create a Role for him to restrict his permission both at namespace and resource level.
ClusterRole for Adam:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: adam
rules:
- apiGroups: [""]
resources: ["pods", "services", "namespaces", "nodes"]
verbs: ["create", "get", "update", "list", "watch", "patch"]
- apiGroups: ["apps"]
resources: ["deployment"]
verbs: ["create", "get", "update", "list", "delete", "watch", "patch"]
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["clusterroles", "clusterrolebindings"]
verbs: ["create", "get", "list", "watch"]
Role for Bill:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: bill
namespace: brown-fox
rules:
- apiGroups: [""]
resources: ["pods", "services", "namespaces", "nodes"]
verbs: ["create", "get", "update", "list", "watch", "patch"]
- apiGroups: ["apps"]
resources: ["deployment"]
verbs: ["create", "get", "update", "list", "delete", "watch", "patch"]
- apiGroups: ["rbac.authorization.k8s.io"]
resources: ["clusterroles", "clusterrolebindings"]
verbs: ["create", "get", "list", "watch"]
Let us now bind ClusterRole to a Adam using ClusterRoleBinding API resource.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: adam
subjects:
- kind: ServiceAccount
name: adam-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: adam
apiGroup: rbac.authorization.k8s.io
For Bill, we will be using RoleBinding API resource.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: bill
namespace: brown-fox
subjects:
- kind: ServiceAccount
name: bill-sa
namespace: brown-fox
roleRef:
kind: Role
name: bill
apiGroup: rbac.authorization.k8s.io
We are now required to create Service Account for both Adam and Bill which are attached to their roles we create above.
Service Account for Adam:
apiVersion: v1
kind: ServiceAccount
metadata:
name: adam-sa
namespace: kube-system
secrets:
- name: adam-secret
Service Account for Bill:
apiVersion: v1
kind: ServiceAccount
metadata:
name: bill-sa
namespace: brown-fox
secrets:
- name: bill-secret
Now we need to create tokens using Secret API resource which will be used in our kubeconfig file.
Secret for Adam:
apiVersion: v1
kind: Secret
metadata:
name: adam-secret
namespace: kube-system
annotations:
kubernetes.io/service-account.name: adam-sa
type: kubernetes.io/service-account-token
Secret for Bill:
apiVersion: v1
kind: Secret
metadata:
name: bill-secret
namespace: brown-fox
annotations:
kubernetes.io/service-account.name: bill-sa
type: kubernetes.io/service-account-token
Finally, let us generate kubeconfig file for Adam and Bill.
Sample File:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: {cluster-ca}
server: {server-dns}
name: {cluster-name}
contexts:
- context:
cluster: {cluster-name}
user: {user-name}
name: {context-name}
current-context: {context-name}
kind: Config
users:
- name: {user-name}
user:
token: {secret-token}
We need to replace all the values within curly braces with their actual values, so let's keep going.
Type the below command to fetch Certificate and Server information:
kubectl config view --flatten --minify
Copy certificate-authority-data, server and name field from the output and replace them with their respective fields in the sample file.
It's now time to grab the secret token. Use the following command:
# Fetching Adam's secret
kubectl get secrets -n kube-system
# Copy the token name starting adam-secret
kubectl describe secrets {token-name} -n kube-system
# Fetching Bill's secret
kubectl get secrets -n brown-fox
# Copy the token name starting bill-secret
kubectl describe secrets {token-name} -n brown-fox
Copy the token value and paste it in sample file. The kubeconfig file is now complete and should look like this:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0...............Sk1TTG5Ndk1yNmYzdFNl
server: https://01F4..............BEA3.sk1.ap-south-1.eks.amazonaws.com
name: arn:aws:eks:ap-south-1:006378141167:cluster/brown-fox-cluster
contexts:
- context:
cluster: arn:aws:eks:ap-south-1:006378141167:cluster/brown-fox-cluster
user: adam
name: arn:aws:eks:ap-south-1:006378141167:cluster/brown-fox-cluster
current-context: arn:aws:eks:ap-south-1:006378141167:cluster/brown-fox-cluster
kind: Config
users:
- name: adam
user:
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJ...............SNFZmRdIfU
Note: You will have to generate a separate kubeconfing file for Bill using his secret.
Note: The actual values are truncated for security reasons.
To test custom kubeconfig file type the following command:
kubectl get pods --kubeconfig {kubeconfig_file}
You must receive output with Adam's kubeconfig file but not with Bill's kubeconfig file.
Bingo! You have created kubeconfig file for Adam and Bill with limited access to Kubernetes cluster.