Here is a graph of the RBAC setup we are going to implement:

1. Setup keycloak

We’ll use keycloak to proxy our authentication for all monitors, using a single realm. You may use several realms in real-life situations. This is probably the tough part, and you may tweak heavily the following guide. Moreover, I may forgot to write some instructions, or somes are heavily linked to your very own setup.

1.1. Install keycloak

Start by installing Keycloak via the  Helm chart & expose it, using following templates:

•  kubernetes/authentication/01-KeycloakChartValues.yaml
•  kubernetes/authentication/02-PublicRoute.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

extraEnv: |
  - name: PROXY_ADDRESS_FORWARDING
    value: "true"
  - name: KEYCLOAK_USER
    value: {{keycloak.adminUser}}
  - name: KEYCLOAK_PASSWORD
    value: {{keycloak.adminPassword}}  
podLabels:
  app: keycloak
  component: keycloak

service:
  labels:
    app: keycloak
    component: keycloak

  httpsPort: 443 # 8443 by default, but it should be reachable via the same URL from outside than inside, eg `https://keycloak.{{cluster.baseHostName}}`
postgresql:
  enabled: false
  #storageClass: some-storage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: public-route
  namespace: keycloak
spec:
  entryPoints:
    - websecure
  routes:
  - match: Host(`keycloak.{{cluster.baseHostName}}`)
    kind: Rule
    services:
    - name: keycloak-http
      kind: Service
      namespace: keycloak
      port: 80
  tls:
    certResolver: myresolver

Note that the template  kubernetes/authentication/01-KeycloakChartValues.yaml contains the bare minimum settings for the keycloak chart with your route. You should tweak them to match your setup & security requirements.  RTFM the chart

1
2
3
4
5
6

# Add the keycloak chart repository
helm repo add codecentric https://codecentric.github.io/helm-charts
# Install keycloak in its dedicated namespace
helm install keycloak -f ./kubernetes/authentication/01-KeycloakChartValues.yaml codecentric/keycloak --version 9.9.2 --namespace keycloak --create-namespace
# Initialize traefik routing to it.
kubectl apply -f ./kubernetes/authentication/02-PublicRoute.yaml

You can remove this chart by running helm uninstall keycloak -n keycloak, or by deleting namespace keycloak altogether.

This install should end by displaying either the default keycloak user’s password, or the command to get it.

1.2. Alias hostname of keycloak from the cluster

Note : This part is not required if you use a real host name registered with a A record. But if you use a fake hostname (which I still recommend at that point), DNS resolution of services trying to reach keycloak will fail, because your pods don’t know where is https://keycloak.{{cluster.baseHostName}}.

We are going to alias the hostname keycloak.{{cluster.baseHostName}} so that it resolves to the internal name of our keycloak service, keycloak-http.keycloak.svc.cluster.local. For this, you need to edit CoreDNS configuration.

1
2

# Open the CoreDNS config map
kubectl edit configmap coredns -n kube-system

Add the following into your Corefile section:

1

rewrite name keycloak.{{cluster.baseHostName}} keycloak-http.keycloak.svc.cluster.local

I added it just above the ready block (I don’t know if it really matters).

Then, restart CoreDNS pods by deleting them.

1

kubectl -n kube-system delete pods $(kubectl -n kube-system get pods -l k8s-app=kube-dns -o json | jq '.items[].metadata.name' -r)

This may take some time and crash kubectl connection to the API server. Don’t worry, it will reschedule our pods and restart everything after some time.

So, now, we can use keycloak.{{cluster.baseHostName}} as a domain name everywhere we want, and it will resolve to the cluster’s internal IP of keycloak’s service.

1.3. Open keycloak admin panel

Log in to your keycloak admin dashboard by reaching https://keycloak.{{cluster.baseHostName}}/auth/admin/.

• Username: {{keycloak.adminUser}}
• Password: {{keycloak.adminPassword}}

1.4. Configure keycloak

1.4.1. Create groups

To manage our groups, go to Master realm > Manage > Groups section > New button.

Create all the groups you need. For the setup I describe, I have to create groups watchdogs and admins.

1.4.2. Create users

To manage our users, go to Master realm > Manage > Users section > Add user button. Fill the form with at least a username. I usually check email verified just in case because I trust the emails I put in.

If you want to allow this user to log in via username/password, set its password in the Credentials tab. If the Temporary option is set, the user will have to change its password on first login.

Assign your new user to the groups you want by going to the Groups tab.

Go on and create all the users you need, and assign relevant groups to them.

1.4.3. Setup clients & scopes

ℹ️ Info: If you’re not familiar with oauth2, clients are roughly applications that are allowed to authenticate users in your authentication system (keycloak). Clients can ask for grants of scopes, that are user informations they want to access to.

Back in the dashboards setup, we’ll protect our apps Test app, kibana, kube dashboard & traefik using  gogatekeeper/gatekeeper.

1.4.3.1. Create scopes

Since our apps Test app, kibana, kube dashboard & traefik will all rely on the same information (group), we can get our setup easier by adding a shared Client scope. Go to Master realm > Configure > Client Scopes section. Here, check if there is a client scope named groups (I didn’t, but just in case).

If not, create one named groups, then save it. If there is one, check if the following config matches or create a new one.

After, go to the new Mappers tab & create a new mapper. This mapper will put the groups we set up earlier in our user’s token. Set its Mapper Type to Group Membership, & the Token Claim Name to groups. This is the name of the token’s field we’ll use later in our authentication proxy.

Once done, you may add it to the default client scopes. Go to Master realm > Configure > Client Scopes section > Default Client Scopes tab, & add our groups to the assigned column. You should probably not set it as Optional.

1.4.3.2. Create client

We are going to create our authentication for the app {{nginxTest.clientId}} (for instance, nginx-test), with url https://test.{{cluster.baseHostName}}). Fill the Client ID with {{nginxTest.clientId}}.

Since this application will be logged in through a proxy (we’ll get back to that part next in the dashboard setups), set its Access Type to confidential.

Set the valid redirect URIs to https://test.{{cluster.baseHostName}}/oauth/callback (the /oauth/callback part is required by the gatekeeper).

Then, we need to add an audience. This field is required by gatekeeper to check our key. Go to the Mappers tab, & create a new mapper. Name it, for instance, audience, of Mapper Type Audience, & set the Included Client Audience to our {{nginxTest.clientId}}. Save it.

After this, check our client scopes by going to the Client Scopes tab. In the Setup sub-tab, make sure that our groups client scope is assigned. Then, you can test our setup ! Go to the Evaluate sub-tab, pick the User you want to check, click Evaluate then go to Generated Access Token sub-sub-tab. It should contain a key groups with all our user’s groups, prefixed with a /, and a key aud with at least our {{nginxTest.clientId}}. If its okay, we are good to go !

And, finally, go to the Credentials tab & get the secret. It will be used as {{nginxTest.clientSecret}}.

Other clients protected by our gatekeeper will be very similar.

2. Setup our test application protected by authentication

We’ll start with the simplest of our cases: the nginx-test app. This will allow us to get used to keycloak for our authorization mechanism. We’ll proxy a simple nginx default instance behind our authentication proxy.

Look at the  kubernetes/11-NginxTest.yaml template.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160

apiVersion: v1
kind: Namespace
metadata:
  name: nginx-test

# Nginx itself
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx
  namespace: nginx-test
  labels:
    app: nginx-test
    component: nginx

spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-test
      component: nginx

  template:
    metadata:
      labels:
        app: nginx-test
        component: nginx

    spec:
      containers:
        - name: nginx
          image: nginx
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: nginx-test
  labels:
    app: nginx-test
    component: nginx

spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
  selector:
    app: nginx-test
    component: nginx




# Authentication proxy
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gatekeeper
  namespace: nginx-test
  labels: 
    app: nginx-test
    component: gatekeeper

spec:
  replicas: 1
  selector:
    matchLabels: 
      app: nginx-test
      component: gatekeeper

  template:
    metadata:
      labels: 
        app: nginx-test
        component: gatekeeper

    spec:
      containers:
        - name: keycloak-gatekeeper
          image: "quay.io/gogatekeeper/gatekeeper:1.2.0"
          imagePullPolicy: IfNotPresent
          args:
            - --listen=0.0.0.0:3000
            - --discovery-url=https://keycloak.{{cluster.baseHostName}}/auth/realms/master
            - --client-id={{nginxTest.clientId}}
            - --client-secret={{nginxTest.clientSecret}}
            - --upstream-url=http://nginx.nginx-test.svc.cluster.local:80 # See https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#namespaces-and-dns
            - --redirection-url=https://test.{{cluster.baseHostName}}/
            - --skip-openid-provider-tls-verify=true
            - --enable-default-deny=true
            - --enable-logging=true
            - --enable-refresh-tokens=true
            - --enable-session-cookies=true
            - --encryption-key={{random32charsString}}
            - --secure-cookie=false
            - --resources=uri=/*|groups=/watchdogs
          ports:
            - name: http
              containerPort: 3000
              protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: gatekeeper
  namespace: nginx-test
  labels:
      app: nginx-test
      component: gatekeeper

spec:
  ports:
    - port: 80
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app: nginx-test
    component: gatekeeper




# ---
# apiVersion: traefik.containo.us/v1alpha1
# kind: Middleware
# metadata:
#   name: gatekeeper
#   namespace: nginx-test
# spec:
#   forwardAuth:
#     address: http://gatekeeper.nginx-test.svc.cluster.local:80
#     authResponseHeaders: 
#         - "X-Forwarded-User"
#     trustForwardHeader: true
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingress
  namespace: nginx-test
spec:
  entryPoints:
    - websecure
  routes:
  - match: Host(`test.{{cluster.baseHostName}}`)
    kind: Rule
    services:
    - name: gatekeeper
      namespace: nginx-test
      kind: Service
      port: 80
    # middlewares:
    #   - name: gatekeeper
    #     namespace: nginx-test
  tls:
    certResolver: myresolver

As you may notice in this template, the option --discovery-url=https://keycloak.{{cluster.baseHostName}}/auth/realms/master tells our authentication proxy where is our OAuth2 provider. If you are using an aliased hostname, this won’t resolve, as your server can’t resolve an IP for this hostname. That’s why we edited CoreDNS configuration earlier.

So let’s deploy that:

1

kubectl apply -f 11-NginxTest.yaml

Gen random 32 chars str: $(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1)

Now, you should be able to access https://test.{{cluster.baseHostName}}/, that will redirect you to https://keycloak.{{cluster.baseHostName}}/... for authentication. Once logged in, you’ll be redirected to https://test.{{cluster.baseHostName}}/oauth/callback?... that will log you in the proxy, check your infos, & allow you to pass through if you matches the criterions.

To log out from our authentication proxy, simply reach https://test.{{cluster.baseHostName}}/oauth/logout. You may need to refresh the page without cache on the base url again ( / ) in order to re-request to log in, otherwise the nginx page you’ll see would be cached.

Well, now we know that our stuff work ! Let’s delete this test application.

1

kubectl delete -f 11-NginxTest.yaml

3. Persist data from Keycloak

You can use a persistent datastore by setting postgresql.enabled to true. Think about setting postgresql.storageClass.

1

helm update keycloak -f ./kubernetes/authentication/01-KeycloakChartValues.yaml codecentric/keycloak --version 9.9.2 --namespace keycloak

Initialization of PostgreSQL may take some time. Don’t hesitate to look at the logs of both of PostgreSQL and Keycloak.

Once done, you will need to redo all the steps above, because you’re now using a brand new real database.

1
2
3
4

git add .
git commit -m "Setup cluster's authentication

Following guide @ https://gerkindev.github.io/devblog/walkthroughs/kubernetes/07-authentication/"