Kubernetes Secrets – How to Create, Use and Access Secrets

Introduction

Most applications deployed through Kubernetes require access to databases, services, and other resources located externally. The easiest way to manage the login information necessary to access those resources is using Kubernetes secrets. Secrets help organize and distribute sensitive information across a cluster.

In this tutorial, you will learn what Kubernetes secrets are and how to create and use them in your cluster.

Prerequisites

• A Kubernetes cluster (for testing purposes, you can create it with minikube)
• The Kubernetes kubectl command-line tool

What Are Kubernetes Secrets?

A Kubernetes secret is an object storing sensitive pieces of data such as usernames, passwords, tokens, and keys. Secrets are created by the system during an app installation or by users whenever they need to store sensitive information and make it available to a pod.

If passwords, tokens, or keys were simply part of a pod definition or container image, they could be accidentally exposed during Kubernetes operations. Therefore, the most important function of the secret is to prevent accidental exposure of the information stored in it while at the same time making it available wherever the user needs it.

Kubernetes Secret Types

Kubernetes features two categories of secrets:

• The system’s service accounts automatically create built-in secrets and associate them with containers together with API credentials.
• You can also create customized secrets for credentials you need to make available to pods.

Built-in secrets come in several types, corresponding to popular usage scenarios:

Built-in Type	Description
Opaque	This is the default type of secret. The secrets whose configuration file does not contain the type statement are all considered to be of this type. Opaque secrets are designed to store arbitrary user data.
kubernetes.io/service-account-token	Service account token secrets store tokens identifying service accounts. Upon creation of a pod, Kubernetes automatically creates this secret and associates it with the pod, enabling secure access to the API. This behavior can be disabled.
kubernetes.io/dockercfg	Accessing a Docker registry for images requires valid Docker credentials. This type of secret is used to store a serialized ~/.dockercfg legacy format for Docker command-line configuration. It contains the base64-encoded .dockercfg key.
kubernetes.io/dockerconfigjson	This type of secret features a .dockerconfigjson key, which is a base64-encoded version of the ~/.docker/config.json file, a new version of the deprecated .dockercfg.
kubernetes.io/basic-auth	The secret for storing basic authentication data. It must contain two keys – username and password.
kubernetes.io/ssh-auth	For storing data necessary for establishing an SSH connection, use ssh-auth type. This type’s data field must contain an ssh-privatekey key-value pair.
kubernetes.io/tls	This type is used to store TLS certificates and keys. The most common usage scenario is Ingress resource termination, but the tls type is also sometimes used with other resources.
bootstrap.kubernetes.io/token	Tokens used during the node bootstrap process are stored using the token secret type. This type is usually created in the kube-system namespace.

To define a customized type of secret, assign a non-empty string as a value in the type field of the secret file. Leaving the field empty tells Kubernetes to assume the Opaque type. The customized type frees the secret of constraints posed by built-in types.

Using Kubernetes Secrets

When you create a secret, it needs to be referenced by the pod that will use it. To make a secret available for a pod:

1. Mount the secret as a file in a volume available to any number of containers in a pod.

2. Import the secret as an environment variable to a container.

3. Use kubelet, and the imagePullSecrets field.

The following sections explain how to create Kubernetes secrets, as well as how to decode and access them.

Create Kubernetes Secrets

To create a Kubernetes secret, apply one of the following methods:

• Use kubectl for a command-line based approach.
• Create a configuration file for the secret.
• Use a generator, such as Kustomize to generate the secret.

Create Secrets Using kubectl

1. To start creating a secret with kubectl, first create the files to store the sensitive information:

echo -n '[username]' > [file1]
echo -n '[password]' > [file2]

The -n option tells echo not to append a new line at the end of the string. The new line is also treated as a character, so it would be encoded together with the rest of the characters, producing a different encoded value.

2. Now, use kubectl to create a secret using the files from the previous step. Use the generic subcommand to create an Opaque secret. Also, add the --from-file option for each of the files you want to include:

kubectl create secret generic [secret-name] /  
--from-file=[file1] /
--from-file=[file2]

The output confirms the creation of the secret:

3. To provide keys for values stored in the secret, use the following syntax:

kubectl create secret generic [secret-name] /  
--from-file=[key1]=[file1] /  
--from-file=[key2]=[file2]

4. Check that the secret has been successfully created by typing:

kubectl get secrets

The command shows the list of available secrets – their names, types, number of data values they contain, and their age:

Create Secrets in a Configuration File

1. To create a secret by specifying the necessary information in a configuration file, start by encoding the values you wish to store:

echo -n '[value1]' | base64
echo -n '[value2]' | base64

2. Now create a yaml file using a text editor. The file should look like this:

apiVersion: v1
kind: Secret
metadata:  
  name: newsecret
type: Opaque
data:
  username: dXNlcg==
  password: NTRmNDFkMTJlOGZh

3. Save the file and use the kubectl apply command to create the secret:

kubectl apply -f [file]

Create Kubernetes Secret with Generators

Generators such as Kustomize help quickly generate secrets.

1. To create a secret with Kustomize, create a file named kustomization.yaml and format it as follows:

secretGenerator:
- name: db-credentials 
  files:
  - username.txt
  - password.txt

The example above states db-credentials as the name of the secret and uses two previously created files, username.txt, and password.txt, as data values.

2. Alternatively, to provide the unencrypted, literal version of the data values, include the literals section with key-value pairs you wish to store:

secretGenerator:
- name: db-credentials
  literals:
  - username=user
  - password=54f41d12e8fa

3. Save the file and use the following command in the folder where kustomization.yaml is located:

kubectl apply -k .

The output confirms the creation of the secret:

Use kubectl describe to See Created Secrets

The kubectl describe command shows basic information about Kubernetes objects. Use it to view the description of a secret.

kubectl describe secrets/[secret]

The first example shows the secret created by providing files as data values:

The second example describes the secret created using string literals. Notice the change in the Data section, which now shows names of the keys instead of filenames:

Decode Secrets

1. To decode the values in a secret, access them by typing the following command:

kubectl get secret [secret] -o jsonpath='{.data}'

The output shows the encoded key-value pairs stored in the data section:

2. Use the echo command to type the encoded string and pipe the output to the base64 command:

echo '[encoded-value]' | base64 --decode

The decoded strings appear as the output:

Access Secrets Loaded in a Volume

1. To access secrets mounted to a pod in a separate volume, modify the definition of the pod to include a new volume. Choose any volume name you want, but make sure that it is the same as the name of the secret object.

2. Be sure to specify readOnly as true. For example, the pod definition may look like this:

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
  spec:
    containers:
      - name: test-pod
        image: redis
        volumeMounts:
        - name: newsecret
          mountPath: “/etc/newsecret”
          readOnly: true
    volumes:
    - name: newsecret
      secret:
        secretName: newsecret

2. Open another terminal instance and use the kubectl exec command to access the pod’s bash shell:

kubectl exec -it [pod] -- /bin/bash

3. cd into /etc/newsecret, and find the files contained in the secret:

cd /etc/newsecret

Project Secrets into a Container Using Environment Variables

1. Another way to access secrets in a Kubernetes pod is to import them as environment variables by modifying the pod definition to include references to them. For example:

apiVersion: v1 
kind: Pod 
metadata: 
  name: secret-env-pod 
spec: 
  containers: 
  - name: secret-env-pod
    image: redis 
    env: 
      - name: SECRET_USERNAME 
        valueFrom: 
          secretKeyRef: 
            name: newsecret 
            key: username 
      - name: SECRET_PASSWORD 
        valueFrom: 
          secretKeyRef: 
            name: newsecret 
            key: password 
  restartPolicy: Never

2. Use kubectl exec again to bash into a pod.

3. Test the environment variable using the echo command:

echo $[VARIABLE]

The command output shows the value mapped to the variable:

Use Secrets to Pull Docker Images from Private Docker Registries

1. To use private Docker registries, first, you need to log in to Docker:

docker login

2. When prompted, provide your login credentials:

3. If the login is successful, Docker updates the config.json file with your data. Use the cat command to view the file:

cat ~/.docker/config.json

The auths section contains the auth key, which is an encoded version of the Docker credentials.

4. Use kubectl to create a secret, providing the location of the config.json file and the type of the secret:

kubectl create secret generic [secret] /
--from-file=.dockerconfigjson=./.docker/config.json /
--type=kubernetes.io/dockerconfigjson

Alternatively, perform all the steps above, including logging in to Docker, on the same line:

kubectl create secret docker-registry [secret] --docker-server:[address] --docker-username=[username] --docker-password=[password] --docker-email=[email]

5. To create a pod that has access to this secret, create a yaml file that defines it. The file should look like this:

apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec: 
  containers:
  - name: private-reg-container
    image:   
  imagePullSecrets:  
  - name: regcred

6. Finish creating the pod by activating it with kubectl apply:

kubectl apply -f [file]

Kubernetes Secrets Considerations

Kubernetes secrets are a secure way to store sensitive information. However, before you decide on the best method for your usage scenario, you should consider the following points:

• Usernames and passwords in Secrets are encoded with base-64. This text-encoding technique obscures data and prevents accidental exposure, but it is not secure against malicious cyber attacks.
• Secrets are only available in the cluster in which they are located.
• Secrets usually rely on a master key which is used to unlock them all. While there are methods to secure the master key, using them only creates another master key scenario.

To mitigate these problems, apply some of the solutions below:

• Integrate a secrets management tool that uses the Kubernetes Service account to authenticate users who need access to the secret vault.
• Integrate an IAM (Identity and Access Management) tool to allow the system to use tokens from a Secure Token Service.
• Integrate a third-party secrets manager into pods.

Conclusion

After reading this tutorial, you should know what Kubernetes secrets are, which types exist, and how to create a Kubernetes secret. The tutorial also presented ways in which secrets are accessed.

For more information about Kubernetes security, read Kubernetes Security Best Practices.