Vault operator

The Vault operator builds on Bank-Vaults features such as:

• external, API based configuration (secret engines, auth methods, policies) to automatically re/configure a Vault cluster
• automatic unsealing (AWS, GCE, Azure, Alibaba, Kubernetes Secrets (for dev purposes), Oracle)
• TLS support

The operator flow is the following:

The source code can be found in the vault-operator repository.

The operator requires the following cloud permissions.

Deploy a local Vault operator

This is the simplest scenario: you install the Vault operator on a simple cluster. The following commands install a single-node Vault instance that stores unseal and root tokens in Kubernetes secrets. If you want to customize the Helm chart, see the list of vault-operator Helm chart values.

1. Install the Bank-Vaults operator:

   helm upgrade --install --wait vault-operator oci://ghcr.io/bank-vaults/helm-charts/vault-operator
   

   Expected output:

   Release "vault-operator" does not exist. Installing it now.
   Pulled: ghcr.io/bank-vaults/helm-charts/vault-operator:1.20.0
   Digest: sha256:46045be1c3b215f0c734908bb1d4022dc91eae48d2285382bb71d63f72c737d1
   NAME: vault-operator
   LAST DEPLOYED: Thu Jul 27 11:22:55 2023
   NAMESPACE: default
   STATUS: deployed
   REVISION: 1
   TEST SUITE: None
   

2. Create a Vault instance using the Vault custom resources. This will create a Kubernetes CustomResource called vault and a PersistentVolumeClaim for it:

   kubectl kustomize https://github.com/bank-vaults/vault-operator/deploy/rbac | kubectl apply -f -
       

   Expected output:

   serviceaccount/vault created
       role.rbac.authorization.k8s.io/vault created
       role.rbac.authorization.k8s.io/leader-election-role created
       rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created
       rolebinding.rbac.authorization.k8s.io/vault created
       clusterrolebinding.rbac.authorization.k8s.io/vault-auth-delegator created
       

   kubectl apply -f https://raw.githubusercontent.com/bank-vaults/vault-operator/v1.21.0/deploy/examples/cr-raft.yaml
       

   Expected output:

   vault.vault.banzaicloud.com/vault created
       

   Note: If needed, you can install the latest CustomResource from the main branch, but that’s usually under development and might not be stable.

   kubectl kustomize https://github.com/bank-vaults/vault-operator/deploy/crd | kubectl apply -f -
       

3. Wait a few seconds, then check the operator and the vault pods:

   kubectl get pods
   

   Expected output:

   NAME                                                        READY     STATUS    RESTARTS   AGE
   vault-0                                                     3/3       Running   0          10s
   vault-configurer-6c545cb6b4-dmvb5                           1/1       Running   0          10s
   vault-operator-788559bdc5-kgqkg                             1/1       Running   0          23s
   

4. Configure your Vault client to access the Vault instance running in the vault-0 pod.

   1. Port-forward into the pod:

      kubectl port-forward vault-0 8200 &
      

   2. Set the address of the Vault instance.

      export VAULT_ADDR=https://127.0.0.1:8200
      

   3. Import the CA certificate of the Vault instance by running the following commands (otherwise, you’ll get x509: certificate signed by unknown authority errors):

      kubectl get secret vault-tls -o jsonpath="{.data.ca\.crt}" | base64 --decode > $PWD/vault-ca.crt
      export VAULT_CACERT=$PWD/vault-ca.crt
      

      Alternatively, you can instruct the Vault client to skip verifying the certificate of Vault by running: export VAULT_SKIP_VERIFY=true

   4. If you already have the Vault CLI installed, check that you can access the Vault:

      vault status
      

      Expected output:

      Key             Value
      ---             -----
      Seal Type       shamir
      Initialized     true
      Sealed          false
      Total Shares    5
      Threshold       3
      Version         1.5.4
      Cluster Name    vault-cluster-27ecd0e6
      Cluster ID      ed5492f3-7ef3-c600-aef3-bd77897fd1e7
      HA Enabled      false
      

   5. To authenticate to Vault, you can access its root token by running:

      export VAULT_TOKEN=$(kubectl get secrets vault-unseal-keys -o jsonpath={.data.vault-root} | base64 --decode)
      

      Note: Using the root token is recommended only in test environments. In production environment, create dedicated, time-limited tokens.

   6. Now you can interact with Vault. For example, add a secret by running vault kv put secret/demosecret/aws AWS_SECRET_ACCESS_KEY=s3cr3t If you want to access the Vault web interface, open https://127.0.0.1:8200 in your browser using the root token (to reveal the token, run echo $VAULT_TOKEN).

For other configuration examples of the Vault CustomResource, see the YAML files in the deploy/examples and test/deploy directories of the vault-operator repository. After you are done experimenting with Bank-Vaults and you want to delete the operator, you can delete the related CRs:

kubectl kustomize https://github.com/bank-vaults/vault-operator/deploy/rbac | kubectl delete -f -
kubectl delete -f https://raw.githubusercontent.com/bank-vaults/vault-operator/v1.21.0/deploy/examples/cr-raft.yaml

HA setup with Raft

In a production environment you want to run Vault as a cluster. The following CR creates a 3-node Vault instance that uses the Raft storage backend:

1. Install the Bank-Vaults operator:

   helm upgrade --install --wait vault-operator oci://ghcr.io/bank-vaults/helm-charts/vault-operator
   

2. Create a Vault instance using the cr-raft.yaml custom resource. This will create a Kubernetes CustomResource called vault that uses the Raft backend:

   kubectl kustomize https://github.com/bank-vaults/vault-operator/deploy/rbac | kubectl apply -f -
       

   Expected output:

   serviceaccount/vault created
       role.rbac.authorization.k8s.io/vault created
       role.rbac.authorization.k8s.io/leader-election-role created
       rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created
       rolebinding.rbac.authorization.k8s.io/vault created
       clusterrolebinding.rbac.authorization.k8s.io/vault-auth-delegator created
       

   kubectl apply -f https://raw.githubusercontent.com/bank-vaults/vault-operator/v1.21.0/deploy/examples/cr-raft.yaml
       

   Expected output:

   vault.vault.banzaicloud.com/vault created
       

   Note: If needed, you can install the latest CustomResource from the main branch, but that’s usually under development and might not be stable.

   kubectl kustomize https://github.com/bank-vaults/vault-operator/deploy/crd | kubectl apply -f -
       

Pod anti-affinity

If you want to setup pod anti-affinity, you can set podAntiAffinity vault with a topologyKey value. For example, you can use failure-domain.beta.kubernetes.io/zone to force K8S deploy vault on multi AZ.

Delete a resource created by the operator

If you manually delete a resource that the Bank-Vaults operator has created (for example, the Ingress resource), the operator automatically recreates it every 30 seconds. If it doesn’t, then something went wrong, or the operator is not running. In this case, check the logs of the operator.