Monitoring the Webhook with Grafana and Prometheus

To monitor the webhook with Prometheus and Grafana, complete the following steps.

Prerequisites

• An already deployed and configured mutating webhook. For details, see Secret injection webhook.

Steps

1. Install the Prometheus Operator Bundle:

   kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml
   

2. Install the webhook with monitoring and Prometheus Operator ServiceMonitor enabled:

   helm upgrade --wait --install vault-secrets-webhook \
       oci://ghcr.io/bank-vaults/helm-charts/vault-secrets-webhook \
       --namespace vault-infra \
       --set metrics.enabled=true \
       --set metrics.serviceMonitor.enabled={}
   

3. Create a Prometheus instance which monitors the components of Bank-Vaults:

   kubectl apply -f https://raw.githubusercontent.com/bank-vaults/vault-operator/main/test/prometheus.yaml
   

4. Create a Grafana instance and expose it:

   kubectl create deployment grafana --image grafana/grafana
   kubectl expose deployment grafana --port 3000 --type LoadBalancer
   

5. Fetch the external IP address of the Grafana instance, and open it in your browser on port 3000.

   kubectl get service grafana
   

6. Create a Prometheus Data Source in this Grafana instance which grabs data from http://prometheus-operated:9090/.

7. Import the Kubewebhook admission webhook dashboard to Grafana (created by Xabier Larrakoetxea).

8. Select the previously created Data Source to feed this dashboard.