Monitoring the Webhook with Grafana and Prometheus

To monitor the webhook with Prometheus and Grafana, complete the following steps.

Prerequisites

Steps

  1. Install the Prometheus Operator Bundle:

    kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml
    
  2. Install the webhook with monitoring and Prometheus Operator ServiceMonitor enabled:

    helm upgrade --wait --install vault-secrets-webhook \
        oci://ghcr.io/bank-vaults/helm-charts/vault-secrets-webhook \
        --namespace vault-infra \
        --set metrics.enabled=true \
        --set metrics.serviceMonitor.enabled={}
    
  3. Create a Prometheus instance which monitors the components of Bank-Vaults:

    kubectl apply -f https://raw.githubusercontent.com/bank-vaults/vault-operator/main/test/prometheus.yaml
    
  4. Create a Grafana instance and expose it:

    kubectl create deployment grafana --image grafana/grafana
    kubectl expose deployment grafana --port 3000 --type LoadBalancer
    
  5. Fetch the external IP address of the Grafana instance, and open it in your browser on port 3000.

    kubectl get service grafana
    
  6. Create a Prometheus Data Source in this Grafana instance which grabs data from http://prometheus-operated:9090/.

  7. Import the Kubewebhook admission webhook dashboard to Grafana (created by Xabier Larrakoetxea).

  8. Select the previously created Data Source to feed this dashboard.