Metrics not compatible with prometheus which is deployed using operator that has routePrefix in spec

Lens Support & Troubleshooting
#1

Hi, the metrics does not show up in my case.

I think it is because the spec I use has routePrefix in it

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: monitor
  namespace: kube-prometheus-stack-prometheus
spec:
  ...
  routePrefix: /prometheus

I tried following metrics settings in cluster settings

PROMETHEUS: Prometheus Operator
PROMETHEUS SERVICE ADDRESS: monitor/kube-prometheus-stack-prometheus:9090/prometheus

and it is not working.

#2

I got this in kube-api-server log, the requestURI is 404

{
    "kind": "Event",
    "apiVersion": "audit.k8s.io/v1",
    "level": "RequestResponse",
    "auditID": "*****",
    "stage": "ResponseStarted",
    "requestURI": "/api/v1/namespaces/monitor/services/kube-prometheus-stack-prometheus:9090/proxy/api/v1/query_range",
    "verb": "create",
    ...
   "responseStatus": {
        "metadata": {},
        "code": 404
    },
    ...
}

the correct URI should be

/api/v1/namespaces/monitor/services/kube-prometheus-stack-prometheus:9090/proxy/prometheus/api/v1/query_range

#3

Hi, this is a known issue/feature request. We plan to add a configuration option to Metrics settings which would allow setting a prefix to the url. Currently there is no real workaround.

1 Like
#4

Hi, could you try this service address:

kube-prometheus-stack-prometheus/monitor:9090/prometheus

this is assuming the service name is “monitor” - please adjust if it’s something else. It seems the namespace is kube-prometheus-stack-prometheus, not monitor.

Note that the input should be in form “namespace/service:port” but it should optionally accept a prefix.