# Tons of OpenAPI errors

**URL:** https://forums.k8slens.dev/t/tons-of-openapi-errors/932
**Category:** All about k0s
**Created:** [August 14, 2023, 1:31am UTC](https://forums.k8slens.dev/t/tons-of-openapi-errors/932 "2023-08-14T01:31:14Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![davidshen84](https://forums.k8slens.dev/user_avatar/forums.k8slens.dev/davidshen84/32/558_2.png) [@davidshen84](https://forums.k8slens.dev/u/davidshen84)
#### Post date: [August 14, 2023, 1:31am UTC](https://forums.k8slens.dev/t/tons-of-openapi-errors/932/1 "2023-08-14T01:31:14Z")

</div>

Hi,

I got the following errors flooding my system journal. I think the IP `10.101.213.69` refers to the `metrics-server` pod in my cluster.

```auto
Aug 14 11:23:46 l09853 k0s[441]: time="2023-08-14 11:23:46" level=info msg="E0814 11:23:46.339804 705 controller.go:116] loading OpenAPI spec for \"v1beta1.metrics.k8s.io\" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable" component=kube-apiserver stream=stderr Aug 14 11:23:46 l09853 k0s[441]: time="2023-08-14 11:23:46" level=info msg=", Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]" component=kube-apiserver stream=stderr Aug 14 11:23:46 l09853 k0s[441]: time="2023-08-14 11:23:46" level=info msg="I0814 11:23:46.341048 705 controller.go:129] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue." component=kube-apiserver stream=stderr Aug 14 11:23:50 l09853 k0s[441]: time="2023-08-14 11:23:50" level=info msg="E0814 11:23:50.340588 705 available_controller.go:460] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.101.213.69:443/apis/metrics.k8s.io/v1beta1: Get \"https://10.101.213.69:443/apis/metrics.k8s.io/v1beta1\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" component=kube-apiserver stream=stderr Aug 14 11:23:55 l09853 k0s[441]: time="2023-08-14 11:23:55" level=info msg="E0814 11:23:55.347965 705 available_controller.go:460] v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.101.213.69:443/apis/metrics.k8s.io/v1beta1: Get \"https://10.101.213.69:443/apis/metrics.k8s.io/v1beta1\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" component=kube-apiserver stream=stderr
Aug 14 11:23:55 l09853 k0s[441]: time="2023-08-14 11:23:55" level=info msg="I0814 11:23:55.396666 705 handler_discovery.go:325] DiscoveryManager: Failed to download discovery for kube-system/metrics-server:443: 503 error trying to reach service: EOF" component=kube-apiserver stream=stderr
Aug 14 11:23:55 l09853 k0s[441]: time="2023-08-14 11:23:55" level=info msg="I0814 11:23:55.396712 705 handler.go:232] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager" component=kube-apiserver stream=stderr

```

However, the log level of these messages is **info**. So, maybe I can ignore them?

But I hope there’s a better way to resolve it.

Thanks.

---

<div class="post-metadata">

### Author: ![jnummelin](https://forums.k8slens.dev/letter_avatar_proxy/v4/letter/j/c5a1d2/32.png) [@jnummelin](https://forums.k8slens.dev/u/jnummelin)
#### Post date: [August 14, 2023, 9:36am UTC](https://forums.k8slens.dev/t/tons-of-openapi-errors/932/2 "2023-08-14T09:36:06Z")

</div>

kube-apiserver is trying to connect to metrics-server but apparently that is failing. It connect it via the `kube-system/metrics-server` ClusterIP service.

Do you run plain controller node(s) (==no worker enable on the controller) in your cluster? If yes, then this is often a symptom of konnectivity agents on the workers not being able to connect to the konnectivity-server on the controllers.

Check the following docs for some hints on the config:

> **[Control Plane High Availability - Documentation](https://docs.k0sproject.io/stable/high-availability/)**
>
> Documentation for k0s, the Zero friction Kubernetes distribution.

> **[Node-local load balancing - Documentation](https://docs.k0sproject.io/stable/nllb/)**
>
> Documentation for k0s, the Zero friction Kubernetes distribution.

---

<div class="post-metadata">

### Author: ![davidshen84](https://forums.k8slens.dev/user_avatar/forums.k8slens.dev/davidshen84/32/558_2.png) [@davidshen84](https://forums.k8slens.dev/u/davidshen84)
#### Post date: [August 14, 2023, 11:06am UTC](https://forums.k8slens.dev/t/tons-of-openapi-errors/932/3 "2023-08-14T11:06:59Z")

</div>

No, I am using a **single** configuration. Anyway, after I reset the cluster a few times, the metrics server seems to be working.

Maybe it has something to do with `kube-router`. I remember in one experiment, I disabled it by setting `metricsPort` to **0**

Thanks

---

<div class="post-metadata">

### Author: ![jnummelin](https://forums.k8slens.dev/letter_avatar_proxy/v4/letter/j/c5a1d2/32.png) [@jnummelin](https://forums.k8slens.dev/u/jnummelin)
#### Post date: [August 14, 2023, 12:29pm UTC](https://forums.k8slens.dev/t/tons-of-openapi-errors/932/4 "2023-08-14T12:29:17Z")

</div>

It’s is common that when the cluster is booting up there are these errors until all pods etc. are properly up and running.

Metrics server registers itself as API extension on the api-server and based on our testing getting everything “ready” can take some minutes in some cases. The api extension is registered **BEFORE** the pods etc. are are properly running. I _think_ API has some backoff policy when connecting/discovering the extensions and thus it takes a while when images are pulled etc. the first time at least.
