Skip to content
insightd

Kubernetes / k3s Setup

On Kubernetes / k3s the insightd agent runs as a DaemonSet — one pod per node — reporting to the same hub and Mosquitto broker you set up in the Quick Start. Each agent only sees pods on its own node and reports that node as a “host”; each pod’s containers appear in the dashboard.

Prerequisites

  • A Kubernetes or k3s cluster
  • A running hub + Mosquitto (from the Quick Start) reachable from the cluster
  • kubectl configured against your cluster

Step 1: Confirm the hub is reachable

The hub doesn’t need to run inside the cluster — the agents just need to reach its Mosquitto broker on port 1883. If you haven’t set the hub up yet, follow the Quick Start first.

Step 2: Edit the DaemonSet manifest

Download agent/k8s/daemonset.yaml from the insightd repo and set your MQTT broker URL:

- name: INSIGHTD_MQTT_URL
  value: mqtt://your-broker.example.com:1883

Step 3: Optional MQTT credentials

If your broker requires authentication, create a secret:

Terminal window
kubectl create namespace insightd
kubectl create secret generic insightd-mqtt \
  --namespace insightd \
  --from-literal=username=insightd \
  --from-literal=password=yourpassword

The DaemonSet manifest already references this secret with optional: true so it’s safe even if you skip this step.

Step 4: Apply the manifests

Terminal window
kubectl apply -f https://raw.githubusercontent.com/goldenproductions/insightd/main/agent/k8s/rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/goldenproductions/insightd/main/agent/k8s/daemonset.yaml

Step 5: Verify

Terminal window
kubectl get pods -n insightd
kubectl logs -n insightd -l app=insightd-agent

You should see one agent pod per node, with logs like:

Detected Kubernetes environment (running in-cluster)
Connected to cluster — node my-node-01 is Ready
Collected 12 containers on node my-node-01

Open your hub UI — you’ll see one host per node, with all the pods on that node listed as containers.

What you’ll see in the UI

  • One host per node, named after the node
  • Each pod’s containers appear as containers under that host
  • Container names use the format {namespace}/{pod-name}/{container-name}
  • CPU/memory metrics from the kubelet’s cAdvisor endpoint
  • Restart count directly from the pod status
  • Logs via the Kubernetes API
  • k8s badge next to the host name in the host detail page

Namespace filtering

Kubernetes clusters come with many system services (kube-system, kube-proxy, coredns, metrics-server, etc.) that you may not care about monitoring. The host detail page for k8s nodes includes a namespace filter bar above the uptime timeline:

  • Toggle chips for each namespace — click to hide/show
  • Applies to both the Uptime (7 days) timeline and the Containers table
  • Hidden namespaces are persisted per host in your browser (localStorage)
  • All namespaces are visible by default — click to hide what you don’t need
  • A “Show all” link appears when filtering is active, showing how many containers are hidden

For example, if you only care about your workloads in default and monitoring, click kube-system to hide all the system pods. The filter remembers your choice across page refreshes.

The namespace prefix is also dimmed in the container name column so the pod/container name stands out.

What’s not supported in k8s mode

  • Container actions (start/stop/restart/remove) — managing pods is the cluster’s job, not the hub’s. Setting INSIGHTD_ALLOW_ACTIONS=true has no effect; the agent will log a warning at startup if you set it. The container detail page renders Start/Stop/Restart disabled with a tooltip on k8s hosts.
  • Image update checks and remote updates — Kubernetes manages image updates via deployments and rollouts; checking digests against Docker Hub is not meaningful in this context. Setting INSIGHTD_ALLOW_UPDATES=true has no effect; the agent will log a warning at startup if you set it. The Updates page shows k8s agents with a “Managed by cluster” label instead of an Update button.

If you need to perform actions or check image updates, use the Docker runtime mode for those hosts.

RBAC permissions

The DaemonSet uses a ServiceAccount with these read-only cluster permissions:

  • pods and pods/log — get, list, watch (to discover pods on the node and read logs)
  • nodes — get, list (to verify the node exists, read capacity for total memory, read creationTimestamp for uptime)
  • nodes/metrics, nodes/stats, nodes/proxy — get (to query the kubelet’s /metrics/cadvisor and /stats/summary endpoints)
  • replicasets (apps API group) — get, list (to walk pod owner references up to the parent Deployment so containers keep a stable name across rollouts)

The agent never modifies anything in the cluster.

Custom kubelet URL

By default the agent talks to https://${NODE_IP}:10250. If your kubelet listens on a different port or you need to override the URL, set INSIGHTD_KUBELET_URL in the DaemonSet env vars.

In-cluster only

The agent requires in-cluster service account credentials (mounted at /var/run/secrets/kubernetes.io/serviceaccount/) and the KUBERNETES_SERVICE_HOST env var that Kubernetes injects automatically. Running the agent outside the cluster pointing at a kubeconfig is not supported — use the DaemonSet.