Kubernetes Metrics Server 入门与部署实战
Kubecc
|
|
11
|
0

1165 字
|
19 分钟

简介

Metrics Server 是一个可扩展、高效的容器资源指标来源，适用于 Kubernetes 内置的自动扩缩容管道。 Metrics Server 通过聚合来自各个节点的资源使用数据，为 Kubernetes 提供了一个统一的接口，以便进行资源监控和自动扩缩容。

默认情况下，Metrics Server 以每分钟一次的频率收集数据，并将其存储在内存中，以便快速访问。如果需要调整收集频率可以修改部署参数 - --metric-resolution=15s
Metrics Server 还支持通过 API 访问资源使用数据，使得用户和其他 Kubernetes 组件能够轻松地获取和使用这些数据。

大白话总结：我们在部署 Metrics Server 后，可以使用 kubectl top 命令和参数来查看集群资源使用状态

[root@master01 ~]# kubectl top --help 
Display resource (CPU/memory) usage.

 The top command allows you to see the resource consumption for nodes or pods.

 This command requires Metrics Server to be correctly configured and working on the server.

Available Commands:
  node          Display resource (CPU/memory) usage of nodes
  pod           Display resource (CPU/memory) usage of pods

Usage:
  kubectl top [flags] [options]

Use "kubectl top <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).

# 节点资源使用率
[root@master01 ~]# kubectl top nodes 
NAME       CPU(cores)   CPU(%)   MEMORY(bytes)   MEMORY(%)   
master01   322m         2%       2377Mi          17%   
master02   170m         1%       1979Mi          14%   
master03   208m         1%       1951Mi          8%  
worker01   328m         0%       2892Mi          2%  

# pod使用情况
[root@master01 ~]# kubectl top pods -A 
NAMESPACE            NAME                                       CPU(cores)   MEMORY(bytes)   
default              nginx-5869d7778c-7f79h                     0m           48Mi  
devops-tools         jenkins-dfff96685-kxpjk                    2m           689Mi   
kube-system          calico-kube-controllers-6f497d8478-4vb7l   9m           80Mi  
kube-system          calico-node-99hbm                          29m          229Mi   
kube-system          calico-node-bmfzw                          33m          299Mi   
kube-system          calico-node-cqdjk                          32m          223Mi   
kube-system          calico-node-ntglp                          25m          691Mi   
kube-system          coredns-7b66d84dd4-5z5lc                   1m           25Mi  
kube-system          coredns-7b66d84dd4-9z8hl                   2m           20Mi  
kube-system          kube-dns-autoscaler-6cf89b5d5f-htfcv       1m           9Mi   
kube-system          metrics-server-6487f5d894-b5dtm            4m           48Mi  
local-path-storage   local-path-provisioner-6676c566c7-8ztl4    1m           21Mi

Metrics-Server 部署

一般情况下我们部署完 kubernetes 集群后，默认是没有部署 Metrics Server 组件的，我们在执行相关命令和参数会出现下列错误信息：error: Metrics API not available

[root@master01 ~]# kubectl top pod
error: Metrics API not available

集群环境信息

  • k8s v1.32.10 版本，二进制部署
  • Runtime containerd
  • 操作系统版本：Rockylinux 9.6

在新版的 Kubernetes 中，系统资源的采集均使用Metrics-server，可以通过 Metrics 采集节点和 Pod 的内存、磁盘、CPU 和网络的使用率。

部署要求

Metrics Server 对集群和网络配置有特定要求。并非所有集群发行版都默认满足这些要求。请确保您的集群发行版支持这些要求后再使用 Metrics Server：

  • kube-apiserver 必须启用启用聚合层。
  • 节点必须启用 Webhook身份验证和授权。
  • Kubelet 证书需要由集群证书颁发机构签名（或者通过传递–kubelet-insecure-tls给指标服务器来禁用证书验证）。
  • 容器运行时必须实现容器指标 RPC（或具有cAdvisor支持）
  • 网络应支持以下通信：
    • 控制平面与指标服务器通信。控制平面节点需要连接到指标服务器的 Pod IP 地址和端口 10250（如果hostNetwork启用了自定义端口，则需要连接到节点 IP 地址和自定义端口）
    • 指标服务器需要访问所有节点上的 Kubelet。

完整的部署文件

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - nodes/metrics
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  replicas: 2
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 1
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node-role.kubernetes.io/control-plane
                operator: Exists
            - matchExpressions:
              - key: node-role.kubernetes.io/master
                operator: Exists
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchLabels:
                k8s-app: metrics-server
            namespaces:
            - kube-system
            topologyKey: kubernetes.io/hostname
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=4443
        - --kubelet-preferred-address-types=Hostname,InternalDNS,InternalIP,ExternalDNS,ExternalIP # 
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls # Cancel kubelet TLS authentication
        - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem # change to front-proxy-ca.crt for kubeadm
        - --requestheader-username-headers=X-Remote-User 
        - --requestheader-group-headers=X-Remote-Group
        - --requestheader-extra-headers-prefix=X-Remote-Extra-
        image: registry.cn-hangzhou.aliyuncs.com/kubecc/metrics-server:v0.8.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 4443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          initialDelaySeconds: 20
          periodSeconds: 10
        resources:
          requests:
            cpu: 100m
            memory: 200Mi
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
        - mountPath: /etc/kubernetes/pki/front-proxy-ca.pem
          name: pki
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
      - hostPath:
          path: /etc/kubernetes/pki/front-proxy-ca.pem
        name: pki
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true # Skip TLS verification for the APIService
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

部署操作

我这边已经提前部署了，仅作信息展示

[root@master01 metrics-server]# kubectl apply -f Deployment-metrics-server.yaml 
serviceaccount/metrics-server unchanged
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader unchanged
clusterrole.rbac.authorization.k8s.io/system:metrics-server unchanged
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader unchanged
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server unchanged
service/metrics-server unchanged
deployment.apps/metrics-server configured
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io unchanged
[root@master01 metrics-server]# kubectl get pods -n kube-system | grep metrics
metrics-server-75f985cc96-22tkp            1/1     Running   0             143m
metrics-server-75f985cc96-ptbsk            1/1     Running   0             143m
[root@master01 metrics-server]#

特别参数说明

kube-apiserver/前置代理的客户端证书，配置 apiserver 代理到 metrics-server 时的用户身份透传 Header 规则，让 metrics-server 能正确识别请求发起者是谁、属于哪些组、带哪些 extra 信息

  • –kubelet-insecure-tls # Cancel kubelet TLS authentication
  • –requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem # change to front-proxy-ca.crt for kubeadm
  • –requestheader-username-headers=X-Remote-User
  • –requestheader-group-headers=X-Remote-Group
  • –requestheader-extra-headers-prefix=X-Remote-Extra-

根据需求我限制了 metrics-server-pod 运行的所在节点，你根据你的需求调整，如果你使用其他方式部署的集群，请注意更换客户端证书文件的 CA 证书。

整体部署方法可以参考下面相关文献

参考文献

【Metrics-Server Github】

【配置聚合层】

K8SMetrics Server
暂无评论

发送评论 编辑评论 


				

			

						

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
							
													

					

				

			

			
			

				

					
				

			

				

											

							
							
						

																				
					
											
						

	

		
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
（╯‵□′）╯︵┴─┴
￣﹃￣
(/ω＼)
∠( ᐛ 」∠)＿
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ｀)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ(￣∇￣o)
ヾ(´･ ･｀｡)ノ"
( ง ᵒ̌皿ᵒ̌)ง⁼³₌₃
(ó﹏ò｡)
Σ(っ °Д °;)っ
( ,,´･ω･)ﾉ"(´っω･｀｡)
╮(╯▽╰)╭
o(*////▽////*)q
＞﹏＜
( ๑´•ω•) "(ㆆᴗㆆ)
😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣
Source: github.com/k4yt3x/flowerhd
	

	

		
颜文字
Emoji
小恐龙
花!
	


									

			

			
			
		

	






上一篇