Kubernetes性能优化实践提升集群运行效率一、性能优化概述Kubernetes性能优化涉及多个层面从节点配置到应用部署都需要精心优化。优化的目标包括资源利用率提高CPU、内存和存储的利用率响应时间减少应用响应延迟吞吐量提高系统处理能力稳定性减少故障和性能抖动二、节点级优化2.1 资源预留apiVersion: v1 kind: Node metadata: name: node-1 annotations: node.kubernetes.io/role: worker spec: taints: - key: dedicated value: worker effect: NoSchedule2.2 Kubelet配置优化apiVersion: v1 kind: ConfigMap metadata: name: kubelet-config data: kubelet: | apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cpuManagerPolicy: static cpuManagerReconcilePeriod: 10s memoryManagerPolicy: Static reservedSystemCPUs: 0-1 systemReserved: cpu: 500m memory: 1Gi kubeReserved: cpu: 500m memory: 512Mi2.3 节点亲和性配置apiVersion: v1 kind: Pod spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: node-role.kubernetes.io/worker operator: Exists preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: disktype operator: In values: - ssd三、Pod级优化3.1 资源限制配置apiVersion: v1 kind: Pod metadata: name: optimized-pod spec: containers: - name: my-app image: my-app:latest resources: requests: cpu: 250m memory: 512Mi limits: cpu: 500m memory: 1Gi3.2 Liveness和Readiness探针apiVersion: v1 kind: Pod spec: containers: - name: my-app image: my-app:latest livenessProbe: httpGet: path: /health port: 8080 initialDelaySeconds: 10 periodSeconds: 5 failureThreshold: 3 readinessProbe: httpGet: path: /ready port: 8080 initialDelaySeconds: 5 periodSeconds: 3 successThreshold: 23.3 拓扑感知调度apiVersion: v1 kind: Pod spec: topologySpreadConstraints: - maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: ScheduleAnyway labelSelector: matchLabels: app: my-app四、存储优化4.1 选择合适的StorageClassapiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ssd-storage provisioner: kubernetes.io/aws-ebs parameters: type: gp3 iopsPerGB: 3000 throughput: 125 reclaimPolicy: Delete allowVolumeExpansion: true4.2 本地存储配置apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local-storage provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumerapiVersion: v1 kind: PersistentVolume metadata: name: local-pv spec: capacity: storage: 100Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Delete storageClassName: local-storage local: path: /mnt/local-storage nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - node-14.3 存储缓存配置apiVersion: v1 kind: Pod spec: containers: - name: my-app image: my-app:latest volumeMounts: - name: cache-volume mountPath: /cache volumes: - name: cache-volume emptyDir: medium: Memory sizeLimit: 1Gi五、网络优化5.1 配置NetworkPolicyapiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: optimize-network spec: podSelector: matchLabels: app: my-app ingress: - from: - podSelector: matchLabels: app: frontend ports: - protocol: TCP port: 80805.2 服务质量配置apiVersion: v1 kind: Service metadata: name: my-service annotations: service.beta.kubernetes.io/aws-load-balancer-type: nlb spec: type: LoadBalancer selector: app: my-app ports: - port: 80 targetPort: 80805.3 Ingress优化apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: optimized-ingress annotations: nginx.ingress.kubernetes.io/ssl-redirect: true nginx.ingress.kubernetes.io/proxy-buffering: on nginx.ingress.kubernetes.io/proxy-buffer-size: 16k nginx.ingress.kubernetes.io/client-max-body-size: 10m spec: tls: - hosts: - app.example.com secretName: app-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-service port: number: 80六、调度优化6.1 配置调度器策略apiVersion: v1 kind: ConfigMap metadata: name: scheduler-config data: scheduler.config: | apiVersion: kubescheduler.config.k8s.io/v1beta3 kind: KubeSchedulerConfiguration profiles: - name: default pluginConfig: - name: NodeResourcesFit args: scoringStrategy: type: LeastAllocated resources: - name: cpu weight: 1 - name: memory weight: 16.2 Pod优先级配置apiVersion: scheduling.k8s.io/v1 kind: PriorityClass metadata: name: high-priority value: 1000000 globalDefault: false description: High priority podsapiVersion: v1 kind: Pod metadata: name: high-priority-pod spec: priorityClassName: high-priority containers: - name: my-app image: my-app:latest七、监控与调优7.1 Prometheus指标配置apiVersion: v1 kind: Service metadata: name: metrics-service spec: selector: app: my-app ports: - name: metrics port: 8080 targetPort: metrics7.2 配置Horizontal Pod AutoscalerapiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-deployment minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 807.3 配置Vertical Pod AutoscalerapiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: my-vpa spec: targetRef: apiVersion: apps/v1 kind: Deployment name: my-deployment updatePolicy: updateMode: Auto resourcePolicy: containerPolicies: - containerName: my-app minAllowed: cpu: 100m memory: 256Mi maxAllowed: cpu: 1 memory: 2Gi八、应用级优化8.1 启用gzip压缩apiVersion: v1 kind: ConfigMap metadata: name: nginx-config data: nginx.conf: | http { gzip on; gzip_types text/plain text/css application/json application/javascript; gzip_min_length 256; }8.2 配置连接池apiVersion: v1 kind: Pod spec: containers: - name: my-app image: my-app:latest env: - name: DB_POOL_SIZE value: 20 - name: MAX_CONNECTIONS value: 1008.3 启用HTTP/2apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: http2-ingress annotations: nginx.ingress.kubernetes.io/http2-enable: true spec: tls: - hosts: - app.example.com secretName: app-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-service port: number: 443九、性能测试与基准9.1 使用k6进行负载测试import http from k6/http; import { sleep } from k6; export const options { vus: 100, duration: 30s, }; export default function () { http.get(https://api.example.com/users); sleep(1); }9.2 使用hey进行HTTP基准测试# 安装hey go install github.com/rakyll/heylatest # 运行基准测试 hey -n 10000 -c 100 https://api.example.com/users9.3 使用kubectl进行性能分析# 查看Pod资源使用 kubectl top pods # 查看节点资源使用 kubectl top nodes # 查看事件 kubectl get events # 查看Pod日志 kubectl logs my-pod -c my-container十、最佳实践总结10.1 资源配置设置合理的资源请求和限制使用HPA实现自动扩缩容配置VPA优化资源分配10.2 存储优化使用SSD存储提高IO性能配置本地存储减少网络延迟使用emptyDir作为临时缓存10.3 网络优化配置NetworkPolicy减少不必要的流量使用高性能负载均衡器启用HTTP/2和gzip压缩10.4 监控调优配置完整的监控指标设置合理的告警阈值定期进行性能测试参考资料Kubernetes性能优化指南Prometheus官方文档k6官方文档
Kubernetes性能优化实践：提升集群运行效率
Kubernetes性能优化实践提升集群运行效率一、性能优化概述Kubernetes性能优化涉及多个层面从节点配置到应用部署都需要精心优化。优化的目标包括资源利用率提高CPU、内存和存储的利用率响应时间减少应用响应延迟吞吐量提高系统处理能力稳定性减少故障和性能抖动二、节点级优化2.1 资源预留apiVersion: v1 kind: Node metadata: name: node-1 annotations: node.kubernetes.io/role: worker spec: taints: - key: dedicated value: worker effect: NoSchedule2.2 Kubelet配置优化apiVersion: v1 kind: ConfigMap metadata: name: kubelet-config data: kubelet: | apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cpuManagerPolicy: static cpuManagerReconcilePeriod: 10s memoryManagerPolicy: Static reservedSystemCPUs: 0-1 systemReserved: cpu: 500m memory: 1Gi kubeReserved: cpu: 500m memory: 512Mi2.3 节点亲和性配置apiVersion: v1 kind: Pod spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: node-role.kubernetes.io/worker operator: Exists preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: disktype operator: In values: - ssd三、Pod级优化3.1 资源限制配置apiVersion: v1 kind: Pod metadata: name: optimized-pod spec: containers: - name: my-app image: my-app:latest resources: requests: cpu: 250m memory: 512Mi limits: cpu: 500m memory: 1Gi3.2 Liveness和Readiness探针apiVersion: v1 kind: Pod spec: containers: - name: my-app image: my-app:latest livenessProbe: httpGet: path: /health port: 8080 initialDelaySeconds: 10 periodSeconds: 5 failureThreshold: 3 readinessProbe: httpGet: path: /ready port: 8080 initialDelaySeconds: 5 periodSeconds: 3 successThreshold: 23.3 拓扑感知调度apiVersion: v1 kind: Pod spec: topologySpreadConstraints: - maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: ScheduleAnyway labelSelector: matchLabels: app: my-app四、存储优化4.1 选择合适的StorageClassapiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ssd-storage provisioner: kubernetes.io/aws-ebs parameters: type: gp3 iopsPerGB: 3000 throughput: 125 reclaimPolicy: Delete allowVolumeExpansion: true4.2 本地存储配置apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local-storage provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumerapiVersion: v1 kind: PersistentVolume metadata: name: local-pv spec: capacity: storage: 100Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Delete storageClassName: local-storage local: path: /mnt/local-storage nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - node-14.3 存储缓存配置apiVersion: v1 kind: Pod spec: containers: - name: my-app image: my-app:latest volumeMounts: - name: cache-volume mountPath: /cache volumes: - name: cache-volume emptyDir: medium: Memory sizeLimit: 1Gi五、网络优化5.1 配置NetworkPolicyapiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: optimize-network spec: podSelector: matchLabels: app: my-app ingress: - from: - podSelector: matchLabels: app: frontend ports: - protocol: TCP port: 80805.2 服务质量配置apiVersion: v1 kind: Service metadata: name: my-service annotations: service.beta.kubernetes.io/aws-load-balancer-type: nlb spec: type: LoadBalancer selector: app: my-app ports: - port: 80 targetPort: 80805.3 Ingress优化apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: optimized-ingress annotations: nginx.ingress.kubernetes.io/ssl-redirect: true nginx.ingress.kubernetes.io/proxy-buffering: on nginx.ingress.kubernetes.io/proxy-buffer-size: 16k nginx.ingress.kubernetes.io/client-max-body-size: 10m spec: tls: - hosts: - app.example.com secretName: app-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-service port: number: 80六、调度优化6.1 配置调度器策略apiVersion: v1 kind: ConfigMap metadata: name: scheduler-config data: scheduler.config: | apiVersion: kubescheduler.config.k8s.io/v1beta3 kind: KubeSchedulerConfiguration profiles: - name: default pluginConfig: - name: NodeResourcesFit args: scoringStrategy: type: LeastAllocated resources: - name: cpu weight: 1 - name: memory weight: 16.2 Pod优先级配置apiVersion: scheduling.k8s.io/v1 kind: PriorityClass metadata: name: high-priority value: 1000000 globalDefault: false description: High priority podsapiVersion: v1 kind: Pod metadata: name: high-priority-pod spec: priorityClassName: high-priority containers: - name: my-app image: my-app:latest七、监控与调优7.1 Prometheus指标配置apiVersion: v1 kind: Service metadata: name: metrics-service spec: selector: app: my-app ports: - name: metrics port: 8080 targetPort: metrics7.2 配置Horizontal Pod AutoscalerapiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-deployment minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 807.3 配置Vertical Pod AutoscalerapiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: my-vpa spec: targetRef: apiVersion: apps/v1 kind: Deployment name: my-deployment updatePolicy: updateMode: Auto resourcePolicy: containerPolicies: - containerName: my-app minAllowed: cpu: 100m memory: 256Mi maxAllowed: cpu: 1 memory: 2Gi八、应用级优化8.1 启用gzip压缩apiVersion: v1 kind: ConfigMap metadata: name: nginx-config data: nginx.conf: | http { gzip on; gzip_types text/plain text/css application/json application/javascript; gzip_min_length 256; }8.2 配置连接池apiVersion: v1 kind: Pod spec: containers: - name: my-app image: my-app:latest env: - name: DB_POOL_SIZE value: 20 - name: MAX_CONNECTIONS value: 1008.3 启用HTTP/2apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: http2-ingress annotations: nginx.ingress.kubernetes.io/http2-enable: true spec: tls: - hosts: - app.example.com secretName: app-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-service port: number: 443九、性能测试与基准9.1 使用k6进行负载测试import http from k6/http; import { sleep } from k6; export const options { vus: 100, duration: 30s, }; export default function () { http.get(https://api.example.com/users); sleep(1); }9.2 使用hey进行HTTP基准测试# 安装hey go install github.com/rakyll/heylatest # 运行基准测试 hey -n 10000 -c 100 https://api.example.com/users9.3 使用kubectl进行性能分析# 查看Pod资源使用 kubectl top pods # 查看节点资源使用 kubectl top nodes # 查看事件 kubectl get events # 查看Pod日志 kubectl logs my-pod -c my-container十、最佳实践总结10.1 资源配置设置合理的资源请求和限制使用HPA实现自动扩缩容配置VPA优化资源分配10.2 存储优化使用SSD存储提高IO性能配置本地存储减少网络延迟使用emptyDir作为临时缓存10.3 网络优化配置NetworkPolicy减少不必要的流量使用高性能负载均衡器启用HTTP/2和gzip压缩10.4 监控调优配置完整的监控指标设置合理的告警阈值定期进行性能测试参考资料Kubernetes性能优化指南Prometheus官方文档k6官方文档
