ConfigMap与Secret管理完全指南前言在Kubernetes中ConfigMap和Secret是管理应用配置的核心资源。它们允许我们将配置与容器镜像分离实现配置的动态管理和安全存储。本文将详细介绍ConfigMap和Secret的使用方法、最佳实践以及高级特性。一、ConfigMap详解1.1 ConfigMap概述ConfigMap用于存储非敏感的配置文件、命令行参数、环境变量等配置数据。它是一种键值对资源支持多种数据格式。1.2 创建ConfigMap的方式方式一命令行创建# 从文件创建 kubectl create configmap app-config --from-fileapplication.properties./config/application.properties # 从目录创建 kubectl create configmap app-config --from-file./config/ # 从环境变量文件创建 kubectl create configmap app-config --from-env-file./config/.env # 从字面值创建 kubectl create configmap app-config --from-literalAPP_ENVproduction --from-literalLOG_LEVELinfo方式二YAML定义apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: production labels: app: my-app version: v1 data: application.properties: | server.port8080 spring.application.namemy-app spring.profiles.activeproduction logging.level.rootINFO logging.level.com.myappDEBUG database.properties: | db.drivercom.mysql.cj.jdbc.Driver db.urljdbc:mysql://mysql-service:3306/mydb db.username${DB_USERNAME} db.password${DB_PASSWORD} db.pool.maxSize20 db.pool.minIdle5 nginx.conf: | worker_processes auto; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main $remote_addr - $remote_user [$time_local] $request $status $body_bytes_sent $http_referer $http_user_agent $http_x_forwarded_for; access_log /var/log/nginx/access.log main; sendfile on; keepalive_timeout 65; upstream backend { server app-service:8080; } server { listen 80; server_name localhost; location / { proxy_pass http://backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } } }1.3 在Pod中使用ConfigMap环境变量方式apiVersion: v1 kind: Pod metadata: name: my-app-pod labels: app: my-app spec: containers: - name: my-app image: myapp:v1 env: # 方式一单个环境变量 - name: APP_ENV valueFrom: configMapKeyRef: name: app-config key: APP_ENV - name: LOG_LEVEL valueFrom: configMapKeyRef: name: app-config key: LOG_LEVEL # 方式二多个环境变量 envFrom: - configMapRef: name: app-config ports: - containerPort: 8080Volume挂载方式apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 volumeMounts: # 挂载所有配置 - name: config-volume mountPath: /etc/config readOnly: true # 挂载单个文件 - name: nginx-config mountPath: /etc/nginx/nginx.conf subPath: nginx.conf readOnly: true volumes: - name: config-volume configMap: name: app-config - name: nginx-config configMap: name: app-config items: - key: nginx.conf path: nginx.conf1.4 ConfigMap高级配置带权限控制apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | api: baseUrl: https://api.example.com timeout: 30 --- apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 volumeMounts: - name: config mountPath: /app/config volumes: - name: config configMap: name: app-config defaultMode: 0644 # 文件权限 optional: false # ConfigMap必须存在二、Secret详解2.1 Secret概述Secret用于存储敏感信息如密码、OAuth令牌、SSH密钥等。Secret在etcd中以Base64编码存储提供了比ConfigMap更高的安全性。2.2 Secret类型apiVersion: v1 kind: Secret metadata: name: my-secret type: Opaque # 通用类型 data: username: dXNlcm5hbWU password: cGFzc3dvcmQ --- apiVersion: v1 kind: Secret metadata: name: tls-secret type: kubernetes.io/tls data: tls.crt: base64-encoded-cert tls.key: base64-encoded-key --- apiVersion: v1 kind: Secret metadata: name: docker-registry-secret type: kubernetes.io/dockerconfigjson data: .dockerconfigjson: base64-encoded-config2.3 创建Secret# 从文件创建 kubectl create secret generic db-credentials \ --from-fileusername./username.txt \ --from-filepassword./password.txt # 从字面值创建 kubectl create secret generic db-credentials \ --from-literalusernameadmin \ --from-literalpasswordsecret123 # 从Docker Registry创建 kubectl create secret docker-registry my-registry \ --docker-serverregistry.example.com \ --docker-usernameadmin \ --docker-passwordsecret123 \ --docker-emailadminexample.com # 从TLS证书创建 kubectl create secret tls my-tls \ --cert./tls.crt \ --key./tls.key2.4 在Pod中使用Secret环境变量方式apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 env: - name: DB_USERNAME valueFrom: secretKeyRef: name: db-credentials key: username - name: DB_PASSWORD valueFrom: secretKeyRef: name: db-credentials key: password optional: true # Secret不存在时不会导致Pod启动失败 envFrom: - secretRef: name: db-credentialsVolume挂载方式apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 volumeMounts: - name: credentials mountPath: /etc/secrets readOnly: true - name: tls-certs mountPath: /etc/tls readOnly: true volumes: - name: credentials secret: secretName: db-credentials defaultMode: 0600 optional: false - name: tls-certs secret: secretName: my-tls defaultMode: 06442.5 TLS Secret详解apiVersion: v1 kind: Secret metadata: name: my-tls-secret namespace: production type: kubernetes.io/tls data: tls.crt: | -----BEGIN CERTIFICATE----- MIIDXTCCAkWgAwIBAgIJAJC1HiIAZAiUMA0GCSqGSIb3Qw0... ...Base64编码的证书内容 -----END CERTIFICATE----- tls.key: | -----BEGIN RSA PRIVATE KEY----- MIIEowIBAAKCAQEA0Z3VS5JJcds3xfn/ygWyH8Q4h7x2n6... ...Base64编码的私钥内容 -----END RSA PRIVATE KEY-----三、ConfigMap和Secret的最佳实践3.1 使用命名空间隔离配置# 为不同环境创建独立的ConfigMap apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: development data: env: development --- apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: staging data: env: staging --- apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: production data: env: production3.2 使用版本标签apiVersion: v1 kind: ConfigMap metadata: name: app-config-v2 labels: app: my-app version: 2 lastUpdated: 2024-01-15 data: config.yaml: | api: version: v2 features: featureA: enabled featureB: disabled3.3 使用Kustomize管理配置# base/config.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - deployment.yaml - configmap.yaml - secret.yaml configMapGenerator: - name: app-config literals: - APP_ENVdefault - LOG_LEVELinfo secretGenerator: - name: db-credentials literals: - usernameadmin - passwordchangeme# overlays/production/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - ../../base configMapGenerator: - name: app-config behavior: replace literals: - APP_ENVproduction - LOG_LEVELwarn secretGenerator: - name: db-credentials behavior: replace literals: - usernameprodadmin - password替换为实际密码3.4 使用Helm管理配置# values.yaml replicaCount: 3 image: repository: myapp tag: v1.0.0 pullPolicy: IfNotPresent config: APP_ENV: production LOG_LEVEL: info MAX_CONNECTIONS: 100 secrets: DB_USERNAME: admin DB_PASSWORD: changeme # 实际使用时会被外部values覆盖# templates/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: {{ .Release.Name }}-config labels: app: {{ .Chart.Name }} version: {{ .Chart.Version }} data: {{- range $key, $value : .Values.config }} {{ $key }}: {{ $value | quote }} {{- end }}3.5 外部密钥管理集成Vault集成apiVersion: v1 kind: ServiceAccount metadata: name: my-app namespace: production --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: my-app-vault rules: - apiGroups: [] resources: [secrets] verbs: [get] --- apiVersion: v1 kind: Secret metadata: annotations: vault.security.banzaicloud.io/role: my-app vault.security.banzaicloud.io/vault-addr: https://vault.example.com name: my-app-vault-secret type: Opaque stringData: db-password: ${vault:secret/data/myapp/db#password}四、高级使用场景4.1 动态配置更新apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | rate: requests_per_second: 100 burst: 200 --- apiVersion: v1 kind: Deployment metadata: name: my-app spec: template: spec: containers: - name: my-app image: myapp:v1 volumeMounts: - name: config mountPath: /etc/config volumes: - name: config configMap: name: app-config # 启用热更新 reloadStrategy: automatic: true4.2 配置校验apiVersion: v1 kind: ConfigMap metadata: name: validated-config annotations: config.kubernetes.io/function: | exec: path: /usr/local/bin/config-validator data: settings.json: | { maxConnections: 100, timeout: 30 }五、安全最佳实践5.1 加密SecretapiVersion: v1 kind: Secret metadata: name: encrypted-secret type: Opaque data: api-key: base64-encoded encryption: provider: aescbc keyId: key15.2 限制Secret访问apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: config-reader namespace: production rules: - apiGroups: [] resources: [configmaps, secrets] verbs: [get, list, watch] resourceNames: [app-config, app-secrets]5.3 定期轮换密钥# 更新Secret kubectl create secret generic db-credentials \ --from-literalusernameadmin \ --from-literalpasswordnewpassword \ --dry-runclient -o yaml | kubectl apply -f - # 触发应用重新加载配置 kubectl rollout restart deployment/my-app -n production六、常见问题解决方案6.1 ConfigMap更新后应用不生效# 方案一使用subPath挂载不推荐动态更新 volumeMounts: - name: config mountPath: /etc/config/app.properties subPath: app.properties # 方案二重新创建Pod kubectl rollout restart deployment/my-app # 方案三使用Reloader自动检测更新 apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: reloader spec: selector: matchLabels: app: reloader6.2 Secret大小限制# Kubernetes Secret大小限制为1MB # 如果需要存储大文件考虑使用对象存储 apiVersion: v1 kind: Pod metadata: name: my-app spec: containers: - name: my-app image: myapp:v1 env: - name: LARGE_CONFIG_URL value: https://storage.example.com/configs/large-config.json6.3 多环境配置管理# base/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | common: logLevel: ${LOG_LEVEL} maxRetries: 3 --- # overlays/dev/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | common: logLevel: debug maxRetries: 3 --- # overlays/prod/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | common: logLevel: warn maxRetries: 5总结ConfigMap和Secret是Kubernetes配置管理的基石。通过合理使用这些资源结合Kustomize、Helm等工具可以实现配置的版本化管理、环境隔离和安全存储。遵循最佳实践将大大提升应用的可靠性和安全性。
ConfigMap与Secret管理完全指南
ConfigMap与Secret管理完全指南前言在Kubernetes中ConfigMap和Secret是管理应用配置的核心资源。它们允许我们将配置与容器镜像分离实现配置的动态管理和安全存储。本文将详细介绍ConfigMap和Secret的使用方法、最佳实践以及高级特性。一、ConfigMap详解1.1 ConfigMap概述ConfigMap用于存储非敏感的配置文件、命令行参数、环境变量等配置数据。它是一种键值对资源支持多种数据格式。1.2 创建ConfigMap的方式方式一命令行创建# 从文件创建 kubectl create configmap app-config --from-fileapplication.properties./config/application.properties # 从目录创建 kubectl create configmap app-config --from-file./config/ # 从环境变量文件创建 kubectl create configmap app-config --from-env-file./config/.env # 从字面值创建 kubectl create configmap app-config --from-literalAPP_ENVproduction --from-literalLOG_LEVELinfo方式二YAML定义apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: production labels: app: my-app version: v1 data: application.properties: | server.port8080 spring.application.namemy-app spring.profiles.activeproduction logging.level.rootINFO logging.level.com.myappDEBUG database.properties: | db.drivercom.mysql.cj.jdbc.Driver db.urljdbc:mysql://mysql-service:3306/mydb db.username${DB_USERNAME} db.password${DB_PASSWORD} db.pool.maxSize20 db.pool.minIdle5 nginx.conf: | worker_processes auto; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main $remote_addr - $remote_user [$time_local] $request $status $body_bytes_sent $http_referer $http_user_agent $http_x_forwarded_for; access_log /var/log/nginx/access.log main; sendfile on; keepalive_timeout 65; upstream backend { server app-service:8080; } server { listen 80; server_name localhost; location / { proxy_pass http://backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } } }1.3 在Pod中使用ConfigMap环境变量方式apiVersion: v1 kind: Pod metadata: name: my-app-pod labels: app: my-app spec: containers: - name: my-app image: myapp:v1 env: # 方式一单个环境变量 - name: APP_ENV valueFrom: configMapKeyRef: name: app-config key: APP_ENV - name: LOG_LEVEL valueFrom: configMapKeyRef: name: app-config key: LOG_LEVEL # 方式二多个环境变量 envFrom: - configMapRef: name: app-config ports: - containerPort: 8080Volume挂载方式apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 volumeMounts: # 挂载所有配置 - name: config-volume mountPath: /etc/config readOnly: true # 挂载单个文件 - name: nginx-config mountPath: /etc/nginx/nginx.conf subPath: nginx.conf readOnly: true volumes: - name: config-volume configMap: name: app-config - name: nginx-config configMap: name: app-config items: - key: nginx.conf path: nginx.conf1.4 ConfigMap高级配置带权限控制apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | api: baseUrl: https://api.example.com timeout: 30 --- apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 volumeMounts: - name: config mountPath: /app/config volumes: - name: config configMap: name: app-config defaultMode: 0644 # 文件权限 optional: false # ConfigMap必须存在二、Secret详解2.1 Secret概述Secret用于存储敏感信息如密码、OAuth令牌、SSH密钥等。Secret在etcd中以Base64编码存储提供了比ConfigMap更高的安全性。2.2 Secret类型apiVersion: v1 kind: Secret metadata: name: my-secret type: Opaque # 通用类型 data: username: dXNlcm5hbWU password: cGFzc3dvcmQ --- apiVersion: v1 kind: Secret metadata: name: tls-secret type: kubernetes.io/tls data: tls.crt: base64-encoded-cert tls.key: base64-encoded-key --- apiVersion: v1 kind: Secret metadata: name: docker-registry-secret type: kubernetes.io/dockerconfigjson data: .dockerconfigjson: base64-encoded-config2.3 创建Secret# 从文件创建 kubectl create secret generic db-credentials \ --from-fileusername./username.txt \ --from-filepassword./password.txt # 从字面值创建 kubectl create secret generic db-credentials \ --from-literalusernameadmin \ --from-literalpasswordsecret123 # 从Docker Registry创建 kubectl create secret docker-registry my-registry \ --docker-serverregistry.example.com \ --docker-usernameadmin \ --docker-passwordsecret123 \ --docker-emailadminexample.com # 从TLS证书创建 kubectl create secret tls my-tls \ --cert./tls.crt \ --key./tls.key2.4 在Pod中使用Secret环境变量方式apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 env: - name: DB_USERNAME valueFrom: secretKeyRef: name: db-credentials key: username - name: DB_PASSWORD valueFrom: secretKeyRef: name: db-credentials key: password optional: true # Secret不存在时不会导致Pod启动失败 envFrom: - secretRef: name: db-credentialsVolume挂载方式apiVersion: v1 kind: Pod metadata: name: my-app-pod spec: containers: - name: my-app image: myapp:v1 volumeMounts: - name: credentials mountPath: /etc/secrets readOnly: true - name: tls-certs mountPath: /etc/tls readOnly: true volumes: - name: credentials secret: secretName: db-credentials defaultMode: 0600 optional: false - name: tls-certs secret: secretName: my-tls defaultMode: 06442.5 TLS Secret详解apiVersion: v1 kind: Secret metadata: name: my-tls-secret namespace: production type: kubernetes.io/tls data: tls.crt: | -----BEGIN CERTIFICATE----- MIIDXTCCAkWgAwIBAgIJAJC1HiIAZAiUMA0GCSqGSIb3Qw0... ...Base64编码的证书内容 -----END CERTIFICATE----- tls.key: | -----BEGIN RSA PRIVATE KEY----- MIIEowIBAAKCAQEA0Z3VS5JJcds3xfn/ygWyH8Q4h7x2n6... ...Base64编码的私钥内容 -----END RSA PRIVATE KEY-----三、ConfigMap和Secret的最佳实践3.1 使用命名空间隔离配置# 为不同环境创建独立的ConfigMap apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: development data: env: development --- apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: staging data: env: staging --- apiVersion: v1 kind: ConfigMap metadata: name: app-config namespace: production data: env: production3.2 使用版本标签apiVersion: v1 kind: ConfigMap metadata: name: app-config-v2 labels: app: my-app version: 2 lastUpdated: 2024-01-15 data: config.yaml: | api: version: v2 features: featureA: enabled featureB: disabled3.3 使用Kustomize管理配置# base/config.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - deployment.yaml - configmap.yaml - secret.yaml configMapGenerator: - name: app-config literals: - APP_ENVdefault - LOG_LEVELinfo secretGenerator: - name: db-credentials literals: - usernameadmin - passwordchangeme# overlays/production/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - ../../base configMapGenerator: - name: app-config behavior: replace literals: - APP_ENVproduction - LOG_LEVELwarn secretGenerator: - name: db-credentials behavior: replace literals: - usernameprodadmin - password替换为实际密码3.4 使用Helm管理配置# values.yaml replicaCount: 3 image: repository: myapp tag: v1.0.0 pullPolicy: IfNotPresent config: APP_ENV: production LOG_LEVEL: info MAX_CONNECTIONS: 100 secrets: DB_USERNAME: admin DB_PASSWORD: changeme # 实际使用时会被外部values覆盖# templates/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: {{ .Release.Name }}-config labels: app: {{ .Chart.Name }} version: {{ .Chart.Version }} data: {{- range $key, $value : .Values.config }} {{ $key }}: {{ $value | quote }} {{- end }}3.5 外部密钥管理集成Vault集成apiVersion: v1 kind: ServiceAccount metadata: name: my-app namespace: production --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: my-app-vault rules: - apiGroups: [] resources: [secrets] verbs: [get] --- apiVersion: v1 kind: Secret metadata: annotations: vault.security.banzaicloud.io/role: my-app vault.security.banzaicloud.io/vault-addr: https://vault.example.com name: my-app-vault-secret type: Opaque stringData: db-password: ${vault:secret/data/myapp/db#password}四、高级使用场景4.1 动态配置更新apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | rate: requests_per_second: 100 burst: 200 --- apiVersion: v1 kind: Deployment metadata: name: my-app spec: template: spec: containers: - name: my-app image: myapp:v1 volumeMounts: - name: config mountPath: /etc/config volumes: - name: config configMap: name: app-config # 启用热更新 reloadStrategy: automatic: true4.2 配置校验apiVersion: v1 kind: ConfigMap metadata: name: validated-config annotations: config.kubernetes.io/function: | exec: path: /usr/local/bin/config-validator data: settings.json: | { maxConnections: 100, timeout: 30 }五、安全最佳实践5.1 加密SecretapiVersion: v1 kind: Secret metadata: name: encrypted-secret type: Opaque data: api-key: base64-encoded encryption: provider: aescbc keyId: key15.2 限制Secret访问apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: config-reader namespace: production rules: - apiGroups: [] resources: [configmaps, secrets] verbs: [get, list, watch] resourceNames: [app-config, app-secrets]5.3 定期轮换密钥# 更新Secret kubectl create secret generic db-credentials \ --from-literalusernameadmin \ --from-literalpasswordnewpassword \ --dry-runclient -o yaml | kubectl apply -f - # 触发应用重新加载配置 kubectl rollout restart deployment/my-app -n production六、常见问题解决方案6.1 ConfigMap更新后应用不生效# 方案一使用subPath挂载不推荐动态更新 volumeMounts: - name: config mountPath: /etc/config/app.properties subPath: app.properties # 方案二重新创建Pod kubectl rollout restart deployment/my-app # 方案三使用Reloader自动检测更新 apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: reloader spec: selector: matchLabels: app: reloader6.2 Secret大小限制# Kubernetes Secret大小限制为1MB # 如果需要存储大文件考虑使用对象存储 apiVersion: v1 kind: Pod metadata: name: my-app spec: containers: - name: my-app image: myapp:v1 env: - name: LARGE_CONFIG_URL value: https://storage.example.com/configs/large-config.json6.3 多环境配置管理# base/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | common: logLevel: ${LOG_LEVEL} maxRetries: 3 --- # overlays/dev/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | common: logLevel: debug maxRetries: 3 --- # overlays/prod/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: app-config data: config.yaml: | common: logLevel: warn maxRetries: 5总结ConfigMap和Secret是Kubernetes配置管理的基石。通过合理使用这些资源结合Kustomize、Helm等工具可以实现配置的版本化管理、环境隔离和安全存储。遵循最佳实践将大大提升应用的可靠性和安全性。
