说明在k8s环境部署Apache Zookeeper 3.9.5高可用多pod适配Hadoop。准备配置文件注意配置文件里使用了hadoop作为namespace需修改为自己期望的namespacev1.yaml内容如下# ZooKeeper 3节点集群 # 用于 HDFS NameNode HA 自动故障转移ZKFC。若用于其他环境应当修改启动命令中的hadoop字段为真实namespace# 使用官方 zookeeper:3.9.5 镜像# 3.9.x 变化基于 JDK 17配置项 ZOO_4LINEAR_ENABLE 已移除默认启用--- apiVersion: v1 kind: Service metadata: name: zookeeper labels: app: zookeeper spec: clusterIP: None# Headless Service用于 Pod DNS 发现publishNotReadyAddresses:true# 让等待方能在 Pod 未 Ready 时也解析到 IPports: - port:2181name: client - port:2888name: peer - port:3888name: election selector: app: zookeeper --- apiVersion: v1 kind: Service metadata: name: zookeeper-external labels: app: zookeeper spec: type: NodePort ports: - port:2181name: client targetPort:2181# nodePort 留空系统自动分配selector: app: zookeeper --- apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: zookeeper-pdb spec: minAvailable:2selector: matchLabels: app: zookeeper --- apiVersion: apps/v1 kind: StatefulSet metadata: name: zookeeper spec: serviceName: zookeeper replicas:3# OrderedReady默认Pod 按序启动确保前一个就绪后才启动下一个# 必须用 OrderedReady 而非 ParallelParallel 会导致 3 个 Pod 同时启动# 此时 Headless Service DNS 尚未完全传播ZK 节点找不到 Peer 无法形成# Quorum进程崩溃退出 (exit code 14)产生多次无效重启。# OrderedReady 保证 zookeeper-1 启动时 zookeeper-0 的 DNS 已就绪。podManagementPolicy: OrderedReady selector: matchLabels: app: zookeeper template: metadata: labels: app: zookeeper spec: subdomain: zookeeper priorityClassName: hadoop-namenode-high# ZK 节点分散到不同 Node保证仲裁安全affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchLabels: app: zookeeper topologyKey: kubernetes.io/hostname terminationGracePeriodSeconds:30containers: - name: zookeeper# ZK 3.9.5 官方镜像华为云镜像源# 重要3.9.5 Docker entrypoint 在使用 ZOO_SERVERS 时不写 clientPort 到 zoo.cfg# 导致 plain 客户端端口被禁用 (plaindisabled)端口 2181 不监听。# 解决方案覆盖 command手动写完整 zoo.cfg含 clientPort2181然后前台启动 ZK。image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/zookeeper:3.9.5 imagePullPolicy: IfNotPresent command: - /bin/bash --ec-|# 阶段1准备环境 set-emkdir-p/data /datalog /conf /logs# 写入完整的 zoo.cfg显式包含 clientPort2181cat/conf/zoo.cfgZKCFGtickTime2000dataDir/datadataLogDir/datalogclientPort2181initLimit10syncLimit2autopurge.snapRetainCount3autopurge.purgeInterval1server.1zookeeper-0.zookeeper.hadoop:2888:3888server.2zookeeper-1.zookeeper.hadoop:2888:3888server.3zookeeper-2.zookeeper.hadoop:2888:3888 ZKCFGecho /conf/zoo.cfg cat/conf/zoo.cfgecho /data/myid cat/data/myid# JDK 17 反射权限Hadoop ZKFC 依赖exportSERVER_JVMFLAGS--add-opens java.base/java.ioALL-UNNAMED --add-opens java.base/java.langALL-UNNAMED --add-opens java.base/java.utilALL-UNNAMED --add-opens java.base/java.netALL-UNNAMED# 阶段2SIGTERM 陷阱转发给 ZK 进程确保优雅关闭 ZK_PID_term(){echo$(date): Received SIGTERM, stopping ZooKeeper...if[-n$ZK_PID]kill-0$ZK_PID2/dev/null;thenkill-TERM$ZK_PIDwait$ZK_PID2/dev/nullfiexit0}trap_termTERMINT# 阶段3启动 ZK带重试吸收 Quorum 组建期间的偶发崩溃 # OrderedReady 保证前置 Pod 已就绪但 ZK 处于 LOOKING 状态时# readiness probe 已通过新 Pod 加入 Quorum 仍可能有瞬态时序竞争# 导致 ZK 偶发退出 (exit code 14)。通过进程内重试吸收避免 Pod 级重启。seteMAX_RETRY3RETRY0while[$RETRY-lt$MAX_RETRY];doecho$(date): Starting ZooKeeper (attempt$((RETRY1))/$MAX_RETRY)...zkServer.sh start-foregroundZK_PID$!wait$ZK_PIDEXIT$?ZK_PIDecho$(date): ZooKeeper exited with code$EXITif[$EXIT-eq0];thenecho$(date): ZooKeeper exited normallyexit0fiRETRY$((RETRY1))if[$RETRY-lt$MAX_RETRY];thenecho$(date): Will retry in 5 seconds...sleep5fidoneecho$(date): Max retries ($MAX_RETRY) exhausted, giving upexit$EXITports: - containerPort:2181name: client - containerPort:2888name: peer - containerPort:3888name: election resources: requests: memory:1Gicpu:500mlimits: memory:2Gicpu:1000m# startupProbeZK 3.9.5 JDK 17 启动较慢需要充足启动时间# startupProbe 通过后liveness/readiness 才开始工作startupProbe: tcpSocket: port:2181initialDelaySeconds:10periodSeconds:10timeoutSeconds:3failureThreshold:30# 10 10×30 310s 最大启动容忍livenessProbe: tcpSocket: port:2181periodSeconds:15timeoutSeconds:3failureThreshold:5# 75s 容忍startup 通过后 ZK 已稳定readinessProbe: tcpSocket: port:2181periodSeconds:10timeoutSeconds:3failureThreshold:3volumeMounts: - name: zookeeper-data mountPath: /data# initContainer从 Pod 名提取序号写入 /data/myid# zoo.cfg 由主容器的 command 直接写入 /conf/zoo.cfg绕过有 bug 的 Docker entrypointinitContainers: - name: fix-myid image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/zookeeper:3.9.5 imagePullPolicy: IfNotPresent command:[/bin/bash,-c]args: -|# 从 Pod 名提取序号zookeeper-0 → 1zookeeper-1 → 2zookeeper-2 → 3POD_NAME$(hostname)ORD${POD_NAME##*-}MY_ID$((ORD1))echoPod:$POD_NAME- ZOO_MY_ID$MY_IDecho$MY_ID/data/myidechoWritten myid$MY_IDto /data/myidcat/data/myid volumeMounts: - name: zookeeper-data mountPath: /data volumeClaimTemplates: - metadata: name: zookeeper-data spec: accessModes:[ReadWriteOnce]# 不指定 storageClassName自动使用集群默认 StorageClassresources: requests: storage: 10Gi部署再次提醒我部署到了namespace hadoop里需改为你期望的namespace包括文件中配置的hadoop字段都替换。kubectl apply-fv1.yaml-nhadoop
在k8s环境部署Apache zookeeper3.9.5，高可用，多pod
说明在k8s环境部署Apache Zookeeper 3.9.5高可用多pod适配Hadoop。准备配置文件注意配置文件里使用了hadoop作为namespace需修改为自己期望的namespacev1.yaml内容如下# ZooKeeper 3节点集群 # 用于 HDFS NameNode HA 自动故障转移ZKFC。若用于其他环境应当修改启动命令中的hadoop字段为真实namespace# 使用官方 zookeeper:3.9.5 镜像# 3.9.x 变化基于 JDK 17配置项 ZOO_4LINEAR_ENABLE 已移除默认启用--- apiVersion: v1 kind: Service metadata: name: zookeeper labels: app: zookeeper spec: clusterIP: None# Headless Service用于 Pod DNS 发现publishNotReadyAddresses:true# 让等待方能在 Pod 未 Ready 时也解析到 IPports: - port:2181name: client - port:2888name: peer - port:3888name: election selector: app: zookeeper --- apiVersion: v1 kind: Service metadata: name: zookeeper-external labels: app: zookeeper spec: type: NodePort ports: - port:2181name: client targetPort:2181# nodePort 留空系统自动分配selector: app: zookeeper --- apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: zookeeper-pdb spec: minAvailable:2selector: matchLabels: app: zookeeper --- apiVersion: apps/v1 kind: StatefulSet metadata: name: zookeeper spec: serviceName: zookeeper replicas:3# OrderedReady默认Pod 按序启动确保前一个就绪后才启动下一个# 必须用 OrderedReady 而非 ParallelParallel 会导致 3 个 Pod 同时启动# 此时 Headless Service DNS 尚未完全传播ZK 节点找不到 Peer 无法形成# Quorum进程崩溃退出 (exit code 14)产生多次无效重启。# OrderedReady 保证 zookeeper-1 启动时 zookeeper-0 的 DNS 已就绪。podManagementPolicy: OrderedReady selector: matchLabels: app: zookeeper template: metadata: labels: app: zookeeper spec: subdomain: zookeeper priorityClassName: hadoop-namenode-high# ZK 节点分散到不同 Node保证仲裁安全affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchLabels: app: zookeeper topologyKey: kubernetes.io/hostname terminationGracePeriodSeconds:30containers: - name: zookeeper# ZK 3.9.5 官方镜像华为云镜像源# 重要3.9.5 Docker entrypoint 在使用 ZOO_SERVERS 时不写 clientPort 到 zoo.cfg# 导致 plain 客户端端口被禁用 (plaindisabled)端口 2181 不监听。# 解决方案覆盖 command手动写完整 zoo.cfg含 clientPort2181然后前台启动 ZK。image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/zookeeper:3.9.5 imagePullPolicy: IfNotPresent command: - /bin/bash --ec-|# 阶段1准备环境 set-emkdir-p/data /datalog /conf /logs# 写入完整的 zoo.cfg显式包含 clientPort2181cat/conf/zoo.cfgZKCFGtickTime2000dataDir/datadataLogDir/datalogclientPort2181initLimit10syncLimit2autopurge.snapRetainCount3autopurge.purgeInterval1server.1zookeeper-0.zookeeper.hadoop:2888:3888server.2zookeeper-1.zookeeper.hadoop:2888:3888server.3zookeeper-2.zookeeper.hadoop:2888:3888 ZKCFGecho /conf/zoo.cfg cat/conf/zoo.cfgecho /data/myid cat/data/myid# JDK 17 反射权限Hadoop ZKFC 依赖exportSERVER_JVMFLAGS--add-opens java.base/java.ioALL-UNNAMED --add-opens java.base/java.langALL-UNNAMED --add-opens java.base/java.utilALL-UNNAMED --add-opens java.base/java.netALL-UNNAMED# 阶段2SIGTERM 陷阱转发给 ZK 进程确保优雅关闭 ZK_PID_term(){echo$(date): Received SIGTERM, stopping ZooKeeper...if[-n$ZK_PID]kill-0$ZK_PID2/dev/null;thenkill-TERM$ZK_PIDwait$ZK_PID2/dev/nullfiexit0}trap_termTERMINT# 阶段3启动 ZK带重试吸收 Quorum 组建期间的偶发崩溃 # OrderedReady 保证前置 Pod 已就绪但 ZK 处于 LOOKING 状态时# readiness probe 已通过新 Pod 加入 Quorum 仍可能有瞬态时序竞争# 导致 ZK 偶发退出 (exit code 14)。通过进程内重试吸收避免 Pod 级重启。seteMAX_RETRY3RETRY0while[$RETRY-lt$MAX_RETRY];doecho$(date): Starting ZooKeeper (attempt$((RETRY1))/$MAX_RETRY)...zkServer.sh start-foregroundZK_PID$!wait$ZK_PIDEXIT$?ZK_PIDecho$(date): ZooKeeper exited with code$EXITif[$EXIT-eq0];thenecho$(date): ZooKeeper exited normallyexit0fiRETRY$((RETRY1))if[$RETRY-lt$MAX_RETRY];thenecho$(date): Will retry in 5 seconds...sleep5fidoneecho$(date): Max retries ($MAX_RETRY) exhausted, giving upexit$EXITports: - containerPort:2181name: client - containerPort:2888name: peer - containerPort:3888name: election resources: requests: memory:1Gicpu:500mlimits: memory:2Gicpu:1000m# startupProbeZK 3.9.5 JDK 17 启动较慢需要充足启动时间# startupProbe 通过后liveness/readiness 才开始工作startupProbe: tcpSocket: port:2181initialDelaySeconds:10periodSeconds:10timeoutSeconds:3failureThreshold:30# 10 10×30 310s 最大启动容忍livenessProbe: tcpSocket: port:2181periodSeconds:15timeoutSeconds:3failureThreshold:5# 75s 容忍startup 通过后 ZK 已稳定readinessProbe: tcpSocket: port:2181periodSeconds:10timeoutSeconds:3failureThreshold:3volumeMounts: - name: zookeeper-data mountPath: /data# initContainer从 Pod 名提取序号写入 /data/myid# zoo.cfg 由主容器的 command 直接写入 /conf/zoo.cfg绕过有 bug 的 Docker entrypointinitContainers: - name: fix-myid image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/zookeeper:3.9.5 imagePullPolicy: IfNotPresent command:[/bin/bash,-c]args: -|# 从 Pod 名提取序号zookeeper-0 → 1zookeeper-1 → 2zookeeper-2 → 3POD_NAME$(hostname)ORD${POD_NAME##*-}MY_ID$((ORD1))echoPod:$POD_NAME- ZOO_MY_ID$MY_IDecho$MY_ID/data/myidechoWritten myid$MY_IDto /data/myidcat/data/myid volumeMounts: - name: zookeeper-data mountPath: /data volumeClaimTemplates: - metadata: name: zookeeper-data spec: accessModes:[ReadWriteOnce]# 不指定 storageClassName自动使用集群默认 StorageClassresources: requests: storage: 10Gi部署再次提醒我部署到了namespace hadoop里需改为你期望的namespace包括文件中配置的hadoop字段都替换。kubectl apply-fv1.yaml-nhadoop
