5个实战案例带你玩转多智能体强化学习附Python代码当AlphaGo战胜人类围棋冠军时单智能体强化学习展现了惊人潜力。但现实世界远非单打独斗的棋局——从自动驾驶车流协调到工业机器人集群协作多智能体系统才是常态。本文将带您深入五个典型场景用Python代码揭开多智能体强化学习的神秘面纱。1. 双足足球机器人协同进攻足球场上11名队员的配合堪称多智能体协作的典范。我们简化场景用两个智能体模拟前锋配合突破防守的战术。import numpy as np from pettingzoo import football env football.env() observations env.reset() for agent in env.agent_iter(): obs, reward, done, info env.last() action policy(obs) if not done else None env.step(action) env.render()关键参数调优协同奖励系数0.7鼓励传球配合个人突破奖励0.3保持个体能力视野范围90度模拟真实视角注意使用PettingZoo环境时需确保所有智能体都执行完动作再更新环境2. 仓储机器人路径规划四台AGV小车在100×100网格仓库中协同搬运需避免碰撞并优化总运输时间。我们采用MADDPG算法实现分布式决策class MADDPG: def __init__(self, state_dim, action_dim, num_agents): self.actors [Actor(state_dim, action_dim) for _ in range(num_agents)] self.critics [Critic(state_dim*num_agents, action_dim*num_agents) for _ in range(num_agents)] def update(self, transitions): # 集中式训练分布式执行 states torch.cat([t.state for t in transitions]) actions torch.cat([t.action for t in transitions]) for i, agent in enumerate(self.agents): q_values self.critics[i](states, actions) policy_loss -q_values.mean() self.actors[i].optimizer.zero_grad() policy_loss.backward() self.actors[i].optimizer.step()实验数据对比算法平均完成时间碰撞次数独立Q学习328s17MADDPG241s3集中式控制255s03. 智能电网负荷分配三个发电单元需要动态满足区域用电需求我们设计基于博弈论的奖励机制基础奖励发电量-需求匹配度协同惩罚过载系数×0.5效率奖励(1-燃料消耗率)×0.3def calculate_rewards(agents, demand): total_generation sum(a.generation for a in agents) imbalance abs(total_generation - demand) rewards [] for agent in agents: # 个体奖励组成 reward (agent.efficiency * 0.3 - max(0, total_generation - demand) * 0.5 / len(agents)) rewards.append(reward) return np.array(rewards)4. 多无人机区域巡查五架无人机协同巡查山区需覆盖所有重点区域且避免重复。采用分层强化学习架构高层决策器分配区域每50步决策一次底层控制器路径规划每步决策class HierarchicalAgent: def __init__(self): self.high_level PPONetwork(obs_dimglobal_obs_dim, act_dimn_regions) self.low_level [DQN(obs_dimlocal_obs_dim, act_dim4) for _ in range(n_drones)] def act(self, observations): if step % 50 0: # 高层决策时机 region_assignments self.high_level(global_obs) actions [] for i, drone_obs in enumerate(observations): action self.low_level[i](drone_obs) actions.append(action) return actions5. 虚拟股市交易员博弈模拟20个交易智能体在虚拟股市中的博弈采用对手建模技术class OpponentModelingAgent: def __init__(self, n_agents): self.policy_net PolicyNetwork() self.opponent_models [LSTMPredictor() for _ in range(n_agents-1)] def predict_opponent_actions(self, history): predicted_actions [] for model in self.opponent_models: pred model(history[-10:]) # 使用最近10步历史 predicted_actions.append(pred) return predicted_actions def act(self, private_obs, public_obs, history): opponent_actions self.predict_opponent_actions(history) combined_state torch.cat([private_obs, public_obs] opponent_actions) return self.policy_net(combined_state)性能提升技巧对手动作预测准确率提升30%后采用课程学习先固定策略对手再逐步增加复杂度引入元学习每周交易日后更新模型参数
5个实战案例带你玩转多智能体强化学习（附Python代码）
5个实战案例带你玩转多智能体强化学习附Python代码当AlphaGo战胜人类围棋冠军时单智能体强化学习展现了惊人潜力。但现实世界远非单打独斗的棋局——从自动驾驶车流协调到工业机器人集群协作多智能体系统才是常态。本文将带您深入五个典型场景用Python代码揭开多智能体强化学习的神秘面纱。1. 双足足球机器人协同进攻足球场上11名队员的配合堪称多智能体协作的典范。我们简化场景用两个智能体模拟前锋配合突破防守的战术。import numpy as np from pettingzoo import football env football.env() observations env.reset() for agent in env.agent_iter(): obs, reward, done, info env.last() action policy(obs) if not done else None env.step(action) env.render()关键参数调优协同奖励系数0.7鼓励传球配合个人突破奖励0.3保持个体能力视野范围90度模拟真实视角注意使用PettingZoo环境时需确保所有智能体都执行完动作再更新环境2. 仓储机器人路径规划四台AGV小车在100×100网格仓库中协同搬运需避免碰撞并优化总运输时间。我们采用MADDPG算法实现分布式决策class MADDPG: def __init__(self, state_dim, action_dim, num_agents): self.actors [Actor(state_dim, action_dim) for _ in range(num_agents)] self.critics [Critic(state_dim*num_agents, action_dim*num_agents) for _ in range(num_agents)] def update(self, transitions): # 集中式训练分布式执行 states torch.cat([t.state for t in transitions]) actions torch.cat([t.action for t in transitions]) for i, agent in enumerate(self.agents): q_values self.critics[i](states, actions) policy_loss -q_values.mean() self.actors[i].optimizer.zero_grad() policy_loss.backward() self.actors[i].optimizer.step()实验数据对比算法平均完成时间碰撞次数独立Q学习328s17MADDPG241s3集中式控制255s03. 智能电网负荷分配三个发电单元需要动态满足区域用电需求我们设计基于博弈论的奖励机制基础奖励发电量-需求匹配度协同惩罚过载系数×0.5效率奖励(1-燃料消耗率)×0.3def calculate_rewards(agents, demand): total_generation sum(a.generation for a in agents) imbalance abs(total_generation - demand) rewards [] for agent in agents: # 个体奖励组成 reward (agent.efficiency * 0.3 - max(0, total_generation - demand) * 0.5 / len(agents)) rewards.append(reward) return np.array(rewards)4. 多无人机区域巡查五架无人机协同巡查山区需覆盖所有重点区域且避免重复。采用分层强化学习架构高层决策器分配区域每50步决策一次底层控制器路径规划每步决策class HierarchicalAgent: def __init__(self): self.high_level PPONetwork(obs_dimglobal_obs_dim, act_dimn_regions) self.low_level [DQN(obs_dimlocal_obs_dim, act_dim4) for _ in range(n_drones)] def act(self, observations): if step % 50 0: # 高层决策时机 region_assignments self.high_level(global_obs) actions [] for i, drone_obs in enumerate(observations): action self.low_level[i](drone_obs) actions.append(action) return actions5. 虚拟股市交易员博弈模拟20个交易智能体在虚拟股市中的博弈采用对手建模技术class OpponentModelingAgent: def __init__(self, n_agents): self.policy_net PolicyNetwork() self.opponent_models [LSTMPredictor() for _ in range(n_agents-1)] def predict_opponent_actions(self, history): predicted_actions [] for model in self.opponent_models: pred model(history[-10:]) # 使用最近10步历史 predicted_actions.append(pred) return predicted_actions def act(self, private_obs, public_obs, history): opponent_actions self.predict_opponent_actions(history) combined_state torch.cat([private_obs, public_obs] opponent_actions) return self.policy_net(combined_state)性能提升技巧对手动作预测准确率提升30%后采用课程学习先固定策略对手再逐步增加复杂度引入元学习每周交易日后更新模型参数
