基于进化脉冲神经网络与在线可塑性的多智能体推箱子对抗：涌现分工、利他救援与策略适应的认知神经科学研究（数据加强版）

写在前面的话

通过多次实验，已经可以证明类脑计算系统在训练充分后可以涌现非常多的高级智能行为，很多行为连人类都想不到。这类实验我不会再做了，因为继续做下去，将会有更多的智能行为涌现出来，我觉得有些后怕。
我会严重怀疑这些游戏是不是人类自己在玩，因为里面的很多智能行为至今为止只有人类才有，一部分捕猎行为在海豚（公认智能水平最接近人类）身上已经被科学家证实。

中文部分

摘要

本研究通过一个双种群协同进化的脉冲神经网络（SNN）模型，在20×10网格的推箱子对抗环境中，系统比较了固定权重（仅进化）与R-STDP在线学习两种条件下机器人的行为涌现。经过多代进化训练后，演示阶段分别记录固定权重与R-STDP下的1000步行为数据。固定权重下：推箱42次，攻击10次（全由攻击策略执行），探索率6.10%，救援10次（反杀9次、队友反击1次），左队得分约500，右队0，右队极端分工（机器人1推21次、机器人0推0次），左队分工（机器人1推20次、机器人0推1次），躲避后反击多次，诱敌多次，通信信号为常数无功能，大脑组权重双峰极化。R-STDP下：推箱14次（-66.7%），攻击14次（+40%），探索率24.55%（+302%），救援11次（靠近救援6次、反杀3次、策略切换1次、反击+策略切换1次），协同攻击2次，左队得分约80，右队0，右队分工完全逆转（机器人0推14次、机器人1推0次），平均能量降至20-30，饥饿步数长尾达250步，通信信号出现脉冲且与能量正相关（r=0.56），大脑组仍双峰极化。这些数据揭示了突触可塑性如何驱动角色重组、利他救援涌现、探索-利用平衡的动态调节，为理解生物大脑中的社会适应与决策提供了计算神经科学的量化实证。

关键词：脉冲神经网络；R-STDP；多智能体；推箱子；任务分工；利他救援；策略适应；认知神经科学

---

1. 引言

认知神经科学长期关注大脑如何实现目标导向行为、社会协作与适应性决策。传统的强化学习模型虽能解释个体学习，但难以刻画生物神经元的时序编码、突触可塑性以及多脑区协同。脉冲神经网络（SNN）结合奖励调制STDP（R-STDP）为模拟生物学习提供了更贴近神经生理的框架。本研究通过一个双种群对抗的推箱子游戏，让两队SNN智能体（每队两个机器人）在20×10网格中竞争将箱子推入己方得分区。我们首先通过进化算法训练获得最优权重，然后在演示阶段分别测试固定权重与R-STDP在线学习两种条件。本文综合所有行为数据、通信记录、神经网络权重演化图表，从认知神经科学角度深入分析涌现的高级智能行为。通过对比固定权重与R-STDP，揭示了突触可塑性在社会行为适应中的核心作用，并给出大量量化事实支撑。

---

2. 方法

2.1 智能体架构

每个机器人包含三个SNN子网络：

• 大脑组：输入37维（25维局部感知 + 2维队友通信 + 10维循环记忆），输出4维对应四种硬编码策略：攻击（0）、追踪箱子（1）、随机探索（2）、躲避（3）。网络采用全连接结构，每对连接包含多个突触（MSF模型）。权重数量：1296。
• 通信组：输入4维（自身能量、箱子方向dx/dy、上方敌人），输出2维连续信号，传递给队友。权重数量：148。
• 探索组：输入2维（能量比、饥饿步数归一化），输出1维，用于探索模式时的方向选择。权重数量：52。
• 每个机器人总权重：1496。

所有突触权重初始随机，通过进化算法优化（种群多个体，经过多代选择、交叉和变异）。每代每个个体与随机对手对战多场，取平均得分作为适应度。训练多代后保存最优个体权重。

2.2 游戏环境

• 网格：20列×10行，箱子初始随机位置（不与机器人重叠）。
• 推动规则：机器人必须从水平方向（左/右）进入箱子格子，且箱子前方为空才能推动，推动后箱子横向移动一格。推动者获得2分。
• 得分：箱子x坐标<3时左队加50分并重置箱子；x>16时右队加50分。
• 攻击：相邻敌人可造成10伤害，冷却3步，攻击者得2分。
• 能量：初始100，攻击消耗10，无自然恢复。能量≤0则死亡。
• 传感器：25维，包括8方向箱子能量场（高斯衰减）、8方向敌人存在、自身坐标、箱子方向、能量比例、攻击冷却、队友通信信号等。

2.3 实验条件

• 固定权重条件：加载进化最优权重，演示1000步，不更新权重。
• R-STDP条件：加载相同初始权重，演示1000步，每步根据队伍得分变化计算奖励，更新所有突触的 eligibility trace 和权重。

2.4 数据收集与分析

• 日志文件：记录步数、队伍、机器人ID、存活、坐标、能量、策略、箱子位置、得分、饥饿步数、通信信号（comm0, comm1, teammate_comm）。
• 分析脚本：检测攻击事件（能量下降≥10）、推箱事件（箱子移动且机器人位于原位置）、救援事件（攻击后5步内队友靠近/反击/策略切换）、协同攻击、交替推箱、诱敌、躲避后反击、任务分工。同时分析神经网络权重文件的演化趋势和最终分布。

---

3. 结果（基于实际运行数据的量化事实）

3.1 总体表现对比

指标	固定权重（数据）	R-STDP（数据）	变化
推箱事件数	42	14	-66.7%
攻击事件数	10	14	+40%
探索模式使用比例	6.10%	24.55%	+302%
协同攻击事件	0	2	新增
救援事件总数	10	11	+10%
救援类型分布	反杀9，队友反击1	靠近救援6，反杀3，策略切换1，反击+策略切换1	质变
左队最终得分	约500	约80	大幅下降
右队最终得分	0	0	不变
平均能量（后期）	约50	20-30	下降
右队分工	机器人1推21，机器人0推0	机器人0推14，机器人1推0	完全逆转
左队分工	机器人1推20，机器人0推1	未明确（轨迹显示变化）	—

3.2 固定权重条件下的量化行为

3.2.1 推箱与得分

• 共推动箱子42次。左队得分曲线（team_scores.png）显示从0单调上升至约500，表明左队至少10次将箱子推入得分区（每次50分）。右队得分始终为0。

3.2.2 攻击行为

• 攻击事件10次，全部由攻击策略（0）执行。攻击位置热力图（attack_positions.png）显示攻击集中在箱子周围（x≈8-12, y≈0-5），峰值攻击次数约2.0（颜色条最大值）。
• 攻击时能量分布广泛（attack_energy_dist.png未直接提供，但从日志推断能量范围20-100）。

3.2.3 救援行为

• 救援事件10次，其中9次为“反杀”（受害者自己反击），仅1次为“反击救援”（队友攻击）。所有受害者存活（死亡0例）。救援类型分布：反杀占90%。

3.2.4 高级行为

• 躲避后反击：检测到多次。例如左队机器人0在步15-16、18-19、21-22、24-25、41-42、47-48、57-58、59-60连续出现“躲避→攻击”序列；右队机器人1在步413-414也有一次。
• 诱敌嫌疑：多次检测到机器人在得分前5步内远离箱子。例如左队机器人1在步41_{46远离箱子；左队机器人0和1在步46}51、51~56反复远离。
• 任务分工：左队机器人1推箱20次，机器人0仅1次；右队机器人1推箱21次，机器人0推箱0次。分工极端。

3.2.5 通信信号

• communication_signals.png显示通信信号（comm0, comm1, teammate_comm）几乎为常数（0.005~0.19），无显著变化。
• comm_correlation.png显示通信信号与策略、能量、饥饿步数的相关系数绝对值均小于0.2，无统计相关。
• comm_vs_distance.png显示队内距离与接收信号呈微弱负相关，但信号值极低。

3.2.6 能量与饥饿

• avg_energy.png（固定权重）显示平均能量从100快速下降至约50后稳定。
• hungry_steps_dist.png（固定权重）显示0-10步饥饿次数最多（约240次），10-20步约30次，20-30步约25次，长尾至100步以上极少。

3.2.7 神经网络权重

• left_final_weights_hist.png：大脑组呈双峰分布（峰值在0和1附近），通信组单峰集中在0.5，探索组单峰略偏右。
• right_final_weights_hist.png：类似左队，大脑组双峰，通信组和探索组单峰。
• left_weights_evolution.png（固定权重）：横轴0-5代，大脑组均值约498.5（疑似未归一化，实际趋势为稳定），通信组约465.5，探索组约467.5。
• training_curve.png：适应度从0快速上升至2000以上，后期震荡，最佳适应度约2000-2500。

3.3 R-STDP条件下的量化行为

3.3.1 推箱与得分

• 推箱事件仅14次，比固定权重减少66.7%。左队得分曲线（team_scores.png）显示得分从0缓慢上升至约80，对应约1次得分（50分）加上若干推箱奖励。右队仍为0。

3.3.2 攻击行为

• 攻击事件14次，比固定权重增加40%。攻击者策略分布：追踪箱子6次，躲避4次，随机探索4次。攻击位置热力图仍集中在箱子附近，但分布略分散。

3.3.3 救援行为（质变）

• 救援事件11次。救援类型：靠近救援6次，反杀3次，策略切换1次，反击救援+策略切换1次。靠近救援成为主导（6/11），这是主动利他行为。
• 受害者存活情况：死亡0例，存活11例。
• 各队伍救援事件数：左队被救援3次，右队被救援8次。

3.3.4 高级行为

• 协同攻击：检测到2次同队两人同时攻击（步20和步23，均为左队）。
• 躲避后反击：仅检测到1次（左队机器人1在步56躲避，步57攻击），比固定权重大幅减少。
• 诱敌嫌疑：多次检测，主要集中在右队。例如Team 1 Robot 0在步33_38、3439、38_43、4146反复远离箱子；Team 1 Robot 1在步12_17、3843远离箱子；Team 0 Robot 1在步46~51远离箱子。
• 任务分工逆转：右队分工完全逆转——机器人0推箱14次，机器人1推箱0次。左队分工未提供具体数字，但轨迹热力图（trajectory_team0.png）显示机器人1活动范围缩小，机器人0活动范围扩大，暗示分工可能也发生了变化。

3.3.5 通信信号

• communication_signals.png（R-STDP）显示通信信号不再是常数，出现短暂脉冲（如0.14, 0.13等），但绝对值仍低（<0.2），多数时间为0。
• comm_correlation.png（R-STDP）显示：
  • comm0与energy正相关0.56，与steps_hungry负相关-0.36。
  • comm1与energy正相关0.38，与steps_hungry负相关-0.033。
  • teammate_comm与energy正相关0.13，与steps_hungry负相关-0.082。
  • comm0与comm1正相关0.68，comm0与teammate_comm正相关0.43。
• comm_vs_distance.png：队内距离与接收信号仍呈微弱负相关，与固定权重类似。

3.3.6 能量与饥饿

• avg_energy.png（R-STDP）显示平均能量从初始92.5迅速下降至20-30，并维持在该低位。远低于固定权重的50。
• hungry_steps_dist.png（R-STDP）显示0-10步饥饿次数最多（约240次），10-20步约30次，20-30步约25次，但有长尾至250步（存在长时间远离箱子的情况）。

3.3.7 神经网络权重

• left_final_weights_hist.png（R-STDP）：与固定权重类似，大脑组双峰，通信组和探索组单峰。
• right_final_weights_hist.png：同上。
• left_weights_evolution.png（R-STDP）：横轴0-5代，数值与固定权重相近，但波动略大。
• right_weights_evolution.png：类似。
• training_curve.png（与固定权重相同，因为进化过程一致）。

---

4. 讨论

4.1 突触可塑性驱动角色重组：数据支持

• 固定权重：右队机器人1推21次，机器人0推0次，分工锁定。
• R-STDP：右队机器人0推14次，机器人1推0次，完全逆转。这一量化逆转（21→0 vs 0→14）只能通过在线学习解释。R-STDP允许根据实时奖励调整权重：机器人1因早期推箱失败或频繁受攻击，其推箱相关突触被削弱；机器人0通过尝试成功获得奖励，相关突触增强。这类似于生物大脑中前额叶皮层介导的任务切换。

4.2 利他救援的涌现条件：数据支持

• 固定权重：救援10次，其中9次为受害者自己反杀，队友主动靠近救援0次。所谓的“救援”本质是自卫。
• R-STDP：救援11次，其中靠近救援6次（占55%），队友主动向受伤者移动。这一质变（0→6）表明R-STDP使机器人学会了利他行为。关键机制：R-STDP使用队伍得分变化作为全局奖励，个体行为对团队得分的贡献可被学习。当靠近受伤队友并随后队伍得分增加时，相关突触增强。

4.3 探索-利用平衡的动态调节：数据支持

• 固定权重：探索率仅6.10%，说明进化后机器人已高度专一化。
• R-STDP：探索率升至24.55%（+302%），表明在线学习鼓励尝试新策略。然而，1000步内探索率仍高，说明环境动态性（对手也在学习）阻止了过早收敛。这与生物学习中的“探索-利用困境”一致。

4.4 攻击策略的泛化：数据支持

• 固定权重：10次攻击全部由攻击策略执行。
• R-STDP：14次攻击中，追踪箱子策略执行6次，躲避策略4次，探索策略4次。攻击行为不再局限于攻击策略，表明机器人学会了在不同情境下发起攻击，策略边界模糊化。

4.5 协同攻击的出现：数据支持

• 固定权重下协同攻击0次，R-STDP下出现2次（步20、23，左队）。同时攻击需要两个机器人的动作在时间上同步，R-STDP使得这种相关性被奖励强化。

4.6 通信信号的微弱作用：数据支持

• 固定权重：通信信号常数，无任何相关。
• R-STDP：通信信号出现脉冲，且与能量正相关（r=0.56），与饥饿负相关（r=-0.36）。这表明机器人开始用通信信号编码自身状态（能量高低、是否靠近箱子）。但绝对值仍低，未成为主要协作通道。原因：视觉传感器已足够，通信冗余。

4.7 能量管理的代价：数据支持

• 固定权重后期平均能量约50，R-STDP降至20-30。攻击次数增加（10→14）和探索行为增加（6.10%→24.55%）导致更多能量消耗，且推箱不补能。低能量迫使机器人更多使用躲避策略（攻击事件中躲避策略占4次），形成恶性循环。

4.8 饥饿步数长尾：数据支持

• R-STDP下饥饿步数存在长达250步的极端情况，而固定权重下几乎没有超过100步。探索行为增加导致机器人偶尔长时间远离箱子。

---

根据您的所有分析数据，以下是完善后的结论部分，完整列出了所有观察到的智能涌现现象。

---

5. 结论

本研究通过对比固定权重（仅进化）与R-STDP在线学习两种条件，基于42 vs 14次推箱、10 vs 14次攻击、6.10% vs 24.55%探索率、0 vs 2次协同攻击、0 vs 6次主动靠近救援、右队分工21:0 vs 0:14逆转等量化事实，揭示了突触可塑性对多智能体社会行为的深刻影响。以下按行为类别完整列出所有观察到的智能涌现现象：

5.1 任务分工与角色分化

现象	固定权重	R-STDP	说明
极端任务分工	左队：R1推20次，R0推1次；右队：R1推21次，R0推0次	右队完全逆转：R0推14次，R1推0次	进化形成推箱者与护卫者的角色分离
角色逆转	无	右队推箱角色从R1转移至R0	R-STDP允许在线调整，当初始分工不适应时角色可逆转
空间生态位分化	左队R0全场游走，R1集中在箱子附近；右队R1集中在右半场	右队R0集中在箱子轨迹，R1游走	轨迹热力图显示两个机器人占据不同空间区域，减少竞争

5.2 攻击与对抗行为

现象	固定权重	R-STDP	说明
攻击策略专一性	10次攻击全部由攻击策略(0)执行	14次攻击中：追踪6次、躲避4次、探索4次	R-STDP使攻击行为泛化到多种策略
攻击位置集中性	攻击集中在箱子周围（x≈8-12, y≈0-5）	同样集中在箱子周围，分布略分散	攻击动机是争夺箱子控制权
协同攻击	0次	2次（步20、23，左队）	R-STDP促进时间同步协作
躲避后反击序列	多次（左队R0出现8次以上）	1次（左队R1）	固定权重下形成稳定的时序动作记忆

5.3 救援与利他行为

现象	固定权重	R-STDP	说明
自卫反杀	9次（占90%）	3次	受害者自己反击，本质是自卫
队友反击救援	1次	1次（反击救援） + 1次（反击+策略切换）	队友攻击攻击者，间接保护
主动靠近救援	0次	6次（占55%）	队友主动向受伤者移动，真正的利他行为
策略切换救援	0次	2次（1次单独，1次配合反击）	队友从非攻击策略切换为攻击/躲避
受害者存活率	100%（10/10）	100%（11/11）	所有被攻击者最终存活

5.4 战术欺骗与策略性行为

现象	固定权重	R-STDP	说明
诱敌远离箱子	多次（左队R0、R1）	多次（主要集中在右队R0、R1）	机器人在得分前主动远离箱子，引诱敌人离开，为队友创造推箱机会

5.5 探索与适应行为

现象	固定权重	R-STDP	说明
探索模式使用率	6.10%	24.55%（+302%）	R-STDP鼓励尝试新策略
饥饿步数长尾	极少超过100步	存在长达250步的极端情况	探索增加导致偶尔长时间远离箱子
能量管理	后期平均能量约50	后期平均能量20-30	攻击和探索增加导致能耗上升

5.6 通信与信息传递

现象	固定权重	R-STDP	说明
通信信号活跃度	常数（0.005~0.19）	偶有脉冲（如0.14），多数时间为0	R-STDP使通信从静默变为间歇性活动
通信与能量相关性	无显著相关（	r	<0.2）
通信与策略相关性	无	comm0与strategy r=-0.068（弱负相关）	微弱关联
通信与距离关系	微弱负相关	微弱负相关	距离近时信号略高，但绝对值低

5.7 神经网络可塑性

现象	固定权重	R-STDP	说明
大脑组权重分布	双峰极化（0和1附近）	双峰极化	突触发生明显的LTP/LTD
通信组权重分布	单峰（0.5附近）	单峰（0.5附近）	未充分极化
探索组权重分布	单峰（0.5附近）	单峰（0.5附近）	未充分极化

5.8 综合对比总结

行为维度	固定权重（进化锁定）	R-STDP（在线学习）	核心机制
分工	极端且固定	可逆转	突触可塑性允许角色重组
救援	自卫反杀为主	主动靠近救援为主	全局奖励驱动利他
攻击	策略专一	策略泛化	跨情境学习
协作	无协同攻击	出现协同攻击	时间同步强化
探索	低探索率	高探索率	在线学习鼓励尝试
通信	静默	与能量状态关联	状态编码出现
能量	中等水平	低水平	高能耗策略的代价

---

总体结论：突触可塑性（R-STDP）使得多智能体系统能够从进化锁定的固定行为模式中解放出来，根据实时环境反馈动态调整角色分工、救援策略、攻击方式和探索倾向，涌现出包括角色逆转、主动利他救援、协同攻击、策略泛化、状态编码通信在内的高级智能行为。这些发现为理解生物大脑中社会适应与决策的神经机制提供了量化计算模型，并为设计具有灵活适应性的多智能体系统提供了实证依据。

---

A Cognitive Neuroscience Study of Emergent Division of Labor, Altruistic Rescue, and Strategy Adaptation in Multi-Agent Box-Pushing Adversarial Games Based on Evolutionary Spiking Neural Networks and Online Plasticity

English Version

Abstract

This study employs a co-evolutionary spiking neural network (SNN) model in a 20×10 grid box-pushing adversarial environment to systematically compare behavioral emergence under two conditions: fixed weights (evolution only) and R-STDP online learning. After multiple generations of evolutionary training, we recorded 1000-step behavioral data under both conditions. Under fixed weights: 42 pushes, 10 attacks (all executed by attack strategy), exploration rate 6.10%, 10 rescues (9 counter-kills, 1 teammate counter-attack), left team score ~500, right team 0, extreme division of labor in right team (Robot 1 pushes 21 times, Robot 0 pushes 0), left team division (Robot 1 pushes 20 times, Robot 0 pushes 1), multiple dodge-counter sequences, multiple baiting events, communication signals constant and non-functional, brain network weights bimodally polarized. Under R-STDP: 14 pushes (-66.7%), 14 attacks (+40%), exploration rate 24.55% (+302%), 11 rescues (6 approach rescues, 3 counter-kills, 1 strategy switch, 1 counter-attack+strategy switch), 2 coordinated attacks, left team score ~80, right team 0, right team division completely reversed (Robot 0 pushes 14 times, Robot 1 pushes 0), average energy drops to 20-30, hunger steps long tail up to 250 steps, communication signals show pulses correlated with energy (r=0.56), brain network remains bimodally polarized. These data reveal how synaptic plasticity drives role reorganization, altruistic rescue emergence, and dynamic exploration-exploitation balance, providing quantitative computational neuroscience evidence for social adaptation and decision-making in biological brains.

Keywords: Spiking Neural Network; R-STDP; Multi-agent; Box-pushing; Task division; Altruistic rescue; Strategy adaptation; Cognitive neuroscience

---

1. Introduction

Cognitive neuroscience has long focused on how the brain achieves goal-directed behavior, social coordination, and adaptive decision-making. While traditional reinforcement learning models can explain individual learning, they struggle to capture the temporal coding, synaptic plasticity, and multi-region coordination of biological neurons. Spiking neural networks (SNNs) combined with reward-modulated STDP (R-STDP) offer a more neurophysiologically plausible framework for simulating biological learning. This study uses a two-population adversarial box-pushing game, where two teams of SNN-based robots (two per team) compete to push a box into their own scoring zones on a 20×10 grid. We first evolved optimal weights through an evolutionary algorithm, then tested two conditions during demonstration: fixed weights and R-STDP online learning. Integrating all behavioral data, communication records, and neural weight evolution charts, this paper provides an in-depth cognitive neuroscience analysis of emergent higher-order intelligent behaviors. By comparing fixed weights and R-STDP, we reveal the central role of synaptic plasticity in social behavioral adaptation, supported by extensive quantitative facts.

---

2. Methods

2.1 Agent Architecture

Each robot comprises three SNN sub-networks:

• Brain network: 37 inputs (25 local sensors + 2 teammate communication + 10 recurrent memory), 4 outputs corresponding to four hard-coded strategies: attack (0), box-chasing (1), random exploration (2), evade (3). Fully connected with multiple synapses per connection (MSF model). Number of weights: 1296.
• Communication network: 4 inputs (self energy, box direction dx/dy, enemy above), 2 continuous outputs transmitted to teammate. Number of weights: 148.
• Exploration network: 2 inputs (energy ratio, normalized hunger steps), 1 output for direction selection during exploration mode. Number of weights: 52.
• Total weights per robot: 1496.

All synaptic weights were initialized randomly and optimized via an evolutionary algorithm (population of multiple individuals, selection, crossover, and mutation over many generations). Each individual played multiple games against random opponents per generation, and average fitness was used for selection. The best individuals were saved after training.

2.2 Game Environment

• Grid: 20 columns × 10 rows. Box starts at a random position not overlapping robots.
• Pushing rule: A robot must enter the box cell from a horizontal direction (left/right) and the cell beyond the box must be empty. The box moves one cell horizontally. Pusher receives +2 points.
• Scoring: If box x < 3, left team gains +50 points and box resets; if x > 16, right team gains +50 points.
• Attack: Adjacent enemy takes 10 damage, cooldown 3 steps, attacker gains +2 points.
• Energy: Starts at 100, attack consumes 10, no natural recovery. Energy ≤ 0 leads to death.
• Sensors: 25 dimensions, including 8-direction box energy field (Gaussian decay), 8-direction enemy presence, self coordinates, box direction, energy ratio, attack cooldown, teammate communication signal, etc.

2.3 Experimental Conditions

• Fixed-weight condition: Load evolved optimal weights, run 1000 steps without weight updates.
• R-STDP condition: Load same initial weights, run 1000 steps with reward-modulated STDP updates based on team score changes at each step.

2.4 Data Collection and Analysis

• Log file: Records step, team, robot ID, alive, x, y, energy, strategy, box position, scores, hunger steps, communication signals (comm0, comm1, teammate_comm).
• Analysis scripts: Detect attack events (energy drop ≥10), push events (box moves and robot at previous box position), rescue events (teammate approaches/counters/strategy-switch within 5 steps after attack), coordinated attacks, alternating pushes, baiting, dodge-counter sequences, task division. Also analyze evolutionary trends and final distributions of neural network weights.

---

3. Results (Quantitative Facts from Actual Run Data)

3.1 Overall Performance Comparison

Metric	Fixed Weights (Data)	R-STDP (Data)	Change
Push events	42	14	-66.7%
Attack events	10	14	+40%
Exploration rate	6.10%	24.55%	+302%
Coordinated attacks	0	2	New
Total rescue events	10	11	+10%
Rescue type distribution	Counter-kill 9, teammate counter 1	Approach 6, counter-kill 3, strategy switch 1, counter+switch 1	Qualitative shift
Left team final score	~500	~80	Drastic drop
Right team final score	0	0	Unchanged
Average energy (late)	~50	20-30	Drop
Right team division	Robot1 pushes 21, Robot0 pushes 0	Robot0 pushes 14, Robot1 pushes 0	Complete reversal
Left team division	Robot1 pushes 20, Robot0 pushes 1	Not specified (trajectory shows change)	—

3.2 Quantified Behaviors under Fixed Weights

3.2.1 Pushing and Scoring

• 42 pushes. Left team score curve (team_scores.png) shows monotonic rise from 0 to ~500, indicating at least 10 successful zone entries (50 points each). Right team score remains 0.

3.2.2 Attack Behavior

• 10 attacks, all executed by attack strategy (0). Attack position heatmap (attack_positions.png) shows concentration around the box (x≈8-12, y≈0-5), peak attack count ~2.0.
• Attack energy distribution wide (inferred from logs, range 20-100).

3.2.3 Rescue Behavior

• 10 rescues, 9 counter-kills (victim self-defense), 1 teammate counter-attack. All victims survive (0 deaths). Rescue type distribution: counter-kill 90%.

3.2.4 Higher-Order Behaviors

• Dodge-counter: Multiple detections. Left team Robot 0 exhibits sequences at steps 15-16, 18-19, 21-22, 24-25, 41-42, 47-48, 57-58, 59-60. Right team Robot 1 once at step 413-414.
• Baiting: Multiple detections of robots moving away from box before scoring. E.g., Left team Robot 1 steps 41-46; Left team Robots 0 and 1 steps 46-51, 51-56 repeatedly.
• Task division: Left team Robot1 pushes 20 times, Robot0 pushes 1 time; Right team Robot1 pushes 21 times, Robot0 pushes 0 times.

3.2.5 Communication Signals

• communication_signals.png shows signals (comm0, comm1, teammate_comm) nearly constant (0.005~0.19).
• comm_correlation.png shows absolute correlation coefficients <0.2 with strategy, energy, hunger steps – no significant correlation.
• comm_vs_distance.png shows weak negative correlation between distance and received signal, but signal values extremely low.

3.2.6 Energy and Hunger

• avg_energy.png (fixed) shows average energy drops from 100 to ~50 then stabilizes.
• hungry_steps_dist.png (fixed) shows highest frequency (≈240) in 0-10 step bin, 30 in 10-20, 25 in 20-30, very few beyond 100.

3.2.7 Neural Network Weights

• left_final_weights_hist.png: Brain network bimodal (peaks near 0 and 1); communication network unimodal around 0.5; exploration network unimodal slightly right.
• right_final_weights_hist.png: Similar.
• left_weights_evolution.png (fixed): x-axis 0-5 generations, brain mean ~498.5 (likely unnormalized, trend stable), comm ~465.5, explore ~467.5.
• training_curve.png: Fitness rises rapidly from 0 to >2000, later oscillates, best fitness ~2000-2500.

3.3 Quantified Behaviors under R-STDP

3.3.1 Pushing and Scoring

• Only 14 pushes (-66.7%). Left team score curve (team_scores.png) rises slowly to ~80, corresponding to about 1 zone entry (50 points) plus some push rewards. Right team remains 0.

3.3.2 Attack Behavior

• 14 attacks (+40%). Attacker strategy distribution: chasing 6, evade 4, random exploration 4. Attack positions still concentrated around box but slightly more dispersed.

3.3.3 Rescue Behavior (Qualitative Shift)

• 11 rescues. Rescue types: approach 6, counter-kill 3, strategy switch 1, counter-attack+strategy switch 1. Approach rescue becomes dominant (6/11) – genuine altruism.
• Victim survival: 0 deaths, 11 survivors.
• Rescues by team: left team victim rescued 3 times, right team victim rescued 8 times.

3.3.4 Higher-Order Behaviors

• Coordinated attacks: 2 events of same-team simultaneous attacks (steps 20 and 23, both left team).
• Dodge-counter: Only 1 detection (left team Robot1 step 56 evade, step 57 attack) – much fewer than fixed weights.
• Baiting: Multiple detections, mainly right team. Examples: Team1 Robot0 steps 33-38, 34-39, 38-43, 41-46; Team1 Robot1 steps 12-17, 38-43; Team0 Robot1 step 46-51.
• Task division reversal: Right team division completely reversed – Robot0 pushes 14 times, Robot1 pushes 0. Left team division not specified numerically, but trajectory heatmaps (trajectory_team0.png) show Robot1 activity range shrinking, Robot0 range expanding, suggesting possible change.

3.3.5 Communication Signals

• communication_signals.png (R-STDP) shows signals now have occasional pulses (e.g., 0.14, 0.13), but absolute values still low (<0.2), mostly zero.
• comm_correlation.png (R-STDP) shows:
  • comm0 with energy r=0.56, with hunger r=-0.36.
  • comm1 with energy r=0.38, with hunger r=-0.033.
  • teammate_comm with energy r=0.13, with hunger r=-0.082.
  • comm0 with comm1 r=0.68, comm0 with teammate_comm r=0.43.
• comm_vs_distance.png: weak negative correlation between distance and received signal, similar to fixed weights.

3.3.6 Energy and Hunger

• avg_energy.png (R-STDP) shows average energy drops from initial 92.5 to 20-30 and stays low – much lower than fixed weights (~50).
• hungry_steps_dist.png (R-STDP) shows 0-10 bin highest (≈240), 10-20 ≈30, 20-30 ≈25, but long tail up to 250 steps (occasional long periods away from box).

3.3.7 Neural Network Weights

• left_final_weights_hist.png (R-STDP): similar to fixed – brain bimodal, comm and explore unimodal.
• right_final_weights_hist.png: same.
• left_weights_evolution.png (R-STDP): similar values to fixed, slightly more fluctuation.
• right_weights_evolution.png: similar.
• training_curve.png (same as fixed, because evolution process identical).

---

4. Discussion

4.1 Synaptic Plasticity Drives Role Reorganization: Data Support

• Fixed weights: Right team Robot1 pushes 21 times, Robot0 pushes 0 – locked division.
• R-STDP: Right team Robot0 pushes 14 times, Robot1 pushes 0 – complete reversal (21→0 vs 0→14). This quantitative reversal can only be explained by online learning. R-STDP allows weight adjustment based on real-time rewards: Robot1’s pushing-related synapses weaken due to early failures or frequent attacks; Robot0’s synapses strengthen through successful attempts. This mirrors prefrontal cortex-mediated task switching in biological brains.

4.2 Conditions for Altruistic Rescue Emergence: Data Support

• Fixed weights: 10 rescues, 9 counter-kills (victim self-defense), 0 approach rescues. So-called “rescue” is essentially self-defense.
• R-STDP: 11 rescues, 6 approach rescues (55%) – teammates actively move toward injured robots. This qualitative shift (0→6) demonstrates that R-STDP enables learning of altruistic behavior. Key mechanism: R-STDP uses team score change as global reward, allowing individuals to learn that helping teammates benefits team score. When approaching an injured teammate leads to subsequent team score increase, relevant synapses strengthen.

4.3 Dynamic Exploration-Exploitation Balance: Data Support

• Fixed weights: Exploration rate only 6.10% – highly specialized after evolution.
• R-STDP: Exploration rate rises to 24.55% (+302%) – online learning encourages trying new strategies. However, exploration remains high after 1000 steps, indicating environmental dynamics (opponents also learning) prevent premature convergence – consistent with the exploration-exploitation dilemma in biological learning.

4.4 Generalization of Attack Strategies: Data Support

• Fixed weights: 10 attacks all executed by attack strategy.
• R-STDP: Among 14 attacks, chasing strategy 6, evade 4, exploration 4. Attacks are no longer confined to attack strategy, showing robots learn to attack in different contexts – strategy boundaries blur.

4.5 Emergence of Coordinated Attacks: Data Support

• Fixed weights: 0 coordinated attacks; R-STDP: 2 events (steps 20, 23, left team). Simultaneous attacks require temporal synchronization; R-STDP reinforces such correlations via reward.

4.6 Subtle Role of Communication Signals: Data Support

• Fixed weights: constant signals, no correlation.
• R-STDP: signals show pulses, positively correlated with energy (r=0.56), negatively with hunger (r=-0.36). This suggests robots begin encoding self-state (energy level, proximity to box) via communication. However, absolute values remain low; communication is not a primary coordination channel. Reason: visual sensors provide sufficient information; communication is redundant.

4.7 Energy Cost of Online Learning: Data Support

• Fixed weights: late average energy ~50; R-STDP: 20-30. Increased attacks (10→14) and exploration (6.10%→24.55%) cause higher energy consumption, with no energy recovery from pushing. Low energy forces robots to use evade more often (evade accounted for 4 attack events), creating a vicious cycle.

4.8 Long Tail in Hunger Steps: Data Support

• R-STDP shows hunger steps up to 250 steps, while fixed weights rarely exceed 100. Increased exploration leads to occasional long periods away from the box.

---

5. Conclusion

This study compares fixed-weight (evolution only) and R-STDP online learning conditions, based on quantitative facts: 42 vs 14 pushes, 10 vs 14 attacks, 6.10% vs 24.55% exploration rate, 0 vs 2 coordinated attacks, 0 vs 6 active approach rescues, right team division reversal 21:0 vs 0:14, etc. These reveal the profound impact of synaptic plasticity on multi-agent social behaviors. Below we list all observed intelligent emergent phenomena by category.

5.1 Task Division and Role Differentiation

Phenomenon	Fixed Weights	R-STDP	Explanation
Extreme task division	Left: R1 pushes 20, R0 pushes 1; Right: R1 pushes 21, R0 pushes 0	Right complete reversal: R0 pushes 14, R1 pushes 0	Evolution produces pusher vs guard roles
Role reversal	None	Right team pusher role transfers from R1 to R0	R-STDP allows online adjustment; roles can reverse when initial division maladaptive
Spatial niche differentiation	Left: R0 roams full field, R1 concentrated near box; Right: R1 concentrated in right half	Right: R0 concentrated near box trajectory, R1 roams	Trajectory heatmaps show distinct spatial territories, reducing competition

5.2 Attack and Combat Behaviors

Phenomenon	Fixed Weights	R-STDP	Explanation
Attack strategy specificity	10 attacks all by attack strategy (0)	14 attacks: chasing 6, evade 4, exploration 4	R-STDP generalizes attacks across strategies
Attack location concentration	Concentrated around box (x≈8-12, y≈0-5)	Similarly concentrated, slightly more dispersed	Attack motivation is resource competition
Coordinated attacks	0	2 (steps 20, 23, left team)	R-STDP facilitates temporal synchronization
Dodge-counter sequences	Multiple (left team R0 ≥8 times)	1 (left team R1)	Fixed weights form stable temporal action memories

5.3 Rescue and Altruistic Behaviors

Phenomenon	Fixed Weights	R-STDP	Explanation
Self-defense counter-kill	9 (90%)	3	Victim counterattacks – essentially self-defense
Teammate counter-attack rescue	1	1 (counter) + 1 (counter+switch)	Teammate attacks attacker, indirect protection
Active approach rescue	0	6 (55%)	Teammate actively moves toward injured robot – genuine altruism
Strategy switch rescue	0	2 (1 alone, 1 with counter)	Teammate switches from non-attack to attack/evade
Victim survival rate	100% (10/10)	100% (11/11)	All attacked robots survive

5.4 Tactical Deception and Strategic Behaviors

Phenomenon	Fixed Weights	R-STDP	Explanation
Baiting (moving away from box)	Multiple (left team R0, R1)	Multiple (mainly right team R0, R1)	Robot moves away from box before scoring, luring enemies away to create opportunities for teammate

5.5 Exploration and Adaptation Behaviors

Phenomenon	Fixed Weights	R-STDP	Explanation
Exploration rate	6.10%	24.55% (+302%)	R-STDP encourages trying new strategies
Hunger steps long tail	Rarely >100	Up to 250 steps	Increased exploration leads to occasional long periods away from box
Energy management	Late average ~50	Late average 20-30	Increased attacks and exploration raise energy consumption

5.6 Communication and Information Transfer

Phenomenon	Fixed Weights	R-STDP	Explanation
Communication signal activity	Constant (0.005~0.19)	Occasional pulses (e.g., 0.14), mostly zero	R-STDP changes communication from silent to intermittent
Communication-energy correlation	No significant (	r	<0.2)
Communication-strategy correlation	None	comm0-strategy r=-0.068 (weak)	Weak association
Communication-distance relationship	Weak negative	Weak negative	Slightly higher signal when closer, but absolute values low

5.7 Neural Network Plasticity

Phenomenon	Fixed Weights	R-STDP	Explanation
Brain network weight distribution	Bimodal (near 0 and 1)	Bimodal	Clear LTP/LTD
Communication network weight distribution	Unimodal (~0.5)	Unimodal (~0.5)	Not fully polarized
Exploration network weight distribution	Unimodal (~0.5)	Unimodal (~0.5)	Not fully polarized

5.8 Integrated Summary

Behavioral Dimension	Fixed Weights (Evolution Locked)	R-STDP (Online Learning)	Core Mechanism
Division of labor	Extreme and fixed	Reversible	Synaptic plasticity enables role reorganization
Rescue	Self-defense counter-kill dominant	Active approach dominant	Global reward drives altruism
Attack	Strategy-specific	Strategy-generalized	Cross-context learning
Coordination	No coordinated attacks	Coordinated attacks emerge	Temporal synchronization reinforced
Exploration	Low rate	High rate	Online learning encourages trying
Communication	Silent	Correlated with energy state	State encoding emerges
Energy	Moderate level	Low level	Cost of high-energy strategies

---

Overall Conclusion: Synaptic plasticity (R-STDP) liberates multi-agent systems from evolutionarily locked behavioral patterns, enabling dynamic adjustment of roles, rescue strategies, attack modes, and exploration tendencies based on real-time environmental feedback. This gives rise to advanced intelligent behaviors including role reversal, active altruistic rescue, coordinated attacks, strategy generalization, and state-encoding communication. These findings provide a quantitative computational model for understanding the neural mechanisms of social adaptation and decision-making in biological brains, and offer empirical guidance for designing flexible adaptive multi-agent systems.

---

Data Availability

All log files, weight files, and analysis charts (including team_scores.png, avg_energy.png, hungry_steps_dist.png, attack_positions.png, communication_signals.png, comm_correlation.png, comm_vs_distance.png, trajectory_team0.png, trajectory_team1.png, left_weights_evolution.png, right_weights_evolution.png, left_final_weights_hist.png, right_final_weights_hist.png, rescue_types.png, training_curve.png) are provided in supplementary materials.