[07:36:52] ============================================================ [07:36:52] Exp 26 started — PID 962156 [07:36:52] Log: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/run_2026-05-06_073652_warmstart.log [07:36:52] Warm start: exp25 best_model (381r @ 80k) [07:36:52] ============================================================ [07:41:49] [10,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0010000.zip [07:41:49] Reconnecting for fresh road... [07:41:54] Connected (new road) [07:42:20] Eval: gen_road=380.0r/2000s ✅ [07:42:20] NEW BEST: steps=2000 reward=380.0 [07:47:11] [20,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0020000.zip [07:47:11] Reconnecting for fresh road... [07:47:16] Connected (new road) [07:47:42] Eval: gen_road=388.8r/2000s ✅ [07:47:43] NEW BEST: steps=2000 reward=388.8 [07:52:32] [30,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0030000.zip [07:52:32] Reconnecting for fresh road... [07:52:37] Connected (new road) [07:53:03] Eval: gen_road=379.7r/2000s ✅ [07:57:55] [40,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0040000.zip [07:57:55] Reconnecting for fresh road... [07:58:00] Connected (new road) [07:58:27] Eval: gen_road=371.8r/2000s ✅ [08:03:15] [50,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0050000.zip [08:03:15] Reconnecting for fresh road... [08:03:20] Connected (new road) [08:03:47] Eval: gen_road=374.4r/2000s ✅ [08:08:44] [60,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0060000.zip [08:08:44] Reconnecting for fresh road... [08:08:50] Connected (new road) [08:09:16] Eval: gen_road=370.9r/2000s ✅ [08:14:11] [70,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0070000.zip [08:14:11] Reconnecting for fresh road... [08:14:16] Connected (new road) [08:14:42] Eval: gen_road=375.3r/2000s ✅ [08:19:33] [80,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0080000.zip [08:19:33] Reconnecting for fresh road... [08:19:38] Connected (new road) [08:20:04] Eval: gen_road=359.1r/2000s ✅ [08:24:55] [90,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0090000.zip [08:24:55] Reconnecting for fresh road... [08:25:00] Connected (new road) [08:25:27] Eval: gen_road=357.5r/2000s ✅ [08:30:19] [100,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0100000.zip [08:30:19] Reconnecting for fresh road... [08:30:24] Connected (new road) [08:30:51] Eval: gen_road=361.5r/2000s ✅ [08:35:39] [110,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0110000.zip [08:35:39] Reconnecting for fresh road... [08:35:44] Connected (new road) [08:36:10] Eval: gen_road=353.5r/2000s ✅ [08:41:01] [120,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0120000.zip [08:41:01] Reconnecting for fresh road... [08:41:06] Connected (new road) [08:41:32] Eval: gen_road=355.0r/2000s ✅ [08:46:25] [130,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0130000.zip [08:46:25] Reconnecting for fresh road... [08:46:30] Connected (new road) [08:46:56] Eval: gen_road=348.6r/2000s ✅ [08:51:52] [140,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0140000.zip [08:51:52] Reconnecting for fresh road... [08:51:57] Connected (new road) [08:52:23] Eval: gen_road=333.6r/2000s ✅ [08:57:15] [150,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0150000.zip [08:57:15] Reconnecting for fresh road... [08:57:20] Connected (new road) [08:57:46] Eval: gen_road=327.4r/2000s ✅ [09:02:40] [160,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0160000.zip [09:02:40] Reconnecting for fresh road... [09:02:45] Connected (new road) [09:03:11] Eval: gen_road=332.3r/2000s ✅ [09:08:07] [170,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0170000.zip [09:08:07] Reconnecting for fresh road... [09:08:12] Connected (new road) [09:08:22] Eval: gen_road=96.7r/626s ❌@626 [09:13:17] [180,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0180000.zip [09:13:17] Reconnecting for fresh road... [09:13:22] Connected (new road) [09:13:37] Eval: gen_road=157.8r/1009s ❌@1009 [09:18:24] [190,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0190000.zip [09:18:24] Reconnecting for fresh road... [09:18:29] Connected (new road) [09:18:55] Eval: gen_road=327.3r/2000s ✅ [09:23:45] [200,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0200000.zip [09:23:45] Reconnecting for fresh road... [09:23:50] Connected (new road) [09:24:01] Eval: gen_road=107.7r/728s ❌@728 [09:28:57] [210,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0210000.zip [09:28:57] Reconnecting for fresh road... [09:29:02] Connected (new road) [09:29:14] Eval: gen_road=104.2r/726s ❌@726 [09:34:15] [220,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0220000.zip [09:34:15] Reconnecting for fresh road... [09:34:20] Connected (new road) [09:34:40] Eval: gen_road=207.1r/1390s ❌@1390 [09:40:21] [230,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0230000.zip [09:40:21] Reconnecting for fresh road... [09:40:27] Connected (new road) [09:40:53] Eval: gen_road=290.6r/2000s ✅ [09:46:20] [240,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0240000.zip [09:46:20] Reconnecting for fresh road... [09:46:25] Connected (new road) [09:46:51] Eval: gen_road=298.6r/2000s ✅ [09:52:16] [250,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0250000.zip [09:52:16] Reconnecting for fresh road... [09:52:21] Connected (new road) [09:52:47] Eval: gen_road=309.2r/2000s ✅ [09:57:58] [260,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0260000.zip [09:57:58] Reconnecting for fresh road... [09:58:03] Connected (new road) [09:58:30] Eval: gen_road=305.2r/2000s ✅ [10:03:51] [270,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0270000.zip [10:03:51] Reconnecting for fresh road... [10:03:56] Connected (new road) [10:04:01] Eval: gen_road=22.3r/206s ❌@206 [10:09:22] [280,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0280000.zip [10:09:22] Reconnecting for fresh road... [10:09:27] Connected (new road) [10:09:53] Eval: gen_road=311.9r/2000s ✅ [10:14:53] [290,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0290000.zip [10:14:53] Reconnecting for fresh road... [10:14:58] Connected (new road) [10:15:24] Eval: gen_road=295.8r/2000s ✅ [10:20:23] [300,000/300,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/checkpoint_0300000.zip [10:20:23] Reconnecting for fresh road... [10:20:28] Connected (new road) [10:20:42] Eval: gen_road=125.3r/879s ❌@879 [10:20:42] ============================================================ [10:20:42] FINAL EVALUATION: best_model on generated_road (3 fresh roads) [10:20:42] ============================================================ [10:21:15] Set 1: 389.5r / 2000s ✅ [10:21:31] Set 2: 110.7r / 628s ❌@628 [10:22:03] Set 3: 389.2r / 2000s ✅ [10:22:03] Mean: 1543 steps / 296.5 reward [10:22:03] Exp 26 complete.