[10:29:52] ====================================================================== [10:29:52] Evaluating best models on 10 genuinely different random roads [10:29:52] Seeds: [1001, 2002, 3003, 4004, 5005, 6006, 7007, 8008, 9009, 1234] [10:29:52] Log: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/eval_best_models_20260506_102952.log [10:29:52] ====================================================================== [10:29:52] Connecting to sim... [10:29:52] Connected. obs=(3, 120, 160), action=Discrete(7) [10:29:52] [10:29:52] ── exp24 ────────────────────────────────────── [10:29:52] Model: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp24-discrete/best_model.zip [10:29:55] Road 1/10 (seed=1001) — regenerating... [10:30:24] → 371.0r / 2000s ✅ [10:30:24] Road 2/10 (seed=2002) — regenerating... [10:30:53] → 365.2r / 2000s ✅ [10:30:53] Road 3/10 (seed=3003) — regenerating... [10:31:22] → 365.0r / 2000s ✅ [10:31:22] Road 4/10 (seed=4004) — regenerating... [10:31:51] → 372.2r / 2000s ✅ [10:31:51] Road 5/10 (seed=5005) — regenerating... [10:32:21] → 363.3r / 2000s ✅ [10:32:21] Road 6/10 (seed=6006) — regenerating... [10:32:50] → 365.8r / 2000s ✅ [10:32:50] Road 7/10 (seed=7007) — regenerating... [10:33:19] → 371.5r / 2000s ✅ [10:33:19] Road 8/10 (seed=8008) — regenerating... [10:33:36] → 157.7r / 912s ❌@912 [10:33:36] Road 9/10 (seed=9009) — regenerating... [10:34:05] → 371.6r / 2000s ✅ [10:34:05] Road 10/10 (seed=1234) — regenerating... [10:34:35] → 372.1r / 2000s ✅ [10:34:35] exp24 SUMMARY: 9/10 full | mean 1891s / 347.5r [10:34:35] [10:34:35] ── exp25 ────────────────────────────────────── [10:34:35] Model: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp25-wheel-fix/best_model.zip [10:34:36] Road 1/10 (seed=1001) — regenerating... [10:35:05] → 378.5r / 2000s ✅ [10:35:05] Road 2/10 (seed=2002) — regenerating... [10:35:34] → 382.9r / 2000s ✅ [10:35:34] Road 3/10 (seed=3003) — regenerating... [10:36:03] → 382.0r / 2000s ✅ [10:36:03] Road 4/10 (seed=4004) — regenerating... [10:36:18] → 122.8r / 694s ❌@694 [10:36:18] Road 5/10 (seed=5005) — regenerating... [10:36:47] → 384.3r / 2000s ✅ [10:36:47] Road 6/10 (seed=6006) — regenerating... [10:37:16] → 379.7r / 2000s ✅ [10:37:16] Road 7/10 (seed=7007) — regenerating... [10:37:45] → 382.7r / 2000s ✅ [10:37:45] Road 8/10 (seed=8008) — regenerating... [10:38:15] → 382.8r / 2000s ✅ [10:38:15] Road 9/10 (seed=9009) — regenerating... [10:38:44] → 383.2r / 2000s ✅ [10:38:44] Road 10/10 (seed=1234) — regenerating... [10:39:13] → 383.9r / 2000s ✅ [10:39:13] exp25 SUMMARY: 9/10 full | mean 1869s / 356.3r [10:39:13] [10:39:13] ── exp26 ────────────────────────────────────── [10:39:13] Model: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp26-warmstart/best_model.zip [10:39:14] Road 1/10 (seed=1001) — regenerating... [10:39:43] → 392.2r / 2000s ✅ [10:39:43] Road 2/10 (seed=2002) — regenerating... [10:40:10] → 307.0r / 1583s ❌@1583 [10:40:10] Road 3/10 (seed=3003) — regenerating... [10:40:39] → 387.6r / 2000s ✅ [10:40:39] Road 4/10 (seed=4004) — regenerating... [10:41:08] → 392.5r / 2000s ✅ [10:41:08] Road 5/10 (seed=5005) — regenerating... [10:41:37] → 390.6r / 2000s ✅ [10:41:37] Road 6/10 (seed=6006) — regenerating... [10:42:07] → 389.4r / 2000s ✅ [10:42:07] Road 7/10 (seed=7007) — regenerating... [10:42:36] → 388.2r / 2000s ✅ [10:42:36] Road 8/10 (seed=8008) — regenerating... [10:43:05] → 389.1r / 2000s ✅ [10:43:05] Road 9/10 (seed=9009) — regenerating... [10:43:34] → 389.0r / 2000s ✅ [10:43:34] Road 10/10 (seed=1234) — regenerating... [10:44:04] → 386.5r / 2000s ✅ [10:44:04] exp26 SUMMARY: 9/10 full | mean 1958s / 381.2r [10:44:04] [10:44:04] ====================================================================== [10:44:04] FINAL RANKING [10:44:04] ====================================================================== [10:44:04] #1 exp26 9/10 full mean 1958s / 381.2r [10:44:04] #2 exp24 9/10 full mean 1891s / 347.5r [10:44:04] #3 exp25 9/10 full mean 1869s / 356.3r [10:44:04] [10:44:04] Evaluation complete.