paulh
/
donkeycar-rl-autoresearch
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

autoresearch: phase1 trial 5 results 

Agent: pi
Tests: N/A
Tests-Added: 0
TypeScript: N/A
This commit is contained in:
Paul Huliganga 2026-04-13 22:46:54 -04:00
parent c8a495dd22
commit 2d6fe2c962
2 changed files with 102 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

97
agent/outerloop-results/autoresearch_phase2_log.txt
View File

@ -49,3 +49,100 @@
[2026-04-13 20:55:59] [Champion] 🏆 NEW BEST! Trial 3: mean_reward=90.0000 params={'r': 90} [2026-04-13 20:55:59] [Champion] 🏆 NEW BEST! Trial 3: mean_reward=90.0000 params={'r': 90}
[2026-04-13 20:55:59] [Champion] 🏆 NEW BEST! Trial 5: mean_reward=75.0000 params={'n_steer': 8} [2026-04-13 20:55:59] [Champion] 🏆 NEW BEST! Trial 5: mean_reward=75.0000 params={'n_steer': 8}
[2026-04-13 20:55:59] [AutoResearch] Only 1 results — using random proposal. [2026-04-13 20:55:59] [AutoResearch] Only 1 results — using random proposal.
[2026-04-13 20:56:39] ============================================================
[2026-04-13 20:56:39] [AutoResearch] Phase 1 — Real PPO Training + GP+UCB Optimization
[2026-04-13 20:56:39] [AutoResearch] Max trials: 20 | kappa: 2.0 | push every: 5
[2026-04-13 20:56:39] [AutoResearch] Results: /home/paulh/projects/donkeycar-rl-autoresearch/agent/outerloop-results/autoresearch_results_phase2.jsonl
[2026-04-13 20:56:39] [AutoResearch] Champion: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/champion
[2026-04-13 20:56:39] ============================================================
[2026-04-13 20:56:39] [AutoResearch] Loaded 1 existing Phase 1 results.
[2026-04-13 20:56:39] [AutoResearch] Champion: trial=5 mean_reward=4582.7984 params={'n_steer': 7, 'n_throttle': 3, 'learning_rate': 0.0006801262090358742, 'timesteps': 4787, 'agent': 'ppo', 'eval_episodes': 3, 'reward_shaping': True}
[2026-04-13 20:56:39]
[AutoResearch] ========== Trial 1/20 ==========
[2026-04-13 20:56:39] [AutoResearch] Only 1 results — using random proposal.
[2026-04-13 20:56:39] [AutoResearch] Proposed: {'n_steer': 4, 'n_throttle': 2, 'learning_rate': 0.0016410214223984076, 'timesteps': 16101, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 20:56:41] [AutoResearch] Launching trial 1: {'n_steer': 4, 'n_throttle': 2, 'learning_rate': 0.0016410214223984076, 'timesteps': 16101, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:09:13] [AutoResearch] Trial 1 finished in 751.5s, returncode=0
[2026-04-13 21:09:13] [AutoResearch] Trial 1: mean_reward=177.7416 std_reward=142.3977
[2026-04-13 21:09:13] [AutoResearch] === Trial 1 Summary ===
[2026-04-13 21:09:13] Total Phase 1 runs: 2
[2026-04-13 21:09:13] Champion: trial=5 mean_reward=4582.7984 params={'n_steer': 7, 'n_throttle': 3, 'learning_rate': 0.0006801262090358742, 'timesteps': 4787, 'agent': 'ppo', 'eval_episodes': 3, 'reward_shaping': True}
[2026-04-13 21:09:13] Top 5:
[2026-04-13 21:09:13] mean_reward=234.5386 params={'n_steer': 4, 'n_throttle': 3, 'learning_rate': 0.0009737963906394612, 'timesteps': 47325, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:09:13] mean_reward=177.7416 params={'n_steer': 4, 'n_throttle': 2, 'learning_rate': 0.0016410214223984076, 'timesteps': 16101, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:09:15]
[AutoResearch] ========== Trial 2/20 ==========
[2026-04-13 21:09:15] [AutoResearch] Only 2 results — using random proposal.
[2026-04-13 21:09:15] [AutoResearch] Proposed: {'n_steer': 8, 'n_throttle': 2, 'learning_rate': 0.0012716386940916763, 'timesteps': 40768, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:09:17] [AutoResearch] Launching trial 2: {'n_steer': 8, 'n_throttle': 2, 'learning_rate': 0.0012716386940916763, 'timesteps': 40768, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:32:22] [AutoResearch] Trial 2 finished in 1384.9s, returncode=0
[2026-04-13 21:32:22] [AutoResearch] Trial 2: mean_reward=38.1267 std_reward=0.3364
[2026-04-13 21:32:22] [AutoResearch] === Trial 2 Summary ===
[2026-04-13 21:32:22] Total Phase 1 runs: 3
[2026-04-13 21:32:22] Champion: trial=5 mean_reward=4582.7984 params={'n_steer': 7, 'n_throttle': 3, 'learning_rate': 0.0006801262090358742, 'timesteps': 4787, 'agent': 'ppo', 'eval_episodes': 3, 'reward_shaping': True}
[2026-04-13 21:32:22] Top 5:
[2026-04-13 21:32:22] mean_reward=234.5386 params={'n_steer': 4, 'n_throttle': 3, 'learning_rate': 0.0009737963906394612, 'timesteps': 47325, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:32:22] mean_reward=177.7416 params={'n_steer': 4, 'n_throttle': 2, 'learning_rate': 0.0016410214223984076, 'timesteps': 16101, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:32:22] mean_reward=38.1267 params={'n_steer': 8, 'n_throttle': 2, 'learning_rate': 0.0012716386940916763, 'timesteps': 40768, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:32:24]
[AutoResearch] ========== Trial 3/20 ==========
[2026-04-13 21:32:24] [AutoResearch] GP UCB top-5 candidates:
[2026-04-13 21:32:24] UCB=2.2673 mu=0.5045 sigma=0.8814 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.000840799681375933, 'timesteps': 35596}
[2026-04-13 21:32:24] UCB=2.2663 mu=0.4912 sigma=0.8876 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0012733685738093425, 'timesteps': 41802}
[2026-04-13 21:32:24] UCB=2.2632 mu=0.5326 sigma=0.8653 params={'n_steer': 3, 'n_throttle': 3, 'learning_rate': 0.0003737785062265609, 'timesteps': 48369}
[2026-04-13 21:32:24] UCB=2.2622 mu=0.4884 sigma=0.8869 params={'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.0009593125016626112, 'timesteps': 41226}
[2026-04-13 21:32:24] UCB=2.2594 mu=0.4092 sigma=0.9251 params={'n_steer': 3, 'n_throttle': 3, 'learning_rate': 0.0006680173697602083, 'timesteps': 33139}
[2026-04-13 21:32:24] [AutoResearch] Proposed: {'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.000840799681375933, 'timesteps': 35596, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:32:26] [AutoResearch] Launching trial 3: {'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.000840799681375933, 'timesteps': 35596, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:57:48] [AutoResearch] Trial 3 finished in 1522.1s, returncode=0
[2026-04-13 21:57:48] [AutoResearch] Trial 3: mean_reward=615.6443 std_reward=2.4555
[2026-04-13 21:57:48] [AutoResearch] === Trial 3 Summary ===
[2026-04-13 21:57:48] Total Phase 1 runs: 4
[2026-04-13 21:57:48] Champion: trial=5 mean_reward=4582.7984 params={'n_steer': 7, 'n_throttle': 3, 'learning_rate': 0.0006801262090358742, 'timesteps': 4787, 'agent': 'ppo', 'eval_episodes': 3, 'reward_shaping': True}
[2026-04-13 21:57:48] Top 5:
[2026-04-13 21:57:48] mean_reward=615.6443 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.000840799681375933, 'timesteps': 35596, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:57:48] mean_reward=234.5386 params={'n_steer': 4, 'n_throttle': 3, 'learning_rate': 0.0009737963906394612, 'timesteps': 47325, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:57:48] mean_reward=177.7416 params={'n_steer': 4, 'n_throttle': 2, 'learning_rate': 0.0016410214223984076, 'timesteps': 16101, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:57:48] mean_reward=38.1267 params={'n_steer': 8, 'n_throttle': 2, 'learning_rate': 0.0012716386940916763, 'timesteps': 40768, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:57:50]
[AutoResearch] ========== Trial 4/20 ==========
[2026-04-13 21:57:50] [AutoResearch] GP UCB top-5 candidates:
[2026-04-13 21:57:50] UCB=2.6247 mu=1.1138 sigma=0.7554 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0006446108743181142, 'timesteps': 25224}
[2026-04-13 21:57:50] UCB=2.6201 mu=1.0985 sigma=0.7608 params={'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.00040076107708415066, 'timesteps': 34530}
[2026-04-13 21:57:50] UCB=2.6128 mu=0.9229 sigma=0.8449 params={'n_steer': 4, 'n_throttle': 5, 'learning_rate': 0.0007783797179569566, 'timesteps': 28443}
[2026-04-13 21:57:50] UCB=2.5877 mu=1.1874 sigma=0.7001 params={'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.000483120120780932, 'timesteps': 32645}
[2026-04-13 21:57:50] UCB=2.5805 mu=1.1160 sigma=0.7322 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0004185314575094028, 'timesteps': 31606}
[2026-04-13 21:57:50] [AutoResearch] Proposed: {'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0006446108743181142, 'timesteps': 25224, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 21:57:52] [AutoResearch] Launching trial 4: {'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0006446108743181142, 'timesteps': 25224, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:17:49] [AutoResearch] Trial 4 finished in 1196.7s, returncode=0
[2026-04-13 22:17:49] [AutoResearch] Trial 4: mean_reward=56.9474 std_reward=0.4525
[2026-04-13 22:17:49] [AutoResearch] === Trial 4 Summary ===
[2026-04-13 22:17:49] Total Phase 1 runs: 5
[2026-04-13 22:17:49] Champion: trial=5 mean_reward=4582.7984 params={'n_steer': 7, 'n_throttle': 3, 'learning_rate': 0.0006801262090358742, 'timesteps': 4787, 'agent': 'ppo', 'eval_episodes': 3, 'reward_shaping': True}
[2026-04-13 22:17:49] Top 5:
[2026-04-13 22:17:49] mean_reward=615.6443 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.000840799681375933, 'timesteps': 35596, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:17:49] mean_reward=234.5386 params={'n_steer': 4, 'n_throttle': 3, 'learning_rate': 0.0009737963906394612, 'timesteps': 47325, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:17:49] mean_reward=177.7416 params={'n_steer': 4, 'n_throttle': 2, 'learning_rate': 0.0016410214223984076, 'timesteps': 16101, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:17:49] mean_reward=56.9474 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0006446108743181142, 'timesteps': 25224, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:17:49] mean_reward=38.1267 params={'n_steer': 8, 'n_throttle': 2, 'learning_rate': 0.0012716386940916763, 'timesteps': 40768, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:17:51]
[AutoResearch] ========== Trial 5/20 ==========
[2026-04-13 22:17:51] [AutoResearch] GP UCB top-5 candidates:
[2026-04-13 22:17:51] UCB=3.2705 mu=1.9137 sigma=0.6784 params={'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.0010511079430656864, 'timesteps': 43721}
[2026-04-13 22:17:51] UCB=3.0915 mu=1.4459 sigma=0.8228 params={'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.0013020888853863901, 'timesteps': 44447}
[2026-04-13 22:17:51] UCB=3.0371 mu=1.3845 sigma=0.8263 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0014826204762677822, 'timesteps': 36122}
[2026-04-13 22:17:51] UCB=3.0172 mu=1.1871 sigma=0.9150 params={'n_steer': 3, 'n_throttle': 5, 'learning_rate': 0.0008990881268700197, 'timesteps': 47181}
[2026-04-13 22:17:51] UCB=3.0156 mu=1.6493 sigma=0.6832 params={'n_steer': 4, 'n_throttle': 5, 'learning_rate': 0.0009711471794993783, 'timesteps': 36714}
[2026-04-13 22:17:51] [AutoResearch] Proposed: {'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.0010511079430656864, 'timesteps': 43721, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:17:53] [AutoResearch] Launching trial 5: {'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.0010511079430656864, 'timesteps': 43721, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:46:54] [AutoResearch] Trial 5 finished in 1741.0s, returncode=0
[2026-04-13 22:46:54] [AutoResearch] Trial 5: mean_reward=254.5237 std_reward=34.6249
[2026-04-13 22:46:54] [AutoResearch] === Trial 5 Summary ===
[2026-04-13 22:46:54] Total Phase 1 runs: 6
[2026-04-13 22:46:54] Champion: trial=5 mean_reward=4582.7984 params={'n_steer': 7, 'n_throttle': 3, 'learning_rate': 0.0006801262090358742, 'timesteps': 4787, 'agent': 'ppo', 'eval_episodes': 3, 'reward_shaping': True}
[2026-04-13 22:46:54] Top 5:
[2026-04-13 22:46:54] mean_reward=615.6443 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.000840799681375933, 'timesteps': 35596, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:46:54] mean_reward=254.5237 params={'n_steer': 4, 'n_throttle': 4, 'learning_rate': 0.0010511079430656864, 'timesteps': 43721, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:46:54] mean_reward=234.5386 params={'n_steer': 4, 'n_throttle': 3, 'learning_rate': 0.0009737963906394612, 'timesteps': 47325, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:46:54] mean_reward=177.7416 params={'n_steer': 4, 'n_throttle': 2, 'learning_rate': 0.0016410214223984076, 'timesteps': 16101, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}
[2026-04-13 22:46:54] mean_reward=56.9474 params={'n_steer': 3, 'n_throttle': 4, 'learning_rate': 0.0006446108743181142, 'timesteps': 25224, 'agent': 'ppo', 'eval_episodes': 5, 'reward_shaping': True}

5
agent/outerloop-results/autoresearch_results_phase2.jsonl
View File

@ -1 +1,6 @@
{"trial": 1, "timestamp": "2026-04-13T20:05:03.791538", "params": {"n_steer": 4, "n_throttle": 3, "learning_rate": 0.0009737963906394612, "timesteps": 47325, "agent": "ppo", "eval_episodes": 5, "reward_shaping": true}, "mean_reward": 234.5386, "std_reward": 3.1547, "model_path": "/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/trial-0001/model.zip", "champion": false, "run_status": "ok", "elapsed_sec": 1908.32528758049, "reward_hacking_suspected": false} {"trial": 1, "timestamp": "2026-04-13T20:05:03.791538", "params": {"n_steer": 4, "n_throttle": 3, "learning_rate": 0.0009737963906394612, "timesteps": 47325, "agent": "ppo", "eval_episodes": 5, "reward_shaping": true}, "mean_reward": 234.5386, "std_reward": 3.1547, "model_path": "/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/trial-0001/model.zip", "champion": false, "run_status": "ok", "elapsed_sec": 1908.32528758049, "reward_hacking_suspected": false}
{"trial": 1, "timestamp": "2026-04-13T21:09:13.397820", "params": {"n_steer": 4, "n_throttle": 2, "learning_rate": 0.0016410214223984076, "timesteps": 16101, "agent": "ppo", "eval_episodes": 5, "reward_shaping": true}, "mean_reward": 177.7416, "std_reward": 142.3977, "model_path": "/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/trial-0001/model.zip", "champion": false, "run_status": "ok", "elapsed_sec": 751.4716398715973, "reward_hacking_suspected": false}
{"trial": 2, "timestamp": "2026-04-13T21:32:22.333630", "params": {"n_steer": 8, "n_throttle": 2, "learning_rate": 0.0012716386940916763, "timesteps": 40768, "agent": "ppo", "eval_episodes": 5, "reward_shaping": true}, "mean_reward": 38.1267, "std_reward": 0.3364, "model_path": "/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/trial-0002/model.zip", "champion": false, "run_status": "ok", "elapsed_sec": 1384.922327041626, "reward_hacking_suspected": false}
{"trial": 3, "timestamp": "2026-04-13T21:57:48.402590", "params": {"n_steer": 3, "n_throttle": 4, "learning_rate": 0.000840799681375933, "timesteps": 35596, "agent": "ppo", "eval_episodes": 5, "reward_shaping": true}, "mean_reward": 615.6443, "std_reward": 2.4555, "model_path": "/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/trial-0003/model.zip", "champion": false, "run_status": "ok", "elapsed_sec": 1522.0542554855347, "reward_hacking_suspected": false}
{"trial": 4, "timestamp": "2026-04-13T22:17:49.148601", "params": {"n_steer": 3, "n_throttle": 4, "learning_rate": 0.0006446108743181142, "timesteps": 25224, "agent": "ppo", "eval_episodes": 5, "reward_shaping": true}, "mean_reward": 56.9474, "std_reward": 0.4525, "model_path": "/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/trial-0004/model.zip", "champion": false, "run_status": "ok", "elapsed_sec": 1196.7339782714844, "reward_hacking_suspected": false}
{"trial": 5, "timestamp": "2026-04-13T22:46:54.148910", "params": {"n_steer": 4, "n_throttle": 4, "learning_rate": 0.0010511079430656864, "timesteps": 43721, "agent": "ppo", "eval_episodes": 5, "reward_shaping": true}, "mean_reward": 254.5237, "std_reward": 34.6249, "model_path": "/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/trial-0005/model.zip", "champion": false, "run_status": "ok", "elapsed_sec": 1740.9856295585632, "reward_hacking_suspected": false}