paulh
/
donkeycar-rl-autoresearch
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix: fresh PPO verbose=0 suppressed all training output — set verbose=1 

Without this, Wave 4 scratch-trained models produce no rollout stats in
the log, making it impossible to monitor training progress or spot
degenerate policies early.

Warm-start models in Wave 3 showed stats because verbose=1 was baked
into the Phase-2 saved model state; fresh models default to verbose=0.

Agent: pi
Tests: 96 passed
Tests-Added: 0
TypeScript: N/A
This commit is contained in:
Paul Huliganga 2026-04-14 22:44:22 -04:00
parent 7534527722
commit 860e3d6610
4 changed files with 37 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
agent/multitrack_runner.py
View File

@ -230,7 +230,7 @@ def create_or_load_model(env, learning_rate, warm_start_path=None, seed=None):
'CnnPolicy', 'CnnPolicy',
env, env,
learning_rate=learning_rate, learning_rate=learning_rate,
verbose=0, verbose=1, # show rollout stats so training progress is visible in log
seed=seed, seed=seed,
) )
return model return model

13
agent/outerloop-results/autoresearch_phase2_log.txt
View File

@ -593,3 +593,16 @@
[2026-04-14 22:40:11] [Champion] 🏆 NEW BEST! Trial 3: mean_reward=90.0000 params={'r': 90} [2026-04-14 22:40:11] [Champion] 🏆 NEW BEST! Trial 3: mean_reward=90.0000 params={'r': 90}
[2026-04-14 22:40:11] [Champion] 🏆 NEW BEST! Trial 5: mean_reward=75.0000 params={'n_steer': 8} [2026-04-14 22:40:11] [Champion] 🏆 NEW BEST! Trial 5: mean_reward=75.0000 params={'n_steer': 8}
[2026-04-14 22:40:11] [AutoResearch] Only 1 results — using random proposal. [2026-04-14 22:40:11] [AutoResearch] Only 1 results — using random proposal.
[2026-04-14 22:43:59] [AutoResearch] GP UCB top-5 candidates:
[2026-04-14 22:43:59] UCB=2.3107 mu=0.3981 sigma=0.9563 params={'n_steer': 9, 'n_throttle': 2, 'learning_rate': 0.001405531880392808, 'timesteps': 26173}
[2026-04-14 22:43:59] UCB=2.3049 mu=0.8602 sigma=0.7224 params={'n_steer': 9, 'n_throttle': 3, 'learning_rate': 0.001793493447174312, 'timesteps': 19198}
[2026-04-14 22:43:59] UCB=2.2813 mu=0.4904 sigma=0.8954 params={'n_steer': 9, 'n_throttle': 4, 'learning_rate': 0.0011616192816742616, 'timesteps': 13887}
[2026-04-14 22:43:59] UCB=2.2767 mu=0.5194 sigma=0.8787 params={'n_steer': 9, 'n_throttle': 4, 'learning_rate': 0.0011646447444663046, 'timesteps': 21199}
[2026-04-14 22:43:59] UCB=2.2525 mu=0.6254 sigma=0.8136 params={'n_steer': 9, 'n_throttle': 3, 'learning_rate': 0.0010196345864901517, 'timesteps': 22035}
[2026-04-14 22:43:59] [Champion] 🏆 NEW BEST! Trial 1: mean_reward=50.0000 params={'n_steer': 5}
[2026-04-14 22:43:59] [Champion] 🏆 NEW BEST! Trial 1: mean_reward=80.0000 params={'n_steer': 7}
[2026-04-14 22:43:59] [Champion] 🏆 NEW BEST! Trial 0: mean_reward=50.0000 params={'r': 50}
[2026-04-14 22:43:59] [Champion] 🏆 NEW BEST! Trial 1: mean_reward=80.0000 params={'r': 80}
[2026-04-14 22:43:59] [Champion] 🏆 NEW BEST! Trial 3: mean_reward=90.0000 params={'r': 90}
[2026-04-14 22:43:59] [Champion] 🏆 NEW BEST! Trial 5: mean_reward=75.0000 params={'n_steer': 8}
[2026-04-14 22:43:59] [AutoResearch] Only 1 results — using random proposal.

5
agent/outerloop-results/autoresearch_phase3_log.txt
View File

@ -340,3 +340,8 @@
[2026-04-14 22:40:23] [Wave3] Only 0 results — using random proposal. [2026-04-14 22:40:23] [Wave3] Only 0 results — using random proposal.
[2026-04-14 22:40:23] [Champion] 🏆 NEW BEST! Trial 3: score=1500.00 (mini_monaco=1500.0) params={'learning_rate': 0.0002, 'steps_per_switch': 8000, 'total_timesteps': 150000} [2026-04-14 22:40:23] [Champion] 🏆 NEW BEST! Trial 3: score=1500.00 (mini_monaco=1500.0) params={'learning_rate': 0.0002, 'steps_per_switch': 8000, 'total_timesteps': 150000}
[2026-04-14 22:40:23] [Champion] 🏆 NEW BEST! Trial 1: score=2000.00 (mini_monaco=2000.0) params={} [2026-04-14 22:40:23] [Champion] 🏆 NEW BEST! Trial 1: score=2000.00 (mini_monaco=2000.0) params={}
[2026-04-14 22:44:13] [Wave3] Seed trial 1/2: using hardcoded params.
[2026-04-14 22:44:13] [Wave3] Seed trial 2/2: using hardcoded params.
[2026-04-14 22:44:13] [Wave3] Only 0 results — using random proposal.
[2026-04-14 22:44:13] [Champion] 🏆 NEW BEST! Trial 3: score=1500.00 (mini_monaco=1500.0) params={'learning_rate': 0.0002, 'steps_per_switch': 8000, 'total_timesteps': 150000}
[2026-04-14 22:44:13] [Champion] 🏆 NEW BEST! Trial 1: score=2000.00 (mini_monaco=2000.0) params={}

18
agent/outerloop-results/autoresearch_phase4_log.txt Normal file
View File

@ -0,0 +1,18 @@
[2026-04-14 22:40:44] =================================================================
[2026-04-14 22:40:44] [Wave4] Multi-Track Autoresearch — GP+UCB Generalization Search
[2026-04-14 22:40:44] [Wave4] Training tracks : generated_track, mountain_track (no generated_road, no warm-start)
[2026-04-14 22:40:44] [Wave4] Test tracks : mini_monaco only (zero-shot; warren removed — broken done condition)
[2026-04-14 22:40:44] [Wave4] Max trials : 25 | kappa=2.0 | push every 5
[2026-04-14 22:40:44] [Wave4] Results file : /home/paulh/projects/donkeycar-rl-autoresearch/agent/outerloop-results/autoresearch_results_phase4.jsonl
[2026-04-14 22:40:44] [Wave4] Champion dir : /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/wave4-champion
[2026-04-14 22:40:44] [Wave4] Warm start : NONE (training from scratch each trial)
[2026-04-14 22:40:44] =================================================================
[2026-04-14 22:40:44] [Wave4] Loaded 0 existing Phase 3 results.
[2026-04-14 22:40:44] [Wave4] No Wave 3 champion yet.
[2026-04-14 22:40:44] [Wave4] Starting from trial 1.
[2026-04-14 22:40:44]
[Wave4] ========== Trial 1/25 ==========
[2026-04-14 22:40:44] [Wave4] Seed trial 1/2: using hardcoded params.
[2026-04-14 22:40:44] [Wave4] Proposed params: {'learning_rate': 0.0003, 'steps_per_switch': 6000, 'total_timesteps': 80000}
[2026-04-14 22:40:46] [Wave4] Launching trial 1: {'learning_rate': 0.0003, 'steps_per_switch': 6000, 'total_timesteps': 80000}
[2026-04-14 22:40:46] [Wave4] Command: python3 /home/paulh/projects/donkeycar-rl-autoresearch/agent/multitrack_runner.py --total-timesteps 80000 --steps-per-switch 6000 --learning-rate 0.0003 --eval-episodes 3 --save-dir /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/wave4-trial-0001