/home/paulh/.local/lib/python3.10/site-packages/matplotlib/projections/__init__.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available. warnings.warn("Unable to import Axes3D. This may be due to multiple versions of " Gym has been unmaintained since 2022 and does not support NumPy 2.0 amongst other critical functionality. Please upgrade to Gymnasium, the maintained drop-in replacement of Gym, or contact the authors of your software and request that they upgrade. Users of this version of Gym should be able to simply replace 'import gym' with 'import gymnasium as gym' in the vast majority of cases. See the migration guide at https://gymnasium.farama.org/introduction/migration_guide/ for additional information. [16:07:21] ============================================================ [16:07:21] Exp 23: generated_road — clean barriers, clean reward [16:07:21] Sim: localhost:9091 -> generated_road [16:07:21] throttle_min=0.2, lr=0.0003, total=200,000 [16:07:21] Reward: v7 (speed×CTE, efficiency gate, no-progress kill) [16:07:21] Max stuck: 5.0s, episode cap: 120.0s (safety net) [16:07:21] Progress patience: 100 steps [16:07:21] Checkpoints every 10,000 steps [16:07:21] ============================================================ [16:07:21] Creating DummyVecEnv on generated_road... INFO:gym_donkeycar.core.client:connecting to localhost:9091 /home/paulh/.local/lib/python3.10/site-packages/gymnasium/spaces/box.py:236: UserWarning: WARN: Box low's precision lowered by casting to float32, current low.dtype=float64 gym.logger.warn( /home/paulh/.local/lib/python3.10/site-packages/gymnasium/spaces/box.py:306: UserWarning: WARN: Box high's precision lowered by casting to float32, current high.dtype=float64 gym.logger.warn( WARNING:gym_donkeycar.envs.donkey_sim:waiting for sim to start.. INFO:gym_donkeycar.envs.donkey_sim:on need car config INFO:gym_donkeycar.envs.donkey_sim:sending car config. INFO:gym_donkeycar.envs.donkey_sim:sim started! starting DonkeyGym env Setting default: start_delay 5.0 Setting default: max_cte 8.0 Setting default: frame_skip 1 Setting default: cam_resolution (120, 160, 3) Setting default: log_level 20 Setting default: steer_limit 1.0 Setting default: throttle_min 0.0 Setting default: throttle_max 1.0 loading scene generated_road [16:07:22] VecEnv num_envs=1, obs=(3, 120, 160) Using cpu device [16:07:23] Fresh PPO model created. Starting training... INFO:exp23:[16:07:23] ============================================================ INFO:exp23:[16:07:23] Exp 23 started — PID 649531 INFO:exp23:[16:07:23] Log: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/run_2026-05-05_160723_clean.log INFO:exp23:[16:07:23] ============================================================ ----------------------------- | time/ | | | fps | 27 | | iterations | 1 | | time_elapsed | 73 | | total_timesteps | 2048 | ----------------------------- ----------------------------------------- | time/ | | | fps | 21 | | iterations | 2 | | time_elapsed | 193 | | total_timesteps | 4096 | | train/ | | | approx_kl | 0.012727882 | | clip_fraction | 0.0876 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.0534 | | learning_rate | 0.0003 | | loss | 0.0493 | | n_updates | 10 | | policy_gradient_loss | -0.011 | | std | 1.01 | | value_loss | 0.666 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 20 | | iterations | 3 | | time_elapsed | 302 | | total_timesteps | 6144 | | train/ | | | approx_kl | 0.009811729 | | clip_fraction | 0.137 | | clip_range | 0.2 | | entropy_loss | -2.86 | | explained_variance | 0.568 | | learning_rate | 0.0003 | | loss | 0.103 | | n_updates | 20 | | policy_gradient_loss | -0.0206 | | std | 1.02 | | value_loss | 0.318 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 20 | | iterations | 4 | | time_elapsed | 402 | | total_timesteps | 8192 | | train/ | | | approx_kl | 0.015685663 | | clip_fraction | 0.147 | | clip_range | 0.2 | | entropy_loss | -2.87 | | explained_variance | 0.532 | | learning_rate | 0.0003 | | loss | 0.15 | | n_updates | 30 | | policy_gradient_loss | -0.025 | | std | 1.01 | | value_loss | 0.679 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 20 | | iterations | 5 | | time_elapsed | 500 | | total_timesteps | 10240 | | train/ | | | approx_kl | 0.016000155 | | clip_fraction | 0.166 | | clip_range | 0.2 | | entropy_loss | -2.86 | | explained_variance | 0.339 | | learning_rate | 0.0003 | | loss | 0.744 | | n_updates | 40 | | policy_gradient_loss | -0.0195 | | std | 1.01 | | value_loss | 1.54 | ----------------------------------------- INFO:exp23:[16:16:30] [10,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0010000.zip INFO:exp23:[16:16:57] Eval: gen_road=403.8r/2000s ✅ INFO:exp23:[16:16:57] NEW BEST: steps=2000 reward=403.8 ------------------------------ | time/ | | | fps | 52 | | iterations | 1 | | time_elapsed | 38 | | total_timesteps | 12288 | ------------------------------ ----------------------------------------- | time/ | | | fps | 31 | | iterations | 2 | | time_elapsed | 129 | | total_timesteps | 14336 | | train/ | | | approx_kl | 0.018563159 | | clip_fraction | 0.223 | | clip_range | 0.2 | | entropy_loss | -2.85 | | explained_variance | 0.25 | | learning_rate | 0.0003 | | loss | 0.973 | | n_updates | 60 | | policy_gradient_loss | -0.0122 | | std | 1 | | value_loss | 2.35 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 28 | | iterations | 3 | | time_elapsed | 212 | | total_timesteps | 16384 | | train/ | | | approx_kl | 0.015946057 | | clip_fraction | 0.161 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.295 | | learning_rate | 0.0003 | | loss | 1.53 | | n_updates | 70 | | policy_gradient_loss | -0.0121 | | std | 1 | | value_loss | 2.79 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 28 | | iterations | 4 | | time_elapsed | 287 | | total_timesteps | 18432 | | train/ | | | approx_kl | 0.01855317 | | clip_fraction | 0.2 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.3 | | learning_rate | 0.0003 | | loss | 1.27 | | n_updates | 80 | | policy_gradient_loss | -0.0168 | | std | 1 | | value_loss | 3.01 | ---------------------------------------- ----------------------------------------- | time/ | | | fps | 27 | | iterations | 5 | | time_elapsed | 369 | | total_timesteps | 20480 | | train/ | | | approx_kl | 0.017487168 | | clip_fraction | 0.193 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.243 | | learning_rate | 0.0003 | | loss | 1.19 | | n_updates | 90 | | policy_gradient_loss | -0.0139 | | std | 1 | | value_loss | 3.11 | ----------------------------------------- INFO:exp23:[16:23:49] [20,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0020000.zip INFO:exp23:[16:24:16] Eval: gen_road=376.2r/2000s ✅ ------------------------------ | time/ | | | fps | 64 | | iterations | 1 | | time_elapsed | 31 | | total_timesteps | 22528 | ------------------------------ ----------------------------------------- | time/ | | | fps | 38 | | iterations | 2 | | time_elapsed | 107 | | total_timesteps | 24576 | | train/ | | | approx_kl | 0.027157893 | | clip_fraction | 0.238 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.173 | | learning_rate | 0.0003 | | loss | 1.74 | | n_updates | 110 | | policy_gradient_loss | -0.0072 | | std | 1 | | value_loss | 3.58 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 32 | | iterations | 3 | | time_elapsed | 186 | | total_timesteps | 26624 | | train/ | | | approx_kl | 0.019351475 | | clip_fraction | 0.286 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.132 | | learning_rate | 0.0003 | | loss | 2.04 | | n_updates | 120 | | policy_gradient_loss | -0.0109 | | std | 1.01 | | value_loss | 4.1 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 30 | | iterations | 4 | | time_elapsed | 267 | | total_timesteps | 28672 | | train/ | | | approx_kl | 0.017389052 | | clip_fraction | 0.241 | | clip_range | 0.2 | | entropy_loss | -2.85 | | explained_variance | 0.22 | | learning_rate | 0.0003 | | loss | 1.42 | | n_updates | 130 | | policy_gradient_loss | -0.00863 | | std | 1.01 | | value_loss | 4.1 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 28 | | iterations | 5 | | time_elapsed | 355 | | total_timesteps | 30720 | | train/ | | | approx_kl | 0.020130686 | | clip_fraction | 0.263 | | clip_range | 0.2 | | entropy_loss | -2.85 | | explained_variance | 0.0826 | | learning_rate | 0.0003 | | loss | 3.3 | | n_updates | 140 | | policy_gradient_loss | -0.0125 | | std | 1.01 | | value_loss | 6.85 | ----------------------------------------- INFO:exp23:[16:31:10] [30,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0030000.zip INFO:exp23:[16:31:28] Eval: gen_road=289.7r/1219s ❌@1219 ------------------------------ | time/ | | | fps | 66 | | iterations | 1 | | time_elapsed | 30 | | total_timesteps | 32768 | ------------------------------ ----------------------------------------- | time/ | | | fps | 34 | | iterations | 2 | | time_elapsed | 118 | | total_timesteps | 34816 | | train/ | | | approx_kl | 0.022478392 | | clip_fraction | 0.244 | | clip_range | 0.2 | | entropy_loss | -2.85 | | explained_variance | 0.179 | | learning_rate | 0.0003 | | loss | 0.678 | | n_updates | 160 | | policy_gradient_loss | -0.0126 | | std | 1 | | value_loss | 3.3 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 32 | | iterations | 3 | | time_elapsed | 187 | | total_timesteps | 36864 | | train/ | | | approx_kl | 0.030043777 | | clip_fraction | 0.289 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.275 | | learning_rate | 0.0003 | | loss | 2.52 | | n_updates | 170 | | policy_gradient_loss | -0.00318 | | std | 1 | | value_loss | 6.43 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 32 | | iterations | 4 | | time_elapsed | 254 | | total_timesteps | 38912 | | train/ | | | approx_kl | 0.020345446 | | clip_fraction | 0.236 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.361 | | learning_rate | 0.0003 | | loss | 0.235 | | n_updates | 180 | | policy_gradient_loss | -0.00844 | | std | 1 | | value_loss | 2.42 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 31 | | iterations | 5 | | time_elapsed | 325 | | total_timesteps | 40960 | | train/ | | | approx_kl | 0.024092756 | | clip_fraction | 0.237 | | clip_range | 0.2 | | entropy_loss | -2.84 | | explained_variance | 0.486 | | learning_rate | 0.0003 | | loss | 1.78 | | n_updates | 190 | | policy_gradient_loss | -0.0108 | | std | 1 | | value_loss | 1.99 | ----------------------------------------- INFO:exp23:[16:37:29] [40,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0040000.zip INFO:exp23:[16:37:56] Eval: gen_road=452.1r/1951s ❌@1951 ------------------------------ | time/ | | | fps | 73 | | iterations | 1 | | time_elapsed | 28 | | total_timesteps | 43008 | ------------------------------ ----------------------------------------- | time/ | | | fps | 43 | | iterations | 2 | | time_elapsed | 93 | | total_timesteps | 45056 | | train/ | | | approx_kl | 0.027982034 | | clip_fraction | 0.262 | | clip_range | 0.2 | | entropy_loss | -2.87 | | explained_variance | 0.185 | | learning_rate | 0.0003 | | loss | 0.42 | | n_updates | 210 | | policy_gradient_loss | -0.00818 | | std | 1.02 | | value_loss | 3.07 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 37 | | iterations | 3 | | time_elapsed | 161 | | total_timesteps | 47104 | | train/ | | | approx_kl | 0.053084552 | | clip_fraction | 0.33 | | clip_range | 0.2 | | entropy_loss | -2.87 | | explained_variance | 0.103 | | learning_rate | 0.0003 | | loss | 0.131 | | n_updates | 220 | | policy_gradient_loss | 0.00247 | | std | 1.02 | | value_loss | 1.6 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 35 | | iterations | 4 | | time_elapsed | 231 | | total_timesteps | 49152 | | train/ | | | approx_kl | 0.018902654 | | clip_fraction | 0.215 | | clip_range | 0.2 | | entropy_loss | -2.87 | | explained_variance | 0.495 | | learning_rate | 0.0003 | | loss | 0.22 | | n_updates | 230 | | policy_gradient_loss | -0.00188 | | std | 1.01 | | value_loss | 1.51 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 34 | | iterations | 5 | | time_elapsed | 298 | | total_timesteps | 51200 | | train/ | | | approx_kl | 0.020705111 | | clip_fraction | 0.244 | | clip_range | 0.2 | | entropy_loss | -2.86 | | explained_variance | 0.63 | | learning_rate | 0.0003 | | loss | 0.466 | | n_updates | 240 | | policy_gradient_loss | -0.00597 | | std | 1.01 | | value_loss | 1.45 | ----------------------------------------- INFO:exp23:[16:43:32] [50,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0050000.zip INFO:exp23:[16:43:57] Eval: gen_road=457.1r/1753s ❌@1753 ------------------------------ | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 53248 | ------------------------------ ----------------------------------------- | time/ | | | fps | 44 | | iterations | 2 | | time_elapsed | 91 | | total_timesteps | 55296 | | train/ | | | approx_kl | 0.030220592 | | clip_fraction | 0.27 | | clip_range | 0.2 | | entropy_loss | -2.86 | | explained_variance | 0.69 | | learning_rate | 0.0003 | | loss | 0.509 | | n_updates | 260 | | policy_gradient_loss | -0.0111 | | std | 1.01 | | value_loss | 1.48 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 38 | | iterations | 3 | | time_elapsed | 161 | | total_timesteps | 57344 | | train/ | | | approx_kl | 0.024165533 | | clip_fraction | 0.275 | | clip_range | 0.2 | | entropy_loss | -2.87 | | explained_variance | 0.649 | | learning_rate | 0.0003 | | loss | 0.0865 | | n_updates | 270 | | policy_gradient_loss | -0.0089 | | std | 1.02 | | value_loss | 1.54 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 36 | | iterations | 4 | | time_elapsed | 226 | | total_timesteps | 59392 | | train/ | | | approx_kl | 0.022396056 | | clip_fraction | 0.317 | | clip_range | 0.2 | | entropy_loss | -2.89 | | explained_variance | 0.179 | | learning_rate | 0.0003 | | loss | 2.02 | | n_updates | 280 | | policy_gradient_loss | 0.00141 | | std | 1.04 | | value_loss | 3.98 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 34 | | iterations | 5 | | time_elapsed | 293 | | total_timesteps | 61440 | | train/ | | | approx_kl | 0.03801451 | | clip_fraction | 0.328 | | clip_range | 0.2 | | entropy_loss | -2.9 | | explained_variance | 0.149 | | learning_rate | 0.0003 | | loss | 1.31 | | n_updates | 290 | | policy_gradient_loss | 0.00172 | | std | 1.03 | | value_loss | 4.42 | ---------------------------------------- INFO:exp23:[16:49:28] [60,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0060000.zip INFO:exp23:[16:49:32] Eval: gen_road=31.7r/119s ❌@119 ------------------------------ | time/ | | | fps | 70 | | iterations | 1 | | time_elapsed | 29 | | total_timesteps | 63488 | ------------------------------ ----------------------------------------- | time/ | | | fps | 42 | | iterations | 2 | | time_elapsed | 95 | | total_timesteps | 65536 | | train/ | | | approx_kl | 0.042009473 | | clip_fraction | 0.323 | | clip_range | 0.2 | | entropy_loss | -2.91 | | explained_variance | 0.387 | | learning_rate | 0.0003 | | loss | 3.26 | | n_updates | 310 | | policy_gradient_loss | -0.00147 | | std | 1.04 | | value_loss | 6.36 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 37 | | iterations | 3 | | time_elapsed | 161 | | total_timesteps | 67584 | | train/ | | | approx_kl | 0.037721604 | | clip_fraction | 0.274 | | clip_range | 0.2 | | entropy_loss | -2.91 | | explained_variance | 0.277 | | learning_rate | 0.0003 | | loss | 3.1 | | n_updates | 320 | | policy_gradient_loss | -0.00803 | | std | 1.04 | | value_loss | 5.67 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 35 | | iterations | 4 | | time_elapsed | 231 | | total_timesteps | 69632 | | train/ | | | approx_kl | 0.024771407 | | clip_fraction | 0.339 | | clip_range | 0.2 | | entropy_loss | -2.91 | | explained_variance | 0.0675 | | learning_rate | 0.0003 | | loss | 1.94 | | n_updates | 330 | | policy_gradient_loss | -0.000833 | | std | 1.04 | | value_loss | 5.25 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 34 | | iterations | 5 | | time_elapsed | 298 | | total_timesteps | 71680 | | train/ | | | approx_kl | 0.04476459 | | clip_fraction | 0.309 | | clip_range | 0.2 | | entropy_loss | -2.92 | | explained_variance | 0.342 | | learning_rate | 0.0003 | | loss | 4.02 | | n_updates | 340 | | policy_gradient_loss | -0.00952 | | std | 1.05 | | value_loss | 8.05 | ---------------------------------------- INFO:exp23:[16:55:06] [70,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0070000.zip INFO:exp23:[16:55:11] Eval: gen_road=50.2r/171s ❌@171 ------------------------------ | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 73728 | ------------------------------ ---------------------------------------- | time/ | | | fps | 43 | | iterations | 2 | | time_elapsed | 93 | | total_timesteps | 75776 | | train/ | | | approx_kl | 0.03761123 | | clip_fraction | 0.356 | | clip_range | 0.2 | | entropy_loss | -2.94 | | explained_variance | 0.265 | | learning_rate | 0.0003 | | loss | 0.905 | | n_updates | 360 | | policy_gradient_loss | 0.00262 | | std | 1.05 | | value_loss | 1.53 | ---------------------------------------- ----------------------------------------- | time/ | | | fps | 38 | | iterations | 3 | | time_elapsed | 158 | | total_timesteps | 77824 | | train/ | | | approx_kl | 0.038256083 | | clip_fraction | 0.364 | | clip_range | 0.2 | | entropy_loss | -2.95 | | explained_variance | 0.284 | | learning_rate | 0.0003 | | loss | 0.965 | | n_updates | 370 | | policy_gradient_loss | 0.00265 | | std | 1.06 | | value_loss | 4.81 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 36 | | iterations | 4 | | time_elapsed | 222 | | total_timesteps | 79872 | | train/ | | | approx_kl | 0.04706876 | | clip_fraction | 0.376 | | clip_range | 0.2 | | entropy_loss | -2.97 | | explained_variance | 0.293 | | learning_rate | 0.0003 | | loss | 1.2 | | n_updates | 380 | | policy_gradient_loss | 0.00318 | | std | 1.08 | | value_loss | 4.1 | ---------------------------------------- --------------------------------------- | time/ | | | fps | 35 | | iterations | 5 | | time_elapsed | 287 | | total_timesteps | 81920 | | train/ | | | approx_kl | 0.0504843 | | clip_fraction | 0.291 | | clip_range | 0.2 | | entropy_loss | -2.98 | | explained_variance | 0.584 | | learning_rate | 0.0003 | | loss | 0.356 | | n_updates | 390 | | policy_gradient_loss | -0.00369 | | std | 1.07 | | value_loss | 1.98 | --------------------------------------- INFO:exp23:[17:00:34] [80,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0080000.zip INFO:exp23:[17:00:55] Eval: gen_road=450.0r/1501s ❌@1501 ------------------------------ | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 83968 | ------------------------------ ----------------------------------------- | time/ | | | fps | 39 | | iterations | 2 | | time_elapsed | 104 | | total_timesteps | 86016 | | train/ | | | approx_kl | 0.041178867 | | clip_fraction | 0.338 | | clip_range | 0.2 | | entropy_loss | -3 | | explained_variance | 0.406 | | learning_rate | 0.0003 | | loss | 0.39 | | n_updates | 410 | | policy_gradient_loss | -0.00519 | | std | 1.09 | | value_loss | 1.1 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 34 | | iterations | 3 | | time_elapsed | 175 | | total_timesteps | 88064 | | train/ | | | approx_kl | 0.040968597 | | clip_fraction | 0.349 | | clip_range | 0.2 | | entropy_loss | -3.01 | | explained_variance | 0.561 | | learning_rate | 0.0003 | | loss | 0.118 | | n_updates | 420 | | policy_gradient_loss | 0.00356 | | std | 1.09 | | value_loss | 1.68 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 33 | | iterations | 4 | | time_elapsed | 242 | | total_timesteps | 90112 | | train/ | | | approx_kl | 0.038171332 | | clip_fraction | 0.315 | | clip_range | 0.2 | | entropy_loss | -3.02 | | explained_variance | 0.23 | | learning_rate | 0.0003 | | loss | 0.483 | | n_updates | 430 | | policy_gradient_loss | -0.00498 | | std | 1.11 | | value_loss | 2.82 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 32 | | iterations | 5 | | time_elapsed | 319 | | total_timesteps | 92160 | | train/ | | | approx_kl | 0.036780134 | | clip_fraction | 0.344 | | clip_range | 0.2 | | entropy_loss | -3.04 | | explained_variance | 0.531 | | learning_rate | 0.0003 | | loss | 0.554 | | n_updates | 440 | | policy_gradient_loss | -0.000642 | | std | 1.12 | | value_loss | 3.38 | ----------------------------------------- INFO:exp23:[17:06:55] [90,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0090000.zip INFO:exp23:[17:07:16] Eval: gen_road=428.4r/1436s ❌@1436 ------------------------------ | time/ | | | fps | 70 | | iterations | 1 | | time_elapsed | 29 | | total_timesteps | 94208 | ------------------------------ ----------------------------------------- | time/ | | | fps | 41 | | iterations | 2 | | time_elapsed | 99 | | total_timesteps | 96256 | | train/ | | | approx_kl | 0.064061046 | | clip_fraction | 0.368 | | clip_range | 0.2 | | entropy_loss | -3.07 | | explained_variance | 0.106 | | learning_rate | 0.0003 | | loss | 0.313 | | n_updates | 460 | | policy_gradient_loss | -0.000814 | | std | 1.13 | | value_loss | 3.79 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 36 | | iterations | 3 | | time_elapsed | 166 | | total_timesteps | 98304 | | train/ | | | approx_kl | 0.03818226 | | clip_fraction | 0.337 | | clip_range | 0.2 | | entropy_loss | -3.07 | | explained_variance | 0.277 | | learning_rate | 0.0003 | | loss | 1.09 | | n_updates | 470 | | policy_gradient_loss | -0.000984 | | std | 1.13 | | value_loss | 4.3 | ---------------------------------------- ---------------------------------------- | time/ | | | fps | 34 | | iterations | 4 | | time_elapsed | 234 | | total_timesteps | 100352 | | train/ | | | approx_kl | 0.07311188 | | clip_fraction | 0.373 | | clip_range | 0.2 | | entropy_loss | -3.08 | | explained_variance | 0.155 | | learning_rate | 0.0003 | | loss | 0.266 | | n_updates | 480 | | policy_gradient_loss | -0.00255 | | std | 1.13 | | value_loss | 1.48 | ---------------------------------------- ----------------------------------------- | time/ | | | fps | 33 | | iterations | 5 | | time_elapsed | 301 | | total_timesteps | 102400 | | train/ | | | approx_kl | 0.046712708 | | clip_fraction | 0.326 | | clip_range | 0.2 | | entropy_loss | -3.08 | | explained_variance | 0.704 | | learning_rate | 0.0003 | | loss | 0.166 | | n_updates | 490 | | policy_gradient_loss | -0.0018 | | std | 1.14 | | value_loss | 1.42 | ----------------------------------------- INFO:exp23:[17:12:54] [100,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0100000.zip INFO:exp23:[17:13:18] Eval: gen_road=464.0r/1700s ❌@1700 ------------------------------- | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 104448 | ------------------------------- ----------------------------------------- | time/ | | | fps | 43 | | iterations | 2 | | time_elapsed | 94 | | total_timesteps | 106496 | | train/ | | | approx_kl | 0.034854777 | | clip_fraction | 0.304 | | clip_range | 0.2 | | entropy_loss | -3.08 | | explained_variance | 0.701 | | learning_rate | 0.0003 | | loss | 0.148 | | n_updates | 510 | | policy_gradient_loss | -0.011 | | std | 1.14 | | value_loss | 1.25 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 38 | | iterations | 3 | | time_elapsed | 161 | | total_timesteps | 108544 | | train/ | | | approx_kl | 0.045809295 | | clip_fraction | 0.347 | | clip_range | 0.2 | | entropy_loss | -3.08 | | explained_variance | 0.278 | | learning_rate | 0.0003 | | loss | 0.927 | | n_updates | 520 | | policy_gradient_loss | -0.00439 | | std | 1.14 | | value_loss | 2.56 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 35 | | iterations | 4 | | time_elapsed | 233 | | total_timesteps | 110592 | | train/ | | | approx_kl | 0.043633107 | | clip_fraction | 0.362 | | clip_range | 0.2 | | entropy_loss | -3.09 | | explained_variance | 0.604 | | learning_rate | 0.0003 | | loss | 0.117 | | n_updates | 530 | | policy_gradient_loss | -0.00376 | | std | 1.14 | | value_loss | 1.07 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 34 | | iterations | 5 | | time_elapsed | 300 | | total_timesteps | 112640 | | train/ | | | approx_kl | 0.044127725 | | clip_fraction | 0.314 | | clip_range | 0.2 | | entropy_loss | -3.1 | | explained_variance | 0.664 | | learning_rate | 0.0003 | | loss | 0.591 | | n_updates | 540 | | policy_gradient_loss | -0.00432 | | std | 1.16 | | value_loss | 3.08 | ----------------------------------------- INFO:exp23:[17:19:03] [110,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0110000.zip INFO:exp23:[17:19:16] Eval: gen_road=251.4r/823s ❌@823 ------------------------------- | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 114688 | ------------------------------- ----------------------------------------- | time/ | | | fps | 39 | | iterations | 2 | | time_elapsed | 103 | | total_timesteps | 116736 | | train/ | | | approx_kl | 0.022497533 | | clip_fraction | 0.31 | | clip_range | 0.2 | | entropy_loss | -3.12 | | explained_variance | 0.812 | | learning_rate | 0.0003 | | loss | 0.347 | | n_updates | 560 | | policy_gradient_loss | -0.0121 | | std | 1.16 | | value_loss | 1.31 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 35 | | iterations | 3 | | time_elapsed | 174 | | total_timesteps | 118784 | | train/ | | | approx_kl | 0.04331164 | | clip_fraction | 0.343 | | clip_range | 0.2 | | entropy_loss | -3.13 | | explained_variance | 0.603 | | learning_rate | 0.0003 | | loss | 3.45 | | n_updates | 570 | | policy_gradient_loss | -0.0071 | | std | 1.17 | | value_loss | 3.27 | ---------------------------------------- ---------------------------------------- | time/ | | | fps | 34 | | iterations | 4 | | time_elapsed | 239 | | total_timesteps | 120832 | | train/ | | | approx_kl | 0.06629866 | | clip_fraction | 0.358 | | clip_range | 0.2 | | entropy_loss | -3.14 | | explained_variance | 0.731 | | learning_rate | 0.0003 | | loss | 0.298 | | n_updates | 580 | | policy_gradient_loss | -0.00309 | | std | 1.17 | | value_loss | 1.43 | ---------------------------------------- ---------------------------------------- | time/ | | | fps | 33 | | iterations | 5 | | time_elapsed | 304 | | total_timesteps | 122880 | | train/ | | | approx_kl | 0.05148594 | | clip_fraction | 0.378 | | clip_range | 0.2 | | entropy_loss | -3.14 | | explained_variance | 0.289 | | learning_rate | 0.0003 | | loss | 1.4 | | n_updates | 590 | | policy_gradient_loss | -0.00705 | | std | 1.17 | | value_loss | 5.39 | ---------------------------------------- INFO:exp23:[17:25:04] [120,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0120000.zip INFO:exp23:[17:25:08] Eval: gen_road=27.4r/107s ❌@107 ------------------------------- | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 124928 | ------------------------------- ----------------------------------------- | time/ | | | fps | 38 | | iterations | 2 | | time_elapsed | 107 | | total_timesteps | 126976 | | train/ | | | approx_kl | 0.032537233 | | clip_fraction | 0.395 | | clip_range | 0.2 | | entropy_loss | -3.17 | | explained_variance | 0.693 | | learning_rate | 0.0003 | | loss | 0.432 | | n_updates | 610 | | policy_gradient_loss | -0.00625 | | std | 1.19 | | value_loss | 0.953 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 35 | | iterations | 3 | | time_elapsed | 172 | | total_timesteps | 129024 | | train/ | | | approx_kl | 0.06681977 | | clip_fraction | 0.363 | | clip_range | 0.2 | | entropy_loss | -3.18 | | explained_variance | 0.135 | | learning_rate | 0.0003 | | loss | 0.782 | | n_updates | 620 | | policy_gradient_loss | -0.00293 | | std | 1.2 | | value_loss | 6.05 | ---------------------------------------- ----------------------------------------- | time/ | | | fps | 34 | | iterations | 4 | | time_elapsed | 240 | | total_timesteps | 131072 | | train/ | | | approx_kl | 0.044004865 | | clip_fraction | 0.36 | | clip_range | 0.2 | | entropy_loss | -3.2 | | explained_variance | 0.352 | | learning_rate | 0.0003 | | loss | 2.19 | | n_updates | 630 | | policy_gradient_loss | -0.00463 | | std | 1.22 | | value_loss | 4.66 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 33 | | iterations | 5 | | time_elapsed | 304 | | total_timesteps | 133120 | | train/ | | | approx_kl | 0.06260415 | | clip_fraction | 0.405 | | clip_range | 0.2 | | entropy_loss | -3.22 | | explained_variance | 0.164 | | learning_rate | 0.0003 | | loss | 0.2 | | n_updates | 640 | | policy_gradient_loss | -0.00158 | | std | 1.22 | | value_loss | 1.97 | ---------------------------------------- INFO:exp23:[17:30:53] [130,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0130000.zip INFO:exp23:[17:30:58] Eval: gen_road=49.5r/165s ❌@165 ------------------------------- | time/ | | | fps | 73 | | iterations | 1 | | time_elapsed | 28 | | total_timesteps | 135168 | ------------------------------- ----------------------------------------- | time/ | | | fps | 43 | | iterations | 2 | | time_elapsed | 94 | | total_timesteps | 137216 | | train/ | | | approx_kl | 0.049958713 | | clip_fraction | 0.363 | | clip_range | 0.2 | | entropy_loss | -3.26 | | explained_variance | 0.129 | | learning_rate | 0.0003 | | loss | 4.09 | | n_updates | 660 | | policy_gradient_loss | -0.000775 | | std | 1.25 | | value_loss | 3.86 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 38 | | iterations | 3 | | time_elapsed | 159 | | total_timesteps | 139264 | | train/ | | | approx_kl | 0.045727327 | | clip_fraction | 0.344 | | clip_range | 0.2 | | entropy_loss | -3.26 | | explained_variance | 0.273 | | learning_rate | 0.0003 | | loss | 2.35 | | n_updates | 670 | | policy_gradient_loss | 0.00478 | | std | 1.25 | | value_loss | 11.4 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 36 | | iterations | 4 | | time_elapsed | 222 | | total_timesteps | 141312 | | train/ | | | approx_kl | 0.041530177 | | clip_fraction | 0.37 | | clip_range | 0.2 | | entropy_loss | -3.26 | | explained_variance | 0.243 | | learning_rate | 0.0003 | | loss | 1.47 | | n_updates | 680 | | policy_gradient_loss | -0.00742 | | std | 1.24 | | value_loss | 4.23 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 35 | | iterations | 5 | | time_elapsed | 288 | | total_timesteps | 143360 | | train/ | | | approx_kl | 0.04864549 | | clip_fraction | 0.383 | | clip_range | 0.2 | | entropy_loss | -3.25 | | explained_variance | 0.485 | | learning_rate | 0.0003 | | loss | 0.319 | | n_updates | 690 | | policy_gradient_loss | -0.00376 | | std | 1.24 | | value_loss | 1.21 | ---------------------------------------- INFO:exp23:[17:36:25] [140,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0140000.zip INFO:exp23:[17:36:46] Eval: gen_road=466.1r/1496s ❌@1496 ------------------------------- | time/ | | | fps | 73 | | iterations | 1 | | time_elapsed | 27 | | total_timesteps | 145408 | ------------------------------- ----------------------------------------- | time/ | | | fps | 44 | | iterations | 2 | | time_elapsed | 92 | | total_timesteps | 147456 | | train/ | | | approx_kl | 0.057353795 | | clip_fraction | 0.38 | | clip_range | 0.2 | | entropy_loss | -3.26 | | explained_variance | 0.616 | | learning_rate | 0.0003 | | loss | 0.264 | | n_updates | 710 | | policy_gradient_loss | -0.00505 | | std | 1.26 | | value_loss | 2.85 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 37 | | iterations | 3 | | time_elapsed | 165 | | total_timesteps | 149504 | | train/ | | | approx_kl | 0.041733697 | | clip_fraction | 0.38 | | clip_range | 0.2 | | entropy_loss | -3.28 | | explained_variance | 0.76 | | learning_rate | 0.0003 | | loss | 0.291 | | n_updates | 720 | | policy_gradient_loss | -0.000579 | | std | 1.26 | | value_loss | 1.58 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 34 | | iterations | 4 | | time_elapsed | 239 | | total_timesteps | 151552 | | train/ | | | approx_kl | 0.036947723 | | clip_fraction | 0.381 | | clip_range | 0.2 | | entropy_loss | -3.29 | | explained_variance | 0.721 | | learning_rate | 0.0003 | | loss | 0.259 | | n_updates | 730 | | policy_gradient_loss | 0.00236 | | std | 1.27 | | value_loss | 1.99 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 33 | | iterations | 5 | | time_elapsed | 306 | | total_timesteps | 153600 | | train/ | | | approx_kl | 0.022321431 | | clip_fraction | 0.325 | | clip_range | 0.2 | | entropy_loss | -3.3 | | explained_variance | 0.754 | | learning_rate | 0.0003 | | loss | 0.217 | | n_updates | 740 | | policy_gradient_loss | -0.00604 | | std | 1.28 | | value_loss | 1.85 | ----------------------------------------- INFO:exp23:[17:42:29] [150,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0150000.zip INFO:exp23:[17:42:45] Eval: gen_road=361.8r/1104s ❌@1104 ------------------------------- | time/ | | | fps | 75 | | iterations | 1 | | time_elapsed | 27 | | total_timesteps | 155648 | ------------------------------- ----------------------------------------- | time/ | | | fps | 37 | | iterations | 2 | | time_elapsed | 108 | | total_timesteps | 157696 | | train/ | | | approx_kl | 0.044686228 | | clip_fraction | 0.365 | | clip_range | 0.2 | | entropy_loss | -3.33 | | explained_variance | 0.705 | | learning_rate | 0.0003 | | loss | 0.196 | | n_updates | 760 | | policy_gradient_loss | -0.0126 | | std | 1.29 | | value_loss | 1.28 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 34 | | iterations | 3 | | time_elapsed | 179 | | total_timesteps | 159744 | | train/ | | | approx_kl | 0.055111866 | | clip_fraction | 0.366 | | clip_range | 0.2 | | entropy_loss | -3.33 | | explained_variance | 0.619 | | learning_rate | 0.0003 | | loss | 0.603 | | n_updates | 770 | | policy_gradient_loss | 0.000912 | | std | 1.29 | | value_loss | 3.45 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 33 | | iterations | 4 | | time_elapsed | 244 | | total_timesteps | 161792 | | train/ | | | approx_kl | 0.058212373 | | clip_fraction | 0.384 | | clip_range | 0.2 | | entropy_loss | -3.33 | | explained_variance | 0.789 | | learning_rate | 0.0003 | | loss | 0.187 | | n_updates | 780 | | policy_gradient_loss | -0.00413 | | std | 1.29 | | value_loss | 1.37 | ----------------------------------------- --------------------------------------- | time/ | | | fps | 33 | | iterations | 5 | | time_elapsed | 305 | | total_timesteps | 163840 | | train/ | | | approx_kl | 0.0781488 | | clip_fraction | 0.384 | | clip_range | 0.2 | | entropy_loss | -3.33 | | explained_variance | 0.378 | | learning_rate | 0.0003 | | loss | 2.8 | | n_updates | 790 | | policy_gradient_loss | -0.00318 | | std | 1.3 | | value_loss | 4.7 | --------------------------------------- INFO:exp23:[17:48:24] [160,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0160000.zip INFO:exp23:[17:48:27] Eval: gen_road=29.7r/105s ❌@105 ------------------------------- | time/ | | | fps | 67 | | iterations | 1 | | time_elapsed | 30 | | total_timesteps | 165888 | ------------------------------- ----------------------------------------- | time/ | | | fps | 41 | | iterations | 2 | | time_elapsed | 99 | | total_timesteps | 167936 | | train/ | | | approx_kl | 0.105973095 | | clip_fraction | 0.426 | | clip_range | 0.2 | | entropy_loss | -3.37 | | explained_variance | 0.472 | | learning_rate | 0.0003 | | loss | 2.6 | | n_updates | 810 | | policy_gradient_loss | 0.00471 | | std | 1.33 | | value_loss | 6.32 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 37 | | iterations | 3 | | time_elapsed | 162 | | total_timesteps | 169984 | | train/ | | | approx_kl | 0.07749827 | | clip_fraction | 0.43 | | clip_range | 0.2 | | entropy_loss | -3.39 | | explained_variance | 0.363 | | learning_rate | 0.0003 | | loss | 0.975 | | n_updates | 820 | | policy_gradient_loss | 0.00222 | | std | 1.34 | | value_loss | 8.92 | ---------------------------------------- ---------------------------------------- | time/ | | | fps | 36 | | iterations | 4 | | time_elapsed | 226 | | total_timesteps | 172032 | | train/ | | | approx_kl | 0.05261411 | | clip_fraction | 0.376 | | clip_range | 0.2 | | entropy_loss | -3.41 | | explained_variance | 0.282 | | learning_rate | 0.0003 | | loss | 3.72 | | n_updates | 830 | | policy_gradient_loss | -0.0017 | | std | 1.35 | | value_loss | 7.82 | ---------------------------------------- ----------------------------------------- | time/ | | | fps | 34 | | iterations | 5 | | time_elapsed | 292 | | total_timesteps | 174080 | | train/ | | | approx_kl | 0.059629906 | | clip_fraction | 0.4 | | clip_range | 0.2 | | entropy_loss | -3.43 | | explained_variance | 0.389 | | learning_rate | 0.0003 | | loss | 4.8 | | n_updates | 840 | | policy_gradient_loss | -0.00594 | | std | 1.37 | | value_loss | 7.4 | ----------------------------------------- INFO:exp23:[17:53:58] [170,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0170000.zip INFO:exp23:[17:54:15] Eval: gen_road=364.8r/1129s ❌@1129 ------------------------------- | time/ | | | fps | 75 | | iterations | 1 | | time_elapsed | 27 | | total_timesteps | 176128 | ------------------------------- ----------------------------------------- | time/ | | | fps | 43 | | iterations | 2 | | time_elapsed | 94 | | total_timesteps | 178176 | | train/ | | | approx_kl | 0.052496605 | | clip_fraction | 0.403 | | clip_range | 0.2 | | entropy_loss | -3.48 | | explained_variance | 0.772 | | learning_rate | 0.0003 | | loss | 0.0482 | | n_updates | 860 | | policy_gradient_loss | -0.00453 | | std | 1.4 | | value_loss | 0.785 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 38 | | iterations | 3 | | time_elapsed | 161 | | total_timesteps | 180224 | | train/ | | | approx_kl | 0.053490236 | | clip_fraction | 0.396 | | clip_range | 0.2 | | entropy_loss | -3.49 | | explained_variance | 0.371 | | learning_rate | 0.0003 | | loss | 2.18 | | n_updates | 870 | | policy_gradient_loss | -0.00656 | | std | 1.41 | | value_loss | 4.69 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 35 | | iterations | 4 | | time_elapsed | 228 | | total_timesteps | 182272 | | train/ | | | approx_kl | 0.046204574 | | clip_fraction | 0.416 | | clip_range | 0.2 | | entropy_loss | -3.5 | | explained_variance | 0.484 | | learning_rate | 0.0003 | | loss | 2.56 | | n_updates | 880 | | policy_gradient_loss | -0.00268 | | std | 1.41 | | value_loss | 3.26 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 34 | | iterations | 5 | | time_elapsed | 293 | | total_timesteps | 184320 | | train/ | | | approx_kl | 0.05884172 | | clip_fraction | 0.365 | | clip_range | 0.2 | | entropy_loss | -3.52 | | explained_variance | 0.305 | | learning_rate | 0.0003 | | loss | 0.759 | | n_updates | 890 | | policy_gradient_loss | -0.0112 | | std | 1.42 | | value_loss | 4.66 | ---------------------------------------- INFO:exp23:[17:59:46] [180,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0180000.zip INFO:exp23:[18:00:07] Eval: gen_road=452.4r/1503s ❌@1503 ------------------------------- | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 186368 | ------------------------------- ----------------------------------------- | time/ | | | fps | 44 | | iterations | 2 | | time_elapsed | 92 | | total_timesteps | 188416 | | train/ | | | approx_kl | 0.031784095 | | clip_fraction | 0.329 | | clip_range | 0.2 | | entropy_loss | -3.56 | | explained_variance | 0.761 | | learning_rate | 0.0003 | | loss | 0.162 | | n_updates | 910 | | policy_gradient_loss | -0.0117 | | std | 1.46 | | value_loss | 0.771 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 39 | | iterations | 3 | | time_elapsed | 155 | | total_timesteps | 190464 | | train/ | | | approx_kl | 0.04083346 | | clip_fraction | 0.37 | | clip_range | 0.2 | | entropy_loss | -3.58 | | explained_variance | 0.8 | | learning_rate | 0.0003 | | loss | 0.0687 | | n_updates | 920 | | policy_gradient_loss | -0.00859 | | std | 1.47 | | value_loss | 1.26 | ---------------------------------------- ----------------------------------------- | time/ | | | fps | 37 | | iterations | 4 | | time_elapsed | 217 | | total_timesteps | 192512 | | train/ | | | approx_kl | 0.038500346 | | clip_fraction | 0.315 | | clip_range | 0.2 | | entropy_loss | -3.59 | | explained_variance | 0.85 | | learning_rate | 0.0003 | | loss | 0.633 | | n_updates | 930 | | policy_gradient_loss | -0.00934 | | std | 1.49 | | value_loss | 1.29 | ----------------------------------------- ---------------------------------------- | time/ | | | fps | 36 | | iterations | 5 | | time_elapsed | 280 | | total_timesteps | 194560 | | train/ | | | approx_kl | 0.06231237 | | clip_fraction | 0.387 | | clip_range | 0.2 | | entropy_loss | -3.61 | | explained_variance | 0.156 | | learning_rate | 0.0003 | | loss | 0.403 | | n_updates | 940 | | policy_gradient_loss | -0.00441 | | std | 1.49 | | value_loss | 2.07 | ---------------------------------------- INFO:exp23:[18:05:25] [190,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0190000.zip INFO:exp23:[18:05:49] Eval: gen_road=466.0r/1684s ❌@1684 ------------------------------- | time/ | | | fps | 76 | | iterations | 1 | | time_elapsed | 26 | | total_timesteps | 196608 | ------------------------------- ----------------------------------------- | time/ | | | fps | 43 | | iterations | 2 | | time_elapsed | 93 | | total_timesteps | 198656 | | train/ | | | approx_kl | 0.068352714 | | clip_fraction | 0.406 | | clip_range | 0.2 | | entropy_loss | -3.62 | | explained_variance | 0.165 | | learning_rate | 0.0003 | | loss | 0.609 | | n_updates | 960 | | policy_gradient_loss | -0.00677 | | std | 1.51 | | value_loss | 2.48 | ----------------------------------------- ----------------------------------------- | time/ | | | fps | 38 | | iterations | 3 | | time_elapsed | 158 | | total_timesteps | 200704 | | train/ | | | approx_kl | 0.054212958 | | clip_fraction | 0.389 | | clip_range | 0.2 | | entropy_loss | -3.64 | | explained_variance | 0.232 | | learning_rate | 0.0003 | | loss | 0.221 | | n_updates | 970 | | policy_gradient_loss | -0.0117 | | std | 1.51 | | value_loss | 1.84 | ----------------------------------------- --------------------------------------- | time/ | | | fps | 37 | | iterations | 4 | | time_elapsed | 218 | | total_timesteps | 202752 | | train/ | | | approx_kl | 0.0474802 | | clip_fraction | 0.375 | | clip_range | 0.2 | | entropy_loss | -3.62 | | explained_variance | 0.292 | | learning_rate | 0.0003 | | loss | 0.236 | | n_updates | 980 | | policy_gradient_loss | -0.00871 | | std | 1.49 | | value_loss | 1.7 | --------------------------------------- ----------------------------------------- | time/ | | | fps | 36 | | iterations | 5 | | time_elapsed | 280 | | total_timesteps | 204800 | | train/ | | | approx_kl | 0.045135833 | | clip_fraction | 0.402 | | clip_range | 0.2 | | entropy_loss | -3.62 | | explained_variance | 0.566 | | learning_rate | 0.0003 | | loss | 0.0875 | | n_updates | 990 | | policy_gradient_loss | -0.00507 | | std | 1.5 | | value_loss | 0.959 | ----------------------------------------- INFO:exp23:[18:11:02] [200,000/200,000] Checkpoint saved: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/checkpoint_0200000.zip INFO:exp23:[18:11:14] Eval: gen_road=248.2r/795s ❌@795 INFO:exp23:[18:11:14] ============================================================ INFO:exp23:[18:11:14] FINAL EVALUATION: best_model on generated_road INFO:exp23:[18:11:14] ============================================================ INFO:gym_donkeycar.core.client:connecting to localhost:9091 /home/paulh/.local/lib/python3.10/site-packages/gymnasium/spaces/box.py:236: UserWarning: WARN: Box low's precision lowered by casting to float32, current low.dtype=float64 gym.logger.warn( /home/paulh/.local/lib/python3.10/site-packages/gymnasium/spaces/box.py:306: UserWarning: WARN: Box high's precision lowered by casting to float32, current high.dtype=float64 gym.logger.warn( INFO:gym_donkeycar.envs.donkey_sim:on need car config INFO:gym_donkeycar.envs.donkey_sim:sending car config. INFO:gym_donkeycar.envs.donkey_sim:sim started! INFO:exp23:[18:11:42] Set 1: 409.9r / 2000s ✅ INFO:gym_donkeycar.core.client:connecting to localhost:9091 INFO:gym_donkeycar.envs.donkey_sim:on need car config INFO:gym_donkeycar.envs.donkey_sim:sending car config. INFO:gym_donkeycar.envs.donkey_sim:sim started! INFO:exp23:[18:12:09] Set 2: 407.9r / 2000s ✅ INFO:gym_donkeycar.core.client:connecting to localhost:9091 INFO:gym_donkeycar.envs.donkey_sim:on need car config INFO:gym_donkeycar.envs.donkey_sim:sending car config. INFO:gym_donkeycar.envs.donkey_sim:sim started! INFO:exp23:[18:12:36] Set 3: 407.9r / 2000s ✅ INFO:exp23:[18:12:36] Mean: 2000 steps / 408.6 reward INFO:exp23:[18:12:36] Exp 23 complete. starting DonkeyGym env Setting default: start_delay 5.0 Setting default: max_cte 8.0 Setting default: frame_skip 1 Setting default: cam_resolution (120, 160, 3) Setting default: log_level 20 Setting default: steer_limit 1.0 Setting default: throttle_min 0.0 Setting default: throttle_max 1.0 starting DonkeyGym env Setting default: start_delay 5.0 Setting default: max_cte 8.0 Setting default: frame_skip 1 Setting default: cam_resolution (120, 160, 3) Setting default: log_level 20 Setting default: steer_limit 1.0 Setting default: throttle_min 0.0 Setting default: throttle_max 1.0 starting DonkeyGym env Setting default: start_delay 5.0 Setting default: max_cte 8.0 Setting default: frame_skip 1 Setting default: cam_resolution (120, 160, 3) Setting default: log_level 20 Setting default: steer_limit 1.0 Setting default: throttle_min 0.0 Setting default: throttle_max 1.0