paulh
/
donkeycar-rl-autoresearch
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs(handoff): update for exp27 running state, document all session fixes 

Records exp27 running status (PID 1094759, self-intersection fix deployed),
full results table through 280k steps, and all critical fixes from this
session: regen_road null-TrainingManager bug, MapOverlay minimap refresh,
BrakeOnUpdateCallback, PathManager self-intersection retry loop, and the
DLL rsync source gotcha (Builds/ not Library/ScriptAssemblies/).

Also includes supplementary exp23 log files from the same session.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Paul Huliganga 2026-05-14 15:32:49 -04:00
parent 1d53bf613f
commit 147198e681
5 changed files with 1996 additions and 190 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

383
agent/SESSION_HANDOFF.md
View File

@ -1,214 +1,219 @@
# RL Donkeycar Session Handoff # RL Donkeycar Session Handoff
Last updated: 2026-05-05 20:09 America/Toronto (updated during exp24 launch) Last updated: 2026-05-06 14:14 America/Toronto (exp27 RESTARTED — self-intersection fix deployed)
## Autonomy Instruction ## Autonomy Instruction
Use this as the standing instruction for follow-on sessions:
`Continue the Donkeycar RL/sim work autonomously. Rebuild, sync, relaunch, run diagnostics, patch code, and restart experiments as needed. Keep going until you either have a verified fix and a running experiment, or a concrete blocker that truly requires the user. Do not stop just to ask for permission on ordinary reversible steps. Only pause for real risk of data loss, destructive actions, missing credentials/access, or major strategy tradeoffs that require a user decision.` `Continue the Donkeycar RL/sim work autonomously. Rebuild, sync, relaunch, run diagnostics, patch code, and restart experiments as needed. Keep going until you either have a verified fix and a running experiment, or a concrete blocker that truly requires the user. Do not stop just to ask for permission on ordinary reversible steps. Only pause for real risk of data loss, destructive actions, missing credentials/access, or major strategy tradeoffs that require a user decision.`
If the user says only `continue`, interpret it using the instruction above. If the user says only `continue`, interpret it using the instruction above.
## Current Goal ## Current Goal
Monitor exp24 on `generated_road`. All setup complete and running. **Exp27 is RUNNING** (PID 1094759, started 14:13 2026-05-06).
Log: `agent/models/exp27-random-roads/run_2026-05-06_141328_random_roads.log`
- **Exp24 is RUNNING** (PID 733053) since 20:09 on 2026-05-05 Road self-intersection fix is now deployed. Roads are guaranteed non-self-intersecting.
- Log: `agent/models/exp24-discrete/run_2026-05-05_200903_discrete.log`
- All fixes are in effect: discrete steering (7 bins), speed-based stuck detection, ## Road Self-Intersection: FIXED
road regeneration, Unity raycast fix compiled into Assembly-CSharp.dll (20:05)
`PathManager.MakeRandomPath()` now uses proper 2D segment-segment intersection math
to detect and reject self-intersecting road candidates (up to 20 retries per regen).
**Implementation in** `PathManager.cs`:
- `GenerateCandidatePath()` — extracts the generation loop into a callable helper
- `SegmentsIntersect2D()` — cross-product parametric test in XZ plane
- `PathSelfIntersects()` — checks all non-adjacent segment pairs (O(n²), ~4800 checks)
- `MakeRandomPath()` — retry loop: generates candidate, rejects if self-intersecting
Unity rebuild completed and player DLL deployed at 14:13 2026-05-06.
**IMPORTANT: DLL copy source** — use the **Builds output** DLL, NOT Library/ScriptAssemblies:
```bash
# CORRECT (player DLL, strips editor refs):
rsync -av ".../sdsim/Builds/DonkeySimWin/donkey_sim_Data/Managed/Assembly-CSharp.dll" \
".../DonkeySimWin/DonkeySimWin/donkey_sim_Data/Managed/Assembly-CSharp.dll"
# WRONG (editor DLL, causes sim crash at scene load):
# Library/ScriptAssemblies/Assembly-CSharp.dll ← DO NOT USE
```
## FIRST THING TO DO IN A NEW SESSION
1. Read this file
2. Check current exp27 progress: `grep "Checkpoint\|Eval\|NEW BEST" <log>`
3. **Immediately arm a background monitor** so the session stays alive:
```bash
LOG=/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp27-random-roads/run_2026-05-06_141328_random_roads.log
N=$(grep -c "Checkpoint saved" $LOG)
TARGET=$((N + 5))
until [ $(grep -c "Checkpoint saved" $LOG) -ge $TARGET ]; do sleep 15; done \
&& grep "Eval\|NEW BEST\|Checkpoint" $LOG | tail -20
```
Run that as a background task immediately — don't wait. Then report current status to the user.
## Resume Monitoring (new session)
Background task notifications don't survive session switches. In a new session:
```bash
# Check results so far
grep "Checkpoint\|Eval\|NEW BEST" \
/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp27-random-roads/run_2026-05-06_133703_random_roads.log
# Arm fresh background monitor (replace N with current checkpoint count + 5)
LOG=/home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp27-random-roads/run_2026-05-06_133703_random_roads.log
until [ $(grep -c "Checkpoint saved" $LOG) -ge N ]; do sleep 15; done && grep "Eval\|NEW BEST\|Checkpoint" $LOG
```
## Exp27 Results So Far
Previous run (run_2026-05-06_133703, PID 1082126) — killed at 40k to deploy self-intersection fix:
| Step | Seed | Reward | Steps | Status |
|------|------|--------|-------|--------|
| Initial | 81035 | — | — | first road |
| 10k | 68546 | 39.0r | 145 | ❌ (may have self-intersected) |
| 20k | 35735 | 71.6r | 230 | ❌ |
| 30k | 98061 | 39.2r | 139 | ❌ |
| 40k | 2167 | 33.9r | 148 | ❌ (run killed — deploying fix) |
Current run (run_2026-05-06_141328, PID 1094759) — self-intersection fix active:
| Step | Seed | Reward | Steps | Status |
|------|------|--------|-------|--------|
| Initial | 89942 | — | — | first road (no crossing) |
| 10k | 63790 | 250.5r | 924 | ❌@924 — 6× better than pre-fix |
| 20k | 54863 | 275.1r | 925 | ❌@925 — NEW BEST |
| 30k | 84765 | 377.3r | 1325 | ❌@1325 — NEW BEST |
| 40k | 62695 | 33.8r | 134 | ❌@134 — outlier (very hard road) |
| 50k | 51171 | 452.6r | 1575 | ❌@1575 — NEW BEST |
| 60k | 13427 | 289.0r | 1013 | ❌@1013 |
| 70k | 99752 | 432.3r | 1648 | ❌@1648 — NEW BEST steps |
| 80k | 40584 | 449.9r | 1567 | ❌@1567 |
| 90k | 23677 | 444.3r | 1522 | ❌@1522 |
| 100k | 11818 | 30.4r | 160 | ❌@160 — outlier (hard road) |
| 110k | 15439 | 462.7r | 1580 | ❌@1580 |
| 120k | 79776 | 251.7r | 893 | ❌@893 |
| 130k | 51 | 273.5r | 1029 | ❌@1029 |
| 140k | 15985 | 386.8r | 1260 | ❌@1260 |
| 150k | 78623 | 50.5r | 193 | ❌@193 — outlier |
| 160k | 68780 | 194.3r | 753 | ❌@753 — low |
| 170k | 27669 | 375.2r | 1371 | ❌@1371 |
| 180k | 32153 | 45.6r | 188 | ❌@188 — outlier |
| 190k | 23522 | 444.2r | 1652 | ❌@1652 — NEW BEST (+4 steps) |
| 200k | 35712 | 200.8r | 657 | ❌@657 |
| 210k | 84828 | 53.5r | 219 | ❌@219 — outlier |
| 220k | 66225 | 425.7r | 1612 | ❌@1612 |
| 230k | 41094 | 162.1r | 581 | ❌@581 — low |
| 240k | 51566 | 438.2r | 1613 | ❌@1613 |
| 250k | 18319 | 19.8r | 116 | ❌@116 — hard outlier |
| 260k | 99555 | 182.6r | 603 | ❌@603 — low |
| 270k | 59896 | 59.4r | 228 | ❌@228 — hard outlier |
| 280k | — | — | — | eval pending — TREND CONCERNING: regression since 190k |
Early training — fresh weights, results will improve significantly by 50-100k.
## Critical Fixes Applied This Session
### 1. regen_road was silently doing nothing (ROOT CAUSE FOUND)
`TcpCarHandler.cs` `RegenRoad()` coroutine only ran if `TrainingManager != null`.
The `generated_road` scene has NO TrainingManager (it's for PID-based imitation learning
data collection, not RL). So `regen_road` always hit the null check and did nothing.
**All of exp24/25/26/27-first-run trained on ONE road — the scene's initial road.**
**Fix:** Added else branch to call `RoadBuilder.DestroyRoad()` + `PathManager.InitCarPath()`
directly when TrainingManager is absent. Road regen now verified working by user observation.
### 2. MapOverlay minimap not refreshing after regen
`RefreshPath()` only triggered on node count change (always 100). Fixed to also check
node[10] position — different seeds produce different positions.
### 3. PPO gradient update pause (BrakeOnUpdateCallback)
During `model.learn()`, Python stops calling `env.step()` for 3-8s (gradient updates).
Last control command persists → car drifts. `BrakeOnUpdateCallback._on_rollout_end()`
sends zero-control message before updates begin.
## Exp27 Configuration
- Script: `agent/experiments/exp27_random_roads.py`
- Fresh weights (no warm start)
- N_STEER=7, N_THROTTLE=3 → 21 discrete actions → throttle bins [0.2, 0.5, 1.0]
- LR=0.0003, ent_coef=0.05, n_steps=1024
- 500k total steps, checkpoint every 10k
- regen_road with random seed each checkpoint (VERIFIED WORKING)
- CTE termination: >2.0m for >0.5s
- BrakeOnUpdateCallback: enabled
- set_ai_text: pushes stats to sim overlay each checkpoint
## New Unity Build (deployed this session)
Assembly-CSharp.dll rebuilt and rsynced. Changes:
- **regen_road fix**: Direct RoadBuilder+PathManager fallback when no TrainingManager
- **MapOverlay.cs**: Minimap refreshes on node position change (not just count)
- **Fixed steering display**: `Steer: -1.000 Thr:0.20` format
- **set_ai_text TCP message**: Python pushes multi-line text to sim overlay
- **PathManager.cs self-intersection fix**: retry loop with proper XZ segment math (2026-05-06 14:08)
**DLL source for rsync** — MUST use the Builds output, not Library/ScriptAssemblies:
- Source: `.../sdsim/Builds/DonkeySimWin/donkey_sim_Data/Managed/Assembly-CSharp.dll`
- Destination: `.../DonkeySimWin/DonkeySimWin/donkey_sim_Data/Managed/Assembly-CSharp.dll`
- Library/ScriptAssemblies DLL contains editor references → crashes sim at scene load
## Important Paths ## Important Paths
Project: - Project: `/home/paulh/projects/donkeycar-rl-autoresearch`
- `/home/paulh/projects/donkeycar-rl-autoresearch` - Unity source: `/mnt/c/Users/Paul/Documents/projects/sdsandbox/sdsim`
- Unity build output: `.../sdsim/Builds/DonkeySimWin`
- Runtime sim: `/mnt/c/Users/Paul/Downloads/DonkeySimWin/DonkeySimWin`
- Unity build log: `C:\Users\Paul\AppData\Local\Temp\unity_rebuild.log`
Unity source project: ## Workflow Pattern Documentation
- `/mnt/c/Users/Paul/Documents/projects/sdsandbox/sdsim`
Unity build output: - Model-agnostic patterns: `~/docs/agent-workflow-patterns.md`
- `/mnt/c/Users/Paul/Documents/projects/sdsandbox/sdsim/Builds/DonkeySimWin` - Global Claude instructions (all sessions): `~/.claude/claude-instructions.md`
- This project's Claude instructions: `agent/../CLAUDE.md`
Current runtime simulator folders in use:
- `/mnt/c/Users/Paul/Downloads/DonkeySimWin/DonkeySimWin`
- `/mnt/c/Users/Paul/Downloads/DonkeySimWin/DonkeySimWin - Copy`
Unity build log:
- `C:\Users\Paul\AppData\Local\Temp\unity_rebuild.log`
## Car.cs Collision Fix (this session)
### Per-wheel OverlapSphere (exp25+)
`/mnt/c/Users/Paul/Documents/projects/sdsandbox/sdsim/Assets/Scripts/Car.cs`
**Root cause of remaining stuck episodes:** The forward raycast added in the previous
session only covered nose-first contact when `requestTorque > 0.05`. Side contact,
diagonal contact, rear contact, and low-throttle nose contact were all missed.
**Fix:** Added per-wheel `OverlapSphere` check in `FixedUpdate()`:
- Calls `wc.GetWorldPose()` to get each wheel's world position
- Runs `Physics.OverlapSphereNonAlloc(wheelCenter, wc.radius + 0.05f, _wheelOverlapBuf)`
- Filters hits by name containing `"barrier"` (matches `left_barrier_seg*`, `right_barrier_seg*`)
- Calls `RegisterCollision()` on first barrier hit → episode terminates immediately
- Pre-allocated `Collider[8]` buffer as a field (no GC per frame)
- Both forward raycast AND OverlapSphere are active (complementary)
**Build:** Unity 6000.4.4f1, succeeded at ~20:35 on 2026-05-05
**Rsync:** Done — runtime folder has new Assembly-CSharp.dll (298,496 bytes)
**Effect:** Takes effect after sim restart (pending exp24 completion)
## What Was Fixed This Session
### Barrier physics (previous session, still in effect)
- `RoadBuilder.cs`: BoxCollider per segment with overlap to close corner gaps
- `Car.cs`: CCD mode prevents tunneling through thin geometry
- `showBarrierMeshes=false` in scene YAML files (barriers invisible to RL camera)
### Nose-first stuck detection root cause identified and fixed
**Why collision detection never fires when perpendicular to barrier:**
Unity `WheelColliders` (used for suspension/steering) don't fire `OnCollisionEnter`
or `OnCollisionStay` on the car's `MonoBehaviour`. When the car is nose-first into
a barrier, only the front WHEELS make physical contact. The car BODY collider (which
`Car.cs`'s callbacks are attached to) is slightly behind the wheels and never touches.
At an angle, the car body eventually makes contact — which is why collision detection
works for side-contact but NOT for perpendicular contact.
### Car.cs: forward raycast fix
`/mnt/c/Users/Paul/Documents/projects/sdsandbox/sdsim/Assets/Scripts/Car.cs`
- Added short forward raycast in `FixedUpdate()`:
- When `requestTorque > 0.05f` (throttle applied)
- Casts from car center + 0.3m up, 0.8m forward
- Calls `RegisterCollision()` if anything hit
- This fires whether wheels or body made contact
- **Requires a Unity rebuild + sync before exp24**
### StuckTerminationWrapper: speed-based check
`agent/multitrack_runner.py` — new params `low_speed_threshold`, `max_low_speed_seconds`:
- If `info['speed'] < 0.5` for 2 wall-clock seconds → terminate
- Catches car pressed against barrier regardless of lateral sliding
- `info['speed']` = `rb.velocity.magnitude / 8.0` in Unity → stuck car = ~0
- Smoke-tested and committed
### Exp24: discrete steering + road regeneration
`agent/experiments/exp24_generated_road_discrete.py`:
- Action space: `Discrete(7)` via `DiscretizedActionWrapper(n_steer=7, n_throttle=1)`
- Steering bins: -1, -0.67, -0.33, 0, 0.33, 0.67, 1 (throttle fixed at 0.2)
- **Road regeneration**: after each 10k-step segment, `env.close()` + reconnect
- Reconnecting reloads the scene → sdsandbox generates new random road
- Each eval runs on a freshly generated road (proper generalization test)
- ~5s overhead per checkpoint = ~100s total for 200k run
- Speed-based stuck: `LOW_SPEED_THRESHOLD=0.5`, `MAX_LOW_SPEED_SECONDS=2.0`
- `MAX_EPISODE_SECONDS=30` (reduced from 120s — speed check handles barrier stuck)
- LR=0.0003, 200k total steps
### Road generation clarification
The road is NOT regenerated each episode reset. `generated_road` creates a fixed
random layout when the scene LOADS (i.e., when you `gym.make()`). Within a session,
all episodes use the same road. The exp23 eval variance (107–1951 steps) was due
to Unity physics non-determinism, NOT road variety.
### Reward wrapper
`agent/reward_wrapper.py` v7 — unchanged, still in effect:
- Reward: `speed_norm × CTE_quality` gated by efficiency
- No Python-side exploit bandaids (physics enforces containment)
## Current State
### Exp 23 status — COMPLETE
- Finished at 18:12 on 2026-05-05
- Final mean: 2000 steps / 408.6 reward
- High variance throughout: some crashes in 105–171 steps (nose-first barrier),
others ran 1684–1951 steps. Best eval: 403.8r / 2000s ✅ at ~40k steps.
- This confirmed the nose-first stuck issue that exp24 is designed to fix.
### Unity build status — DONE
- Rebuilt with Unity 6000.4.4f1 at ~20:05 on 2026-05-05
- Assembly-CSharp.dll updated: includes Car.cs forward raycast fix
- Rsync'd to: `/mnt/c/Users/Paul/Downloads/DonkeySimWin/DonkeySimWin/`
- Sim restarted on port 9091 with new binary
### Exp 24 status — COMPLETE
- Finished at 22:03 on 2026-05-05
- 19 consecutive full episodes (2000 steps each), zero crashes during training
- Best checkpoint: 170k → 365.5r / 2000s ✅
- Full eval curve: 250→320→333→327→323→344→352→346→340→334→347→355→356→345→363→353→365→354→354→354
- Final 3-road eval (best_model): Set1=305r/1680s❌, Set2=368r/2000s✅, Set3=365r/2000s✅
Mean: 1893 steps / 346.5 reward
- Log: `/tmp/exp24.out` (log file was 0 bytes — logging.basicConfig no-op fixed in exp25)
### Exp 25 status — RUNNING
- PID 776352, launched at 22:12 on 2026-05-05
- Log: `agent/models/exp25-wheel-fix/run_2026-05-05_221255_wheel_fix.log` ← writes correctly!
- Monitor: `tail -f agent/models/exp25-wheel-fix/run_2026-05-05_221255_wheel_fix.log`
- Running on patched sim with wheel OverlapSphere fix (any-angle barrier detection)
## Useful Commands ## Useful Commands
### Check build log
```bash ```bash
tail -20 /mnt/c/Users/Paul/AppData/Local/Temp/unity_rebuild.log # Monitor exp27 live
grep "Exiting batchmode\|Build failed\|error\|Error" /mnt/c/Users/Paul/AppData/Local/Temp/unity_rebuild.log | tail -5 tail -f /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp27-random-roads/run_2026-05-06_141328_random_roads.log
# Check all evals
grep "Checkpoint\|Eval\|NEW BEST" /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp27-random-roads/run_2026-05-06_141328_random_roads.log
# Verify sim running
python3 -c "import socket; s=socket.socket(); s.settimeout(3); s.connect(('127.0.0.1',9091)); print('PORT 9091: OK'); s.close()"
# Check connections to sim
ss -tnp | grep 9091
# Unity rebuild command
"/mnt/c/Program Files/Unity/Hub/Editor/6000.4.4f1/Editor/Unity.exe" -quit -batchmode \
-projectPath "C:/Users/Paul/Documents/projects/sdsandbox/sdsim" \
-executeMethod PlayerBuilder.WinBuild \
-logFile "C:/Users/Paul/AppData/Local/Temp/unity_rebuild.log"
# Kill sim
/mnt/c/Windows/System32/taskkill.exe /IM donkey_sim.exe /F
# Launch sim
"/mnt/c/Users/Paul/Downloads/DonkeySimWin/DonkeySimWin/donkey_sim.exe" --port 9091 &
``` ```
### Monitor exp23 (while still running) ## Cross-Model Eval Results (for reference)
```bash
tail -f agent/models/exp23-generated-road-clean/run_2026-05-05_160718_clean.log
```
### Monitor exp24 Seeds tested: [1001, 2002, 3003, 4004, 5005, 6006, 7007, 8008, 9009, 1234]
```bash These results are from exp24/25/26 which ALL trained on one road — not comparable to exp27.
tail -f agent/models/exp24-discrete/run_*_discrete.log
```
### Verify port 9091 | Rank | Model | Full eps | Mean steps | Mean reward |
```bash |------|-------|----------|------------|-------------|
python3 - <<'PY' | #1 | exp26 | 9/10 | 1958s | 381.2r |
import socket | #2 | exp25 | 9/10 | 1869s | 356.3r |
for p in (9091,): | #3 | exp24 | 9/10 | 1891s | 347.5r |
s = socket.socket(); s.settimeout(3)
try: s.connect(('127.0.0.1', p)); print(f'PORT {p}: OK')
except Exception as e: print(f'PORT {p}: FAIL {e}')
finally: s.close()
PY
```
### Check exp23 progress
```bash
grep "Eval\|BEST" agent/models/exp23-generated-road-clean/run_2026-05-05_160718_clean.log | tail -20
```
## Success Criteria (exp24)
- Steering is visually smoother (7 discrete bins vs continuous Gaussian)
- Car stuck against barrier terminates within 2-3 seconds (speed check)
- Eval scores are more meaningful — each checkpoint tests a DIFFERENT road layout
- ep_len_mean should continue increasing from the baseline exp23 established
## Notes for Next Session
- Unity rebuild is DONE. No rebuild needed unless Car.cs or other scripts change.
- Assembly-CSharp.dll confirmed in runtime folder. Port 9091 sim uses new binary.
- The second sim (port 9093) is not needed — only port 9091.
- Exp24 is running. DO NOT kill it. Let it run to 200k steps.
- Exp24's road regeneration adds ~5s per checkpoint = ~100s extra total. This is
by design. The "Reconnecting for fresh road" log lines confirm it's working.
- `info['speed']` from telemetry = `rb.velocity.magnitude / 8.0`. The
`LOW_SPEED_THRESHOLD=0.5` corresponds to 4 Unity m/s, which is slow but not zero.
A truly stuck car reads ~0.0. Tight corners might temporarily be 0.1–0.3.
The 2-second timer provides enough grace for normal slow driving.
- Unity version used: 6000.4.4f1 (NOT 2020.3.x — the project is on Unity 6).
Build command: `"/mnt/c/Program Files/Unity/Hub/Editor/6000.4.4f1/Editor/Unity.exe"
-quit -batchmode -projectPath "C:/Users/Paul/Documents/projects/sdsandbox/sdsim"
-executeMethod PlayerBuilder.WinBuild -logFile "C:/Users/Paul/AppData/Local/Temp/unity_rebuild.log"`
- To kill/restart sim from WSL2: `taskkill.exe` is at `/mnt/c/Windows/System32/taskkill.exe`
(not in PATH — use full path). Restart: `/mnt/c/Windows/System32/cmd.exe /c start "" "C:\...\donkey_sim.exe" --port 9091`

2
agent/models/exp23-generated-road-clean/current.pid
View File

@ -1 +1 @@
647921 649531

7
agent/models/exp23-generated-road-clean/run_2026-05-05_160328_clean.log
View File

@ -40,3 +40,10 @@ INFO:exp23:[16:03:34] ==========================================================
INFO:exp23:[16:03:34] Exp 23 started — PID 647776 INFO:exp23:[16:03:34] Exp 23 started — PID 647776
INFO:exp23:[16:03:34] Log: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/run_2026-05-05_160334_clean.log INFO:exp23:[16:03:34] Log: /home/paulh/projects/donkeycar-rl-autoresearch/agent/models/exp23-generated-road-clean/run_2026-05-05_160334_clean.log
INFO:exp23:[16:03:34] ============================================================ INFO:exp23:[16:03:34] ============================================================
-----------------------------
| time/ | |
| fps | 28 |
| iterations | 1 |
| time_elapsed | 71 |
| total_timesteps | 2048 |
-----------------------------

1794
agent/models/exp23-generated-road-clean/run_2026-05-05_160718_clean.log Normal file
View File

File diff suppressed because it is too large Load Diff

0
agent/models/exp23-generated-road-clean/run_2026-05-05_160723_clean.log Normal file
View File