YannByron opened a new pull request, #9466: URL: https://github.com/apache/paimon/pull/9466
## Summary Add a paired ACT benchmark that compares the original RoboMIND AgileX HDF5 path with the Paimon path under one deterministic training contract. Both backends share the same configuration, train-only normalization, random seed, window sequence, ACT policy, optimizer, and correctness gates. ## Changes - Add a snapshot-pinned, lazy `ContiguousWindowDataset` for map-style PyTorch access to complete row windows. - Coalesce overlapping rows during batched access and support anchor-only columns so observation BLOBs are loaded once per action window. - Add HDF5 and Paimon ACT adapters plus a shared deterministic CPU harness with tensor and loss parity checks. - Alternate backend execution for at least three rounds and report first-batch latency, DataLoader throughput, fixed-step time, and separately measured Python allocations. - Reuse the RoboMIND pipeline's versioned canonical-action statistics and reject source, snapshot, normalization, or window-sequence mismatches. - Add `act` and `hdf5` optional dependencies, CLI guidance, and PyTorch/window API documentation. - Fall back to `UNKNOWN` source provenance when the installed package is not inside a Git checkout. ## Testing - [x] `python -m pytest pypaimon/tests/paired_act_benchmark_test.py pypaimon/tests/contiguous_window_dataset_test.py pypaimon/tests/robomind_agilex_pipeline_test.py pypaimon/tests/ray_hdf5_test.py -q` (38 passed, 1 skipped) - [x] Non-E2E Python suite (4523 passed, 134 skipped; one unrelated Ray shuffle ordering assertion failed once and passed on isolated rerun) - [x] Isolated Ray shuffle rerun (1 passed) - [x] Ruff on all changed Python files - [x] Python license checks (707 files valid) - [x] `compileall` and `git diff --check` - [x] Wheel build and content check for the benchmark and window-dataset modules - [ ] Full ACT benchmark on a downloaded RoboMIND dataset (requires dataset and LeRobot model assets) ## Notes This builds directly on the RoboMIND AgileX HDF5 pipeline merged in #9445. Ingestion and canonical-action backfill are intentionally outside the measured benchmark scope. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
