XiaoHongbo-Hope opened a new pull request, #9498: URL: https://github.com/apache/paimon/pull/9498
### Purpose Add a map-style `PaimonLeRobotDataset` for training from LeRobot v3 image datasets that were imported into a multimodal Paimon table. ### Changes - Accept matching `LeRobotDatasetMetadata` or a local v3 dataset/`meta` path and expose it as `dataset.meta`. - Reuse the lazy row-ID Torch reader from #9486, including batched `__getitems__` and multi-worker `DataLoader` reads, without materializing the full table. - Read image BLOBs in coalesced batches and return Torch tensors. - Preserve episode order and support episode subsets, delta windows, padding masks, and image transforms. - Validate the table against the metadata dtype, shape, BLOB, VECTOR, and task contract. - Keep exact top-level LeRobot field names containing dots on the lazy reader path; only true nested projection still falls back. ### Scope This first version reuses external LeRobot v3 metadata; the imported table does not persist the complete `meta/` directory and is not presented as a self-contained round-trip format. It supports image-backed datasets. Video-backed reads are an independent follow-up to #9494, and this PR does not depend on that PR. `PaimonLeRobotDataset` is a minimal PyTorch Dataset-compatible wrapper rather than a subclass of the official `LeRobotDataset`. ### Tests - `53 passed` in `pypaimon/tests/torch_read_test.py`, including `DataLoader(batch_size=2, num_workers=2)`. - `15 passed, 4 skipped` in `pypaimon/tests/multimodal_lerobot_test.py` locally; LeRobot-dependent cases run in the optional-dependency CI lane. - Flake8, `py_compile`, and `git diff --check` pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
