XiaoHongbo-Hope commented on code in PR #9580:
URL: https://github.com/apache/paimon/pull/9580#discussion_r3924616350
##########
docs/docs/pypaimon/multimodal-api.mdx:
##########
@@ -866,6 +866,64 @@ Notes:
few large reads); scattered point reads coalesce less.
- Blob reads are available only on `scan()`, not on the `search()` queries.
+### Contiguous windows for PyTorch
+
+Install the `torch` extra, then use `to_contiguous_window_dataset` to expose
+map-style windows without loading the selected rows or BLOB payloads into
Python
+memory up front. The Dataset builds a compact index from the group column,
order
+column, and Paimon row IDs. Each `__getitem__` call fetches only that window
from
+the snapshot recorded in `dataset.snapshot_id`.
+
+```shell
+pip install pypaimon[torch]
+```
+
+```python
+import torch
+
+
+def float32_window(values):
+ return torch.tensor(values, dtype=torch.float32)
+
+
+windows = (
+ frames.scan()
+ .where("split = 'train'")
+ .to_contiguous_window_dataset(
+ window_size=16,
+ columns=["state", "action"],
+ group_key="episode_id",
Review Comment:
These defaults do not match the native LeRobot schema preserved by #9529,
which uses episode_index and frame_index. Should these arguments be required
for a generic Dataset, or default to the LeRobot field names?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
