JingsongLi opened a new pull request, #10957:
URL: https://github.com/apache/paimon/pull/10957

   ### Purpose
   
   `take_row_ids([2, 0, 2])` is a membership filter and can return only rows 
`[0, 2]`. Training samplers need a separate contract that preserves request 
order and repeated samples.
   
   Add `MultimodalTable.take_batch(row_ids, columns=..., snapshot_id=..., 
tag_name=...)`, returning an Arrow table aligned one-for-one with the supplied 
row IDs. Missing or deleted IDs raise `KeyError`; empty input preserves the 
projected schema. Keep `take_row_ids` as a filtering query and report the 
row-tracking requirement explicitly.
   
   Document the distinction between Paimon row IDs and Torch Dataset positions. 
Add regression coverage for ordered repeated row IDs, missing/deleted rows, 
snapshot/tag reads, and DataLoader sampling with replacement across lazy and 
materialized reads.
   
   ### Tests
   
   - `python -m pytest pypaimon/tests/multimodal_table_test.py 
pypaimon/tests/torch_read_test.py -q` — passed (209 tests); final row-tracking 
and sampler assertions were also rerun with focused tests.
   - `python -m flake8 --config=dev/cfg.ini pypaimon/multimodal/table.py 
pypaimon/read/datasource/torch_dataset.py 
pypaimon/tests/multimodal_table_test.py pypaimon/tests/torch_read_test.py` — 
passed.
   - `git diff --check` — passed.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to