XiaoHongbo-Hope opened a new pull request, #11095:
URL: https://github.com/apache/paimon/pull/11095

   Two independent Ray jobs appending to the same three-table LeRobot group can 
interleave commits. In a DLF Ray reproduction, one job failed with overlapping 
global-index row ranges and the group briefly had 2 Dataset rows but only the 
original 2 Episodes and 5 Frames; the reader rejected it.
   
   Document the supported pattern: one driver owns each table group's append 
calls, while Ray workers prepare a list of sources in parallel within a single 
call. Clarify that `recover_lerobot_append` addresses an interrupted sole 
writer, not concurrent interleaving. This PR does not add cross-job locking or 
claim that independent concurrent append calls are safe.
   
   Validation: on the Ray cluster, `append_to_lerobot(group, [source1, 
source2], parallelism=2)` produced Dataset indices 0–2, 6 Episodes, and 15 
Frames, all readable; test tables were removed. Local targeted regression: 2 
passed, 1 skipped; configured flake8 and `git diff --check` passed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to