XiaoHongbo-Hope commented on PR #9831: URL: https://github.com/apache/paimon/pull/9831#issuecomment-5770691719
> The format work is careful, and the bounded-validation issue from my earlier review appears addressed. However, this PR still does not deliver the claimed random-read benefit to a Paimon user: `PaimonLeRobotDataset`/PyAV neither generates indexes nor consumes them here, and the sparse decode is implemented only inside `video_format_test.py`. The normal LeRobot loader writes descriptors with no index (`-1, 0`), so ordinary writes continue to produce unindexed videos. This adds a persisted Java/Python format and descriptor change without an end-to-end producer/consumer path. Please bring back the format together with index generation and a real table-reader path that fetches fewer bytes for a frame, with a test through that public path. Closing this PR for now on end-to-end value grounds. Thanks, I did the read related change in the previous commit of this PR, but removed later. I will add back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
