YannByron opened a new pull request, #10811: URL: https://github.com/apache/paimon/pull/10811
Contiguous window training examples currently convert eight-bit images to float32 in DataLoader workers. This documents the existing `images_to_tensor(..., return_uint8=True)` path and GPU-side normalization so image tensors use one byte per pixel through worker collation and host-to-device transfer. Tests cover multi-frame value equivalence and mixed bit depth, where the uint8 path must not silently normalize high-bit-depth frames. The change is deliberately limited to the documented training configuration; it does not introduce a new sampler or decoder backend. Real-image A10 measurements show the configuration improves Paimon’s own small CNN training proxy, but the same optimization applies to HDF5, so it is not evidence of a Paimon format advantage or the 30% end-to-end target. Validation: the affected PyPaimon image/window tests passed before the final documentation wording adjustment; `git diff --check` passes after rebasing onto current master. The final wording change only moves transform configuration into Dataset construction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
