jackylee-ch opened a new pull request, #785: URL: https://github.com/apache/paimon-rust/pull/785
A deletion vector written by Java in the 64-bit format (`Bitmap64DeletionVector`) cannot be read: `read_from_bytes` accepts only the v1 magic, so every scan touching such a file fails with `Invalid magic number`, naming the v1 magic alone. Java's `DeletionVector.read` dispatches on both. Two things differ beyond the value. The v2 magic is written **little-endian**, and `DeletionFile.length()` counts the length prefix and the CRC for v2 but neither for v1. The second also breaks the read range: the factory asks for `length + 8`, past the end of a v2 entry, so a vector ending its index file is rejected by the storage layer before parsing. **Fix**: detect either magic, apply each format's length convention, and decode the payload as portable 64-bit roaring. Java run-length encodes before writing and emits every bucket key densely, so tests cover both shapes. Read side only; the writer still emits v1. Positions above `u32::MAX` are rejected rather than truncated — every API here is already roaring32-bound, and reaching one needs a data file with over 4.29e9 rows. A negative `bitmapLength` read from the file now errors instead of wrapping the size guard into passing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
