jiayuasu commented on PR #3108:
URL: https://github.com/apache/sedona/pull/3108#issuecomment-5077076492
Both confirmed against the JDK and fixed in 642de29.
[P1] `MultiPixelPackedSampleModel(TYPE_INT, 3, 2, 32, 4, 0)` is accepted by
Java and reads every pixel as zero, because `bitMask = (1 << 32) - 1` is
evaluated on an int whose shift count is taken modulo 32. The mask is now
derived as `(1 << (num_bits % 32)) - 1`, which reproduces that and leaves every
other width alone (8 on byte and 16 on ushort still mask normally).
[P2] Java leaves the bit offset of a zero mask at 0 and reads the band as
zero; `getBitOffsets()` returns `[16, 8, 0, 0]` for `{0xFF0000, 0xFF00, 0xFF,
0}`. The offset is now 0 for a zero mask. Worth noting the crash was numpy-2
only: on numpy 1.26 the `-1` shift happened to yield the right answer, so this
would not have shown up in CI.
Rather than patch just these two, I swept both packed models over 1004
layouts recorded from `java.awt.image`: every `num_bits` that divides the
sample size for byte, ushort and int samples, aligned and unaligned data bit
offsets, zero and non-zero bank offsets, and mask sets covering zero masks,
whole-sample masks and masks over the sign bit. All 1004 match on numpy 1.26
and 2.5, with numpy warnings raised as errors; 510 of them do not match on
master. No third case of this kind is left.
One thing the sweep turned up: Java rejects non-contiguous bit masks in the
`SinglePixelPackedSampleModel` constructor (`Mask 195 must be contiguous`), so
those layouts cannot reach the reader and need no handling.
Two new regression tests cover P1 and P2
(`test_multi_pixel_packed_whole_sample_pixels_read_zero`,
`test_single_pixel_packed_zero_mask_reads_zero`), taking their expected values
from the same JDK run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
