jiayuasu commented on PR #3108:
URL: https://github.com/apache/sedona/pull/3108#issuecomment-5077300048
Added the warning in 3fce157. `UserWarning` via `warnings.warn`, matching
how this package already reports its geometry speedup fallback, one message per
quirk:
> This raster packs one pixel per 32 bit sample. java.awt.image derives the
bit mask for it as `(1 << 32) - 1`, which is zero on an int, so Java reads
every pixel of such a raster as zero and writes to it are no-ops. Returning
zeroes to match.
> This raster's data bit offset (4) is not a multiple of its 8 bits per
pixel, so some pixels straddle two samples. java.awt.image shifts those by a
negative distance, which it takes modulo 32, reading the top bits of the sample
instead; Java's own writes to those pixels are lossy in the same way. Returning
what Java reads.
The trigger is the negative shift distance itself, not the alignment of the
data bit offset, because whether a pixel straddles also depends on the width: 4
bit pixels at offset 2 in a 32 bit sample fit until x reaches 8. Checking the
distance keeps it exact in both directions.
Behaviour, all three verified rather than assumed:
- reading the packed layout `RS_MakeRasterForTesting` builds today emits
nothing, asserted in the test under `warnings.simplefilter("error")` so a
future false positive fails CI
- 500 reads of one odd raster produce 1 warning, since Python's default
filter dedups per message and location; distinct layouts warn separately
- values did not move: the 7580 case sweep against `java.awt.image` is still
exact
Worth knowing where this does and does not help: inside a pandas UDF the
warning lands on executor stderr, which is easy to miss. It is aimed at
driver-side reads and notebooks, so it is a signpost for whoever is debugging
an odd raster interactively rather than a guard rail on a cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
