jiayuasu opened a new issue, #3166:
URL: https://github.com/apache/sedona/issues/3166
## Expected behavior
`SedonaRaster.as_numpy()` should return the samples that `java.awt.image`
reads from the same sample model and data buffer. The Python sample models in
`python/sedona/spark/raster/sample_model.py` are ports of the Java AWT ones, so
a sample of band `b` at `(x, y)` should resolve exactly the way
`Raster.getSample(x, y, b)` does:
```java
// ComponentSampleModel
data.getElem(bankIndices[b], y * scanlineStride + x * pixelStride +
bandOffsets[b])
// DataBufferInt and friends
public int getElem(int bank, int i) { return bankdata[bank][i +
offsets[bank]]; }
```
## Actual behavior
The Python implementation disagrees with Java AWT in five ways. Rasters
affected either come back with silently wrong pixel values or fail to be read
at all.
**1. `DataBuffer.offsets` is ignored by all four sample models.** Nothing in
`sample_model.py` reads `data_buffer.offsets`; the models index
`data_buffer.bank_data[i]` directly. Java resolves a sample as
`bankdata[bank][i + offsets[bank]]`. Every sample of a bank with a non-zero
offset is therefore read from the wrong position. `DataBufferSerializer` writes
these offsets and `raster_serde._read_data_buffer` deserializes them, so they
do arrive on the Python side and are then dropped.
**2. The fast paths reshape a whole bank instead of a `width * height`
window.** `ComponentSampleModel.as_numpy` narrows the bank only when the band
offset is non-zero, and `PixelInterleavedSampleModel.as_numpy` never narrows
it. A bank that holds more samples than the image needs — which is exactly what
a non-zero buffer offset implies, but also happens with a zero band offset over
a padded bank — raises:
```
ValueError: cannot reshape array of size 6 into shape (2,2)
```
**3. `ComponentSampleModel`'s fast path indexes band offsets by bank
number.** It uses `self.band_offsets[bank_index]` where the band position `k`
is meant, so a raster whose `bank_indices` is not the identity mapping reads
bands at each other's offsets. The slow path already uses `band_offsets[k]`.
**4. `PixelInterleavedSampleModel`'s slow path assumes band offsets fall
inside one pixel stride.** It slices a single pixel out of the bank and then
indexes it with the band offsets:
```python
pixel = bank_data[begin:end][self.band_offsets] # end = begin + num_bands
```
Band offsets are positions within a scanline, not within one pixel, so a
layout such as `pixelStride = 4` with `bandOffsets = {3, 1, 0}` raises:
```
IndexError: index 3 is out of bounds for axis 0 with size 3
```
**5. `SinglePixelPackedSampleModel` masks and shifts signed samples.** Java
extracts a band with `(bits & bitMasks[b]) >>> bitOffsets[b]`, an unsigned
shift. The Python code applies `>>` to the signed sample, so a mask that covers
the sign bit — the alpha mask `0xFF000000` of a standard ARGB raster — yields
sign-extended negative values instead of `0..255`.
**6. `MultiPixelPackedSampleModel` reads one sample past the end of every
scanline.** The `pos += 1; value = bank_data[pos]` at the bottom of its loop
runs even when the scanline is already complete, so a bank that ends where the
last scanline ends raises:
```
IndexError: index 1 is out of bounds for axis 0 with size 1
```
Its running bit shift also diverges from Java when `data_bit_offset` is not
a multiple of `num_bits`, the case where a pixel spans two data elements. For
every `data_bit_offset` that is a multiple of `num_bits`, the loop agrees with
Java (checked over 1323 parameter combinations).
None of these are covered today: every layout `RS_MakeRasterForTesting`
builds uses zero buffer offsets, and its band offsets are either an identity
mapping or within the pixel stride, so `python/tests/raster/test_serde.py`
passes throughout.
## Steps to reproduce the problem
The layouts are easier to build directly than to produce through the
serializer, and the sample models are the unit under test either way:
```python
import numpy as np
from sedona.spark.raster.data_buffer import DataBuffer
from sedona.spark.raster.sample_model import (
ComponentSampleModel,
MultiPixelPackedSampleModel,
PixelInterleavedSampleModel,
SinglePixelPackedSampleModel,
)
# 1 + 2: BandedSampleModel(TYPE_INT, 2, 2, 2, {0, 1}, {0, 0})
# over DataBufferInt(banks of 6 samples, 4, {2, 2})
sm = ComponentSampleModel(DataBuffer.TYPE_INT, 2, 2, 1, 2, [0, 1], [0, 0])
db = DataBuffer(
DataBuffer.TYPE_INT,
[np.arange(10, 16, dtype=np.int32), np.arange(20, 26, dtype=np.int32)],
4,
[2, 2],
)
sm.as_numpy(db)
# ValueError: cannot reshape array of size 6 into shape (2,2)
# java.awt.image reads [[[12, 13], [14, 15]], [[22, 23], [24, 25]]]
# 4: PixelInterleavedSampleModel(TYPE_INT, 2, 2, 4, 8, {3, 1, 0})
sm = PixelInterleavedSampleModel(DataBuffer.TYPE_INT, 2, 2, 4, 8, [3, 1, 0])
db = DataBuffer(DataBuffer.TYPE_INT, [np.arange(10, 26, dtype=np.int32)],
16, [0])
sm.as_numpy(db)
# IndexError: index 3 is out of bounds for axis 0 with size 3
# java.awt.image reads
# [[[13, 17], [21, 25]], [[11, 15], [19, 23]], [[10, 14], [18, 22]]]
# 5: SinglePixelPackedSampleModel(TYPE_INT, 2, 1, 2, {0xFF000000, 0xFF0000,
0xFF00, 0xFF})
sm = SinglePixelPackedSampleModel(
DataBuffer.TYPE_INT, 2, 1, 2, [0xFF000000 - (1 << 32), 0xFF0000, 0xFF00,
0xFF]
)
db = DataBuffer(
DataBuffer.TYPE_INT,
[np.array([0x8090A0B0, 0xFFFFFFFF], dtype=np.uint32).astype(np.int32)],
2,
[0],
)
sm.as_numpy(db).tolist()
# [[[-128, -1]], [[144, 255]], [[160, 255]], [[176, 255]]] <- alpha sign
extended
# java.awt.image reads [[[128, 255]], [[144, 255]], [[160, 255]], [[176,
255]]]
# 6: MultiPixelPackedSampleModel(TYPE_BYTE, 8, 1, 1, 1, 0) over a bank of
one byte
sm = MultiPixelPackedSampleModel(DataBuffer.TYPE_BYTE, 8, 1, 1, 1, 0)
db = DataBuffer(DataBuffer.TYPE_BYTE, [np.array([0xA9], dtype=np.uint8)], 1,
[0])
sm.as_numpy(db)
# IndexError: index 1 is out of bounds for axis 0 with size 1
# java.awt.image reads [[[1, 0, 1, 0, 1, 0, 0, 1]]]
```
Every "java.awt.image reads" line above was produced by handing the same
sample model and data buffer to `java.awt.image` and reading the samples back
with `Raster.getSample`.
## Settings
Sedona version = 1.9.0 and master
Apache Spark version = 3.5.x (not specific to a Spark version)
Apache Flink version = n/a
API type = Python
Scala version = 2.12
JRE version = 1.17
Python version = 3.12
Environment = Standalone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
