#30489: Django RasterField deserialization bug with pixeltype flags
-------------------------------+--------------------------------------
Reporter: Ivor Bosloper | Owner: nobody
Type: Bug | Status: new
Component: GIS | Version: 2.2
Severity: Normal | Resolution:
Keywords: RasterField | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Description changed by Ivor Bosloper:
Old description:
> After inserting some raster data with raster2pgsql into a Django model
> table with a RasterField column, I get a `list index out of range` when
> querying the table with a Django Queryset.
>
> {{{
> ...
> File "django/contrib/gis/db/models/fields.py" in from_db_value
> 360. return connection.ops.parse_raster(value)
> File "django/contrib/gis/db/backends/postgis/operations.py" in
> parse_raster
> 369. return from_pgraster(value)
> File "django/contrib/gis/db/backends/postgis/pgraster.py" in
> from_pgraster
> 57. pixeltype = POSTGIS_TO_GDAL[pixeltype]
> }}}
>
> It turns out the `pixeltype` value used is 39 while the `POSTGIS_TO_GDAL`
> list is only 16 elements long. The database field contains valid data but
> can not be deserialized with Django.
>
> **Steps for reproduction:**
>
> {{{
> # Django model
> class RasterModel(models.Model):
> rast = models.RasterField(srid=4326)
>
> # raw sql, single pixel raster with nodata bit set
> insert into app_rastermodel values(1, REPLACE('01 0000 0100
> 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000 0000000000000000 E6100000 0100 0100 6 2 03 03', ' ',
> '')::raster);
>
> # query generating Exception
> RasterModel.objects.get(pk=1)
> }}}
>
> **Analysis**: if we look at the
> [https://trac.osgeo.org/postgis/wiki/WKTRaster/RFC/RFC1_V0SerialFormat
> Raster specification], the pixeltype is a byte of which the 4 highest
> bits are flags and the lowest 4 bits are the real pixeltype, but the
> Django deserialization code only considers one bit-flag:
>
> {{{
> # django/contrib/gis/db/backends/postgis/pgraster.py
> def from_pgraster(data):
> ...
> # Subtract nodata byte from band nodata value if it exists
> has_nodata = pixeltype >= 64
> if has_nodata:
> pixeltype -= 64
> ...
> }}}
>
> I have created (my first django) patch and hope somebody can assist me in
> getting it correct and merged.
New description:
After inserting some raster data with raster2pgsql into a Django model
table with a RasterField column, I get a `list index out of range` when
querying the table with a Django Queryset.
{{{
...
File "django/contrib/gis/db/models/fields.py" in from_db_value
360. return connection.ops.parse_raster(value)
File "django/contrib/gis/db/backends/postgis/operations.py" in
parse_raster
369. return from_pgraster(value)
File "django/contrib/gis/db/backends/postgis/pgraster.py" in from_pgraster
57. pixeltype = POSTGIS_TO_GDAL[pixeltype]
}}}
It turns out the `pixeltype` value used is 39 while the `POSTGIS_TO_GDAL`
list is only 16 elements long. The database field contains valid data but
can not be deserialized with Django.
**Steps for reproduction:**
{{{
# Django model
class RasterModel(models.Model):
rast = models.RasterField(srid=4326)
# raw sql, single pixel raster with nodata bit set
insert into app_rastermodel values(1, REPLACE('01 0000 0100
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 E6100000 0100 0100 6 2 03 03', ' ',
'')::raster);
# query generating Exception
RasterModel.objects.get(pk=1)
}}}
**Analysis**: if we look at the
[https://trac.osgeo.org/postgis/wiki/WKTRaster/RFC/RFC1_V0SerialFormat
Raster specification], the pixeltype is a byte of which the 4 highest bits
are flags and the lowest 4 bits are the real pixeltype, but the Django
deserialization code only considers one bit-flag:
{{{
# django/contrib/gis/db/backends/postgis/pgraster.py
def from_pgraster(data):
...
# Subtract nodata byte from band nodata value if it exists
has_nodata = pixeltype >= 64
if has_nodata:
pixeltype -= 64
...
}}}
The erroneous pixeltype 39 in my example actually had the
`BANDTYPE_FLAG_ISNODATA` (32) bit set which indicates all rastervalues are
nodata.
I have created (my first django) patch and hope somebody can assist me in
getting it correct and merged.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30489#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/070.31947f125575455d264fc49ae875a0ba%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
