[gdal-dev] Lost in decimals... geotransform in python 2.7
Hi, I am trying to get the four corners of a raster the size of Canada (very big therefore decimals count). If I get the geotransform, I get: geotransform: (-132.000417, 0.0008334, 0.0, 53.0004166, 0.0, -0.0008334) In Python 2.7: print (geotransform[0]) -> -132.000416667 print float(geotransform[0]) -> -132.000416667 print str(geotransform[0]) -> -132.000416667 print Decimal(geotransform[0]) -> -132.0004166287710540927946567535400390625 In Qgis3x (on Windows64), I get: -132.00041663 in the layer properties. Qgis looks like the end result of: getcontext().prec = 19 print Decimal(geotransform[0])/Decimal(1) Must I use "Decimals" to deal with the geotransform output? Using float will truncate the decimal place and I may end up with an extra pixel row or column because of the size of the raster. So what is the proper way to work with this? Thanks Nicolas ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] FlatGeobuf; proposal for a new performance oriented vector file format
Hi GDAL/OGR folks, In my spare time I've been working on a vector file format called FlatGeobuf (tentatively). The main reason, besides curiosity and learning, I'm putting time into it is that I think shapefile still rules the read/query static data performance game, which is kind of sad, and probably one of the reasons it is still so widely popular. Also, the main competitor (GeoPackage) isn't suitable for streaming access (AFAIK) which shapefiles also handles surprisingly (?) well. By using a performance focused write once binary encoding (flatbuffers), schema constraint and focusing on read/query performance by clustering on an optimal spatial index (Packed Hilbert R-Tree) I hope to be able to beat shapefile performance and at the same time be optimal for streaming/cloud access. I think I'm starting to get somewhere, more details and source is at https://github.com/bjornharrtell/flatgeobuf and I have an early proof of concept driver implementation at https://github.com/bjornharrtell/gdal/tree/flatgeobuf and results are already quite promising - it can do roundtrip read/write and is already quite a bit faster than shapefile. I also have implemented naive read only QGIS provider for experimental purposes. Basically I'm fishing for interest in this effort, hoping that others will see potential in it even if it's "yet another format" and far from final/stable yet. Any feedback is welcome. As I see it, GDAL is a good place for a format specification and reference implementation to incubate. Some additional food for thought that I've had during the experimentation: 1. The main in memory representations of geometry/coordinates seem to be either separate arrays per dimension (GDAL (partially?) and QGIS) or a single array with dimension as stride. I've chosen the latter as of yet but I'm still a bit undecided. There is of course a substantial involved in transforming between the two representations so the situation with two competing models is a bit unfortunate. I'm also not sure about which of these models are objectively "better" than the other? 2. One could get better space efficiency with protobuf instead of flatbuffers, but it has a performance cost. I have not gone far into investigating how much though and one could even reason about supporting both these encodings in a single format specification but it's taking it a bit far I think. 3. Is there a reason to support different packing strategies for the R-Tree or is Packed Hilbert a good compromise (besides it being beautifully simple to implement)? 4. FlatGeobuf is perhaps a too technical name, not catchy enough and has a bad/boring abbreviation. Other candidates I've considered are OpenGeoFile, OpenVectorFile or OpenSpatialFile but I'm undecided. Any ideas? :) Regards, Björn Harrtell ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Lost in decimals... geotransform in python 2.7
On dimanche 9 décembre 2018 11:42:27 CET Nicolas Cadieux wrote:
> Hi,
>
> I am trying to get the four corners of a raster the size of Canada (very
> big therefore decimals count). If I get the geotransform, I get:
>
> geotransform: (-132.000417, 0.0008334, 0.0,
> 53.0004166, 0.0, -0.0008334)
> In Python 2.7:
> print (geotransform[0]) -> -132.000416667
> print float(geotransform[0]) -> -132.000416667
> print str(geotransform[0]) -> -132.000416667
> print Decimal(geotransform[0]) ->
> -132.0004166287710540927946567535400390625
> In Qgis3x (on Windows64), I get:-132.00041663 in the layer
> properties.
> Qgis looks like the end result of:
> getcontext().prec = 19
> print Decimal(geotransform[0])/Decimal(1)
>
> Must I use "Decimals" to deal with the geotransform output? Using float
> will truncate the decimal place and I may end up with an extra pixel row
> or column because of the size of the raster. So what is the proper way
> to work with this?
Nicolas,
Using float is perfectly fine. Internally GDAL stores the geotransform
components as IEEE-754 64bit numbers, and Python float is able to deal with
that (it uses this representation internally on platforms you care about). It
is just that the formatting functions (str, etc.) you use to convert the
number to string truncate the number of decimals. If you use repr() instead
you'll have more decimals
>>> repr(-132.000417)
'-132.000417'
Anyway, when dealing with floating point numbers, independently of their
decimal representation, you must be aware that they don't necessarily capture
perfectly the "real" number ( here -(132 + 0.5 / 1200) would be the exact
number ), so you must be ready to add appropriate rounding in computations
wherever needed.
Use string representation only as late as possible, or use 19 decimal
significant figures ('%.19g' C formatting) to keep the original precision of a
IEEE-754 64bit number.
Even
--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: adopt RFC 72: Run tests with pytest
+0 -Jukka Rahkonen- Even Rouault-2 wrote > PSC members, > > gentle reminder to cast your vote on this. > > Thanks, > > Even -- Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: adopt RFC 72: Run tests with pytest
Jonathan > it's worth spending a little thought on coming up with a scheme for test-ids. I've been through the list of parametrized tests and tweaked the `ids` kwargs to make them a little more helpful at first glance: https://github.com/OSGeo/gdal/pull/963/commits/8db599e7bc08b7dc73d81591898ed0f5f4243a58 I didn't see any way to use `pytest_make_parametrize_id` really; IDs rightly vary enough between tests that I can't see that hook being very useful here. Cheers Craig de Stigter On Mon, 10 Dec 2018 at 09:59 jratike80 wrote: > +0 > > -Jukka Rahkonen- > > > Even Rouault-2 wrote > > PSC members, > > > > gentle reminder to cast your vote on this. > > > > Thanks, > > > > Even > > > > > > -- > Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: adopt RFC 72: Run tests with pytest
I haven't had a chance to read the RFC yet, so I can't yet vote. However, a huge thank you to Craig and everyone else who put in effort to make this happen! On Sun, Dec 9, 2018 at 7:05 PM Craig de Stigter < craig.destig...@koordinates.com> wrote: > Jonathan > > > it's worth spending a little thought on coming up with a scheme for > test-ids. > > I've been through the list of parametrized tests and tweaked the `ids` > kwargs to make them a little more helpful at first glance: > https://github.com/OSGeo/gdal/pull/963/commits/8db599e7bc08b7dc73d81591898ed0f5f4243a58 > > I didn't see any way to use `pytest_make_parametrize_id` really; IDs > rightly vary enough between tests that I can't see that hook being very > useful here. > > Cheers > Craig de Stigter > > On Mon, 10 Dec 2018 at 09:59 jratike80 < > jukka.rahko...@maanmittauslaitos.fi> wrote: > >> +0 >> >> -Jukka Rahkonen- >> >> >> Even Rouault-2 wrote >> > PSC members, >> > >> > gentle reminder to cast your vote on this. >> > >> > Thanks, >> > >> > Even >> >> >> >> >> >> -- >> Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html >> ___ >> gdal-dev mailing list >> gdal-dev@lists.osgeo.org >> https://lists.osgeo.org/mailman/listinfo/gdal-dev > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev -- -- http://schwehr.org ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
