On 12/9/19 10:09 AM, Nyall Dawson wrote:
On Mon, 9 Dec 2019 at 18:07, Matthias Kuhn <[email protected]> wrote:
On 12/8/19 11:35 PM, Nyall Dawson wrote:
On Fri, 6 Dec 2019 at 23:19, Matthias Kuhn <[email protected]> wrote:
Other proposals are very welcome as well. I don't insist on GeoPackage.
All I do is being a bit skeptic that rolling our own format will
magically solve problems that one hundred other formats did not solve :-)
The issue is that the memory provider, by design, MUST be a lossless
reflection of all the capabilities offered in QGIS vector layers. That
means support for every geometry type, and field type handled by QGIS.
Geopackage might be capable of that now... but it won't always be, due
to the inherent limitations of that format. (No multi-geometry
support, no mixed CRS geometry support, no support for advanced field
types like range/interval fields, etc).
Having our own binary format makes sense from a full QGIS data type
support perspective.
As mentioned before, I don't insist on GeoPackage (but now that you say
it, does it not have multi-geometry support? According to the specs it
should. And just like we can binary dump all our field formats into our
own binary file, we could dump them into any file, like e.g. GeoPackage).
Nope: https://www.geopackage.org/guidance/modeling.html
"Allowing multiple geometries per feature table would compromise
GeoPackage's position in the GIS application ecosystem"
(FWIW, that page is ridiculous, arrogant, and completely neglects some
very valid use cases)
The geopackage specs say otherwise
(http://www.geopackage.org/spec/#geometry_types) and so does our very
own dialog to create geopackage.
If someone wrote modelling guidelines, that was probably meant as best
practice (on which one can agree or not).
I don't think it's useful to even think of rolling our own data format
-- that's NOT what's required here. All that we need is a way of
paging the currently in-memory storage of features used by the memory
provider to disk when required. This disk based paging would be
totally temporary (always deleted when the layer is removed), have no
requirement for a stable binary format (we could change the paging
binary structure version by version if needed), and absolutely no need
for ANY other programs to be able to parse or interpret. It's not a
data format at all -- it's just a disk based reflection of what we
currently store in RAM, along with some smart indexing to optimize
access.
Implementing indexing and paged loading of requested data resolved by an
index sound like fun, do you think the effort (or the risks involved) is
smaller than tacking support for the missing bits to another format?
Honestly, yes. I suspect it's more work upfront, but less long-term
pain of applying hacks on hacks to try to duct tape functionality into
a standard format.
Wasn't it just thought as a band aid until flatgeobuf comes (riding on a
unicorn) to solve all our issues?
I am counting the days until someone wants to recover from one of these
files ;)
As soon as anyone tries that, I'm closing the bug report/feature
request as "hell-no-wont-fix" ;)
That will be a good move if we follow this path. But I still think
accessible formats have their merits.
Matthias
_______________________________________________
QGIS-Developer mailing list
[email protected]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
