Hi Even, Thank you for sharing this finding!
In that case I think that QGIS should offer a flag to open Geopackages in read-only mode. Even - since you are "Mr. Geopackage" in the QGIS project anyway - can you please make sure that this can be covered in the upcoming Geopackage improvements? Would a "read-only" mode also be useful for other data formats? I opened a ticket at http://hub.qgis.org/issues/15652 and assigned it to you. I have no idea how much effort this enhancement would mean - can you please do an estimate and send me a quote for that? Maybe we could still include it in the upcoming Geopackage improvements financed by the Swiss QGIS user group. ------------------- This also leads to a follow up question: many OGR data formats have "open options". As far as I know, these options are not exposed to the User in QGIS, other than the encoding - or did I miss something? Should we introduce a "GUI" way in QGIS 3.x to expose these "open options" in the "add vector layer" dialogue? Thanks, Andreas On 2016-10-01 23:38, Even Rouault wrote: > FYI: An interesting finding to improve performance of large Geopackage on a > shared network drive. > > ---------- Message transmis ---------- > > Sujet : Re: [Geopackage] Geopackage on a shared network drive > Date : samedi 01 octobre 2016, 16:41:05 > De : Árni Geirsson via Geopackage <[email protected]> > À : Even Rouault <[email protected]> > CC : [email protected] > > Yes! It renders very quickly after I set the read-only flag in the file > explorer. > That goes a long way towards solving the problem for me because as I > mentioned, the team would like to use this format to store geodata, not to > edit it and certainly not to make edits by multiple concurrent users. > Thank you so much for your help! > > Árni > > Árni Geirsson > *Alta ehf* // +354 582 5000 // +354 897 9549 > www.alta.is [1] // Alta á Twitter <http://twitter.com/alta_ehf> // Alta á > Facebook <http://www.facebook.com/pages/Alta/161795813838691?v=wall> > Gæða- og umhverfisstefna Alta <http://alta.is/pdf/svonavinnumvid.pdf> > > On 1 October 2016 at 14:28, Even Rouault <[email protected]> wrote: > > Le samedi 01 octobre 2016 16:02:38, Árni Geirsson a écrit : Hello Even > Thank you for your suggestion. I made the changes you suggested, adding > set SQLITE_USE_OGR_VFS=YES > after > set VSI_CACHE=TRUE > set VSI_CACHE_SIZE=1000000 > but there was no difference. Was this perhaps not the right way to do it? > Yes, that's correct. > > Hum actually reviewing the code, the cache mechanism is only enabled when > the > file is opened in read-only mode, whereas QGIS will open the file in > read-write > mode. > > You could perhaps try to make the GPKG read-only in the explorer?, at least > just to test if it is a promising optimization. I guess write support for > the > cache mechanism could be added. > > A drawback of a cache machnism is that it would defeat the updates in > SQLite > to deal with concurrent editing, but anyway such mechanisms don't work very > well on network shares. > > Anyway, I would think that I am not alone in seeing this as a problem but I > find very little about it on the web. > > Árni > > Árni Geirsson > *Alta ehf* // +354 582 5000 // +354 897 9549 > www.alta.is [1] // Alta á Twitter <http://twitter.com/alta_ehf> // Alta á > Facebook <http://www.facebook.com/pages/Alta/161795813838691?v=wall> > Gæða- og umhverfisstefna Alta <http://alta.is/pdf/svonavinnumvid.pdf> > > On 30 September 2016 at 19:52, Even Rouault <[email protected]> > > wrote: Le vendredi 30 septembre 2016 00:44:45, Árni Geirsson via Geopackage a > > écrit : Dear Geopackage developers! > I look forward to the day when Geopackage replaces the old shapefiles > and I > > thank you for your efforts. As a user in a small team that uses QGIS > and shares data on a NAS box, basically a Samba server for network > shares, I notice that Geopackage data loads very slowly, much more > slowly than a shapefile stored in exactly the same shared folder. I > have noticed this with Spatialite, too, and suspect that the problem > lies with Sqlite and some problem it has with the networking protocol. > Since Geopackage is intended (as I understand) as a storage and > exchange format for geodata, rather than a general purpose database > format like Spatialite, I was > hoping > > that this problem could be overcome and that the Geopackage data would > load > > as fast as the shapefile data from the network share. When the > Geopackage and shapefile data are both on a local drive, I see no > difference in the loading speed. > > Do you think this will be solved? > QGIS tweaks GDAL/OGR to use a large buffer size to speed up network > access to > shapefiles (or any format that uses the GDAL I/O layer, which is called > "VSI"). > > If you look at the qgis.bat in c:\osgeo4w\bin, you will see : > set VSI_CACHE=TRUE > set VSI_CACHE_SIZE=1000000 > > But by default, the OGR GPKG driver uses SQLite own I/O layer, which > probably > does access with a granualrity of a SQLite page size (4KBytes). You could >> tried to define the >> SQLITE_USE_OGR_VFS=YES environmenet variable, restart QGIS and then >> SQLite should use the GDAL/OGR I/O layer and hopefully benefit from the >> buffering. I >> have not tried that, so not completely sure if it will improve things. >> >> Even >> >> -- >> Spatialys - Geospatial professional services >> http://www.spatialys.com -- Spatialys - Geospatial professional services http://www.spatialys.com ------------------------------------------------------- Links: ------ [1] http://www.alta.is
_______________________________________________ Qgis-developer mailing list [email protected] List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
