Re: [Qgis-user] Qgis-user Digest, Vol 157, Issue 20

[rjwillson]

 Original message From: qgis-user-requ...@lists.osgeo.org Date: 
2019-03-10  13:00  (GMT-06:00) To: qgis-user@lists.osgeo.org Subject: Qgis-user 
Digest, Vol 157, Issue 20 Send Qgis-user mailing list submissions to
qgis-user@lists.osgeo.orgTo subscribe or unsubscribe via the World Wide Web, 
visit  https://lists.osgeo.org/mailman/listinfo/qgis-useror, via email, 
send a message with subject or body 'help' to  
qgis-user-request@lists.osgeo.orgYou can reach the person managing the list at  
qgis-user-owner@lists.osgeo.orgWhen replying, please edit your Subject line so 
it is more specificthan "Re: Contents of Qgis-user digest..."Today's Topics:   
1. Re: Couldn't load plugin - QGIS 3.6 on Mac 10.14.3  (William 
Kyngesburye)   2. Problem exporting layer to GeoPkg (qgis-u...@stripfamily.net) 
  3. Re: Couldn't load plugin - QGIS 3.6 on Mac 10.14.3  (Saber Razmjooei)  
 4. Re: Couldn't load plugin - QGIS 3.6 on Mac 10.14.3 (kspencer)   5. Re: 
Couldn't load plugin - QGIS 3.6 on Mac 10.14.3  (William Kyngesburye)   6. 
Re: Couldn't load plugin - QGIS 3.6 on Mac 10.14.3 
(kspencer)--Message:
 1Date: Sat, 9 Mar 2019 17:22:55 -0600From: William Kyngesburye 
To: kspencer Cc: 
qgis-user@lists.osgeo.orgSubject: Re: [Qgis-user] Couldn't load plugin - QGIS 
3.6 on Mac   10.14.3Message-ID: 
Content-Type: text/plain; 
charset="utf-8"Oh bugger.  I'm uploading a new installer, it should be on the 
QGIS download page tomorrow.> On Mar 9, 2019, at 12:56 PM, kspencer 
 wrote:> > Hello folks,> > I'm a QGIS newb trying to 
install QGIS on my Mac.  Any help would be greatly> appreciated. Got this error 
when trying to run QGIS for the first time:> > Couldn't load plugin 
'processing' > > ImportError:> 
dlopen(/Library/Frameworks/SQLite3.framework/Versions/E/Python/3.6/sqlite3/_sqlite3.cpython-36m-darwin.so,>
 2): Library not loaded:> 
/Library/Frameworks/SQLite3.framework/Versions/D/SQLite3   Referenced from:> 
/Library/Frameworks/SQLite3.framework/Versions/E/Python/3.6/sqlite3/_sqlite3.cpython-36m-darwin.so
  > Reason: image not found > Traceback (most recent call last):>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 309, in loadPlugin>    __import__(packageName)>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, fromlist, 
level)>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/plugins/processing/__init__.py",>
 line 29, in >    from processing.tools.general import *  # NOQA>  
File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, fromlist, 
level)>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/plugins/processing/tools/general.py",>
 line 39, in >    from processing.core.Processing import Processing>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, fromlist, 
level)>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/plugins/processing/core/Processing.py",>
 line 58, in >    from processing.algs.qgis.QgisAlgorithmProvider import> 
QgisAlgorithmProvider  # NOQA>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, fromlist, 
level)>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/plugins/processing/algs/qgis/QgisAlgorithmProvider.py",>
 line 83, in >    from .ImportIntoSpatialite import ImportIntoSpatialite>  
File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, fromlist, 
level)>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/plugins/processing/algs/qgis/ImportIntoSpatialite.py",>
 line 42, in >    from processing.tools import spatialite>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, fromlist, 
level)>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/plugins/processing/tools/spatialite.py",>
 line 29, in >    import sqlite3 as sqlite>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, fromlist, 
level)>  File> 
"/Library/Frameworks/SQLite3.framework/Versions/E/Python/3.6/sqlite3/__init__.py",>
 line 23, in >    from sqlite3.dbapi2 import *>  File> 
"/Applications/QGIS3.6.app/Contents/MacOS/../Resources/python/qgis/utils.py",> 
line 672, in _import>    mod = _builtin_import(name, globals, locals, 

Re: [Qgis-user] How to Prevent GeoPackage file from updating modified date when viewed in a Qgis project when it is not being edited

[rjwillson]

Thanks Even,
It definitely doesn't sound like an easy solution. 
I wonder how others deal with it and maybe someone in the developers list will 
have some ideas
I have just been applying band aid fixes for the last year to prevent the gpkgs 
from updating themselves but yesterday I discovered the beauty of storing 
rasters in the gpkg format. Unfortunately the rasters make the gpkgs large and 
so I can't have them just start uploading on a field tablet via cell connection 
when they get viewed in Q or other app like Roam. 
 I have been trying to get off shapefiles for 15 years now because of their 
limitations but I find I have to keep going back to them
Rob 
 Original message From: Even Rouault 
 Date: 2018-04-09  16:22  (GMT-05:00) To: 
qgis-user@lists.osgeo.org Cc: rjwill...@gmail.com Subject: Re: [Qgis-user] How 
to Prevent GeoPackage file from updating modified date when viewed in a Qgis 
project when it is not being edited 
On lundi 9 avril 2018 15:24:41 CEST rjwill...@gmail.com wrote:
> I would really like to use and promote GeoPackage as the go to data format
> in my organization but I am having the following problem that I haven't been
> able to solve.
> 
> 
> 
> When ever someone opens a Q template project that has a layer with a
> GeoPackage data source, the GeoPackage file (not its contents) will be
> altered and it will show that it has been updated in the Windows file
> manager. I have been told that this happens because the write ahead logging
> has been triggered (creation of *.wal and *.shm) files by simply viewing the
> contents of the GeoPackage in QGIS and/or in Intramaps Roam (works on top
> of QGIS for tablets).
> 
> 
> 
> This is problematic because most users in our organization will then look at
> the windows modified date time stamp and conclude that the file's contents
> have been modified. Additionally, all of the GIS layers are stored in
> Dropbox and subsequently Dropbox concludes that the file has been modified
> and it re-uploads a new copy of the GeoPackage. When it is a large
> GeoPackage > 1 GB in size this is quite problematic and causes a cascade of
> uploads and downloads.
> 
> 
> 
> Over the last few months I have tried to suppress this modification for the
> large GeoPackages in our system in the following ways: (1) make the layer
> read only in the QGIS project (no effect); (2) make the Dropbox folder read
> only for all users except the Dropbox administrator (this is effective for
> the large base layers); (3) change the properties of the file in windows to
> read only (effective for the Dropbox administrator); and (4) add a user
> variable to the windows system environment variables: OGR_SQLITE_JOURNAL =
> DELETE (effective for QGIS but not for Intramaps Roam)
> 
> 
> 
> Combining 2, 3, and 4 allows me to use GeoPackage for layers I can ensure
> are placed in the read-only Dropbox folders.
> 
> 
> 
> It is likely that my organization will never need the simultaneous editing
> of datasets that is facilitated by Write-Ahead Logging, but I would really
> like to be able to replace shapefiles and File geodatabases with GeoPackages
> in all of our projects going forward.
> 
> 
> 
> Therefore, is there a simpler way to suppress the write ahead logging
> behaviour that causes the files to believe they have been updated for all
> GeoPackages created in QGIS or used in a QGIS project?

My memory is already fading about the details, but the reason for the current 
behaviour is that if you open a GeoPackage database in read-only non-WAL mode 
and someone else try to edit it concurrently, unpleasant locking will occur. 
Actually due to how QGIS works, locking could occur even in the same session 
(since background rendering threads can open read-only connections while the 
main connection is writing data). See https://issues.qgis.org/issues/15351

So the solution was:

commit b6b8759efbeb833d0d3dbf6df008087701361ad3
Author: Even Rouault 
Date:   Thu Sep 22 23:23:00 2016 +0200

    Fix database locking when editing GeoPackage
    
    Concurrent read and write can lock a GeoPackage database given
    the default journaling mode of SQLite (delete). Use WAL when
    possible to avoid that.
    
    Fixes #15351

I understand your use case as well, but I don't have a solution in mind that 
would fit all. 
I don't remember if I tried scenarios where one connection would open in 
DELETE mode and another one would try to turn to WAL on. 
A quick try shows that the a connection that opens in DELETE mode doesn't see 
the change to WAL done by another one. And WAL documentation doesn't describe 
this scenario, so it is quite unknown what would happen. So the current 
behaviour is to be on the safe side and always turn on WAL on opening.

Or perhaps have some option in a dialog box somewhere! "I promise that I will 
not edit this file in any concurrent session, or in this session, so you can 
safely open it in 

[Qgis-user] How to Prevent GeoPackage file from updating modified date when viewed in a Qgis project when it is not being edited

[rjwillson]

I would really like to use and promote GeoPackage as the go to data format
in my organization but I am having the following problem that I haven't been
able to solve.

 

When ever someone opens a Q template project that has a layer with a
GeoPackage data source, the GeoPackage file (not its contents) will be
altered and it will show that it has been updated in the Windows file
manager. I have been told that this happens because the write ahead logging
has been triggered (creation of *.wal and *.shm) files by simply viewing the
contents of the GeoPackage in QGIS and/or in Intramaps Roam (works on top of
QGIS for tablets). 

 

This is problematic because most users in our organization will then look at
the windows modified date time stamp and conclude that the file's contents
have been modified. Additionally, all of the GIS layers are stored in
Dropbox and subsequently Dropbox concludes that the file has been modified
and it re-uploads a new copy of the GeoPackage. When it is a large
GeoPackage > 1 GB in size this is quite problematic and causes a cascade of
uploads and downloads.

 

Over the last few months I have tried to suppress this modification for the
large GeoPackages in our system in the following ways: (1) make the layer
read only in the QGIS project (no effect); (2) make the Dropbox folder read
only for all users except the Dropbox administrator (this is effective for
the large base layers); (3) change the properties of the file in windows to
read only (effective for the Dropbox administrator); and (4) add a user
variable to the windows system environment variables: OGR_SQLITE_JOURNAL =
DELETE (effective for QGIS but not for Intramaps Roam)

 

Combining 2, 3, and 4 allows me to use GeoPackage for layers I can ensure
are placed in the read-only Dropbox folders.

 

It is likely that my organization will never need the simultaneous editing
of datasets that is facilitated by Write-Ahead Logging, but I would really
like to be able to replace shapefiles and File geodatabases with GeoPackages
in all of our projects going forward.

 

Therefore, is there a simpler way to suppress the write ahead logging
behaviour that causes the files to believe they have been updated for all
GeoPackages created in QGIS or used in a QGIS project? 

 

Rob

 

 

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user