Re: [QGIS-Developer] The curious case of GeoPackage on exFAT
Hi Jonathan - Thanks for the "heads-up" regarding wal files. The technical problem is using the xFAT file system - when xFAT gets corrupted, it will partly be write-protected. This will inhibit QGIS in making the 2 wal files (or updating or deleting them). But QGIS has an issue here: When the situation occurs, QGIS will quietly tell you about in the relatively obscure log system. And it will show you an *empty* GeoPackage - no layers. That is not smart. It will upset users (It certainly did upset me :-) . They will believe, that their precious layers in the GeoPackage has disappeared. However, the layers is perfectly alright. If you copy the GeoPackage to another filesystem, QGIS will happily open the layers from the copy. IMHO, a better approach would to have a big fat error message informing users that they have a problem with the underlying file system. (I plan to write a feature request about this later today) -- Med venlig hilsen / Kind regards Bo Victor Thomsen Den 18-05-2020 kl. 17:49 skrev Jonathan Moules: Hi Bo, Looking at the WAL docs for SQLite (https://www.sqlite.org/wal.html) (what geopackage is built from) I see this: "The WAL file exists for as long as any database connection has the database open. Usually, the WAL file is deleted automatically when the last connection to the database closes. However, if the last process to have the database open exits without cleanly shutting down the database connection, or if the SQLITE_FCNTL_PERSIST_WAL file control is used, then the WAL file might be retained on disk after all connections to the database have been closed." So it sounds like whatever the last process is to touch the GeoPackage may not be closing the connection cleanly on exFat. Cheers, Jonathan On 2020-05-11 13:10, Bo Victor Thomsen wrote: Hi all - I have a strange problem. I'm have 3 different disk on my windows based system on Mac hardware 1. My system drive. Formatted to NTFS. 2. A flash-drive. Formatted to FAT32. 3. A data drive. Formatted to exFAT. The last is my primary data drive and is shared between my Windows partition and my Mac partition on my MacBook Pro. Hence the use of exFAT. I have a QGIS plugin, which copies a template of a GeoPackage file to "where-ever the user wants it placed" and afterward make some content changes in the copy using the PyQT QSQL module with the QSPATIALITE driver This work if the Geopackage file is copied to either disk no 1 (NTFS) or disk no 2 (FAT32). However, it doesn't work if the file is copied to disk no 3 (exFAT). The process leaves the WAL files even after the database is closed properly. And even more strange: If I reformat the flash-drive to exFAT and repeat the experiment using the reformatted drive it too works without a hitch. The normal "divide et impera" method tells me that my exFat data disk is bork'ed. However this error *only* occurs with the QGIS/GeoPackage creation/modification scenario. Everything else is working OK. The disk is not shared on the network. Has anyone experienced the same type of problems ? And have a solution ?? Just asking before I begin to clean up / reformat my 100 GB data disk System setup: MacBook Pro 2014 / Windows 8.1 OS /QGIS 3.10.5 (the same problem occurs with 3.10.0 , 3.10.2 ...3.12.2) ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer -- Med venlig hilsen / Kind regards Bo Victor Thomsen ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] The curious case of GeoPackage on exFAT
Hi Bo, Looking at the WAL docs for SQLite (https://www.sqlite.org/wal.html) (what geopackage is built from) I see this: "The WAL file exists for as long as any database connection has the database open. Usually, the WAL file is deleted automatically when the last connection to the database closes. However, if the last process to have the database open exits without cleanly shutting down the database connection, or if the SQLITE_FCNTL_PERSIST_WAL file control is used, then the WAL file might be retained on disk after all connections to the database have been closed." So it sounds like whatever the last process is to touch the GeoPackage may not be closing the connection cleanly on exFat. Cheers, Jonathan On 2020-05-11 13:10, Bo Victor Thomsen wrote: Hi all - I have a strange problem. I'm have 3 different disk on my windows based system on Mac hardware 1. My system drive. Formatted to NTFS. 2. A flash-drive. Formatted to FAT32. 3. A data drive. Formatted to exFAT. The last is my primary data drive and is shared between my Windows partition and my Mac partition on my MacBook Pro. Hence the use of exFAT. I have a QGIS plugin, which copies a template of a GeoPackage file to "where-ever the user wants it placed" and afterward make some content changes in the copy using the PyQT QSQL module with the QSPATIALITE driver This work if the Geopackage file is copied to either disk no 1 (NTFS) or disk no 2 (FAT32). However, it doesn't work if the file is copied to disk no 3 (exFAT). The process leaves the WAL files even after the database is closed properly. And even more strange: If I reformat the flash-drive to exFAT and repeat the experiment using the reformatted drive it too works without a hitch. The normal "divide et impera" method tells me that my exFat data disk is bork'ed. However this error *only* occurs with the QGIS/GeoPackage creation/modification scenario. Everything else is working OK. The disk is not shared on the network. Has anyone experienced the same type of problems ? And have a solution ?? Just asking before I begin to clean up / reformat my 100 GB data disk System setup: MacBook Pro 2014 / Windows 8.1 OS /QGIS 3.10.5 (the same problem occurs with 3.10.0 , 3.10.2 ...3.12.2) -- Med venlig hilsen / Kind regards Bo Victor Thomsen ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
[QGIS-Developer] The curious case of GeoPackage on exFAT
Hi all - I have a strange problem. I'm have 3 different disk on my windows based system on Mac hardware 1. My system drive. Formatted to NTFS. 2. A flash-drive. Formatted to FAT32. 3. A data drive. Formatted to exFAT. The last is my primary data drive and is shared between my Windows partition and my Mac partition on my MacBook Pro. Hence the use of exFAT. I have a QGIS plugin, which copies a template of a GeoPackage file to "where-ever the user wants it placed" and afterward make some content changes in the copy using the PyQT QSQL module with the QSPATIALITE driver This work if the Geopackage file is copied to either disk no 1 (NTFS) or disk no 2 (FAT32). However, it doesn't work if the file is copied to disk no 3 (exFAT). The process leaves the WAL files even after the database is closed properly. And even more strange: If I reformat the flash-drive to exFAT and repeat the experiment using the reformatted drive it too works without a hitch. The normal "divide et impera" method tells me that my exFat data disk is bork'ed. However this error *only* occurs with the QGIS/GeoPackage creation/modification scenario. Everything else is working OK. The disk is not shared on the network. Has anyone experienced the same type of problems ? And have a solution ?? Just asking before I begin to clean up / reformat my 100 GB data disk System setup: MacBook Pro 2014 / Windows 8.1 OS /QGIS 3.10.5 (the same problem occurs with 3.10.0 , 3.10.2 ...3.12.2) -- Med venlig hilsen / Kind regards Bo Victor Thomsen ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
