Hi,
I got information from Matthias, that the new "Temporary Output Folder
Path" is only used by algorithms around GDAL, SAGA and ExecuteSQL, and
not for vector based QGIS algorithms. I don't know about GRASS -
probably this also runs through files?
I think we need to address this issue with priority, ideally also with
at backport to our LT version 3.10. Running large vector files always
through memory sounds dangerous to me. It can easily bring a system to
halt.
The question is, if the switch between memory only operations, or disk
based operations can be determined by software or if it requires user
intervention. Often, it is hard to guess in advance, whether an
algorithm runs out of memory or not. Certainly our average users would
have no idea about this.
I wonder, how other systems do it? As far as I know, FME always does
things disk based, to stay on the safe side - or am I wrong?
Greetings,
Andreas
On 2019-12-05 08:18, Richard Duivenvoorde wrote:
On 05/12/2019 01.01, Nyall Dawson wrote:
What's needed/missing is a method for the memory data provider to
detect when a layer is growing too large, and start writing this out
to disk. (Important note: we CAN'T use any of the standard spatial
formats for this (as they are lossy when compared to the memory
provider support), so we'd need to directly serialize the memory layer
features to a custom binary format, with indexing support for rapid
retrieval of features.
If I understand correct, FME is also working with (indexed) temp files?
I'm not an FME user, so I can bu wrong, but I always wondered what kind
of format that is...
IF we (as FOSS community) could come up with some (new?) indexed file
format, it (besides being a in-between product for transformations) also
a potential data exchange format (maybe some zipped version of file+index?)?
Anybody into this?
Regards,
Richard Duivenvoorde
_______________________________________________
QGIS-Developer mailing list
[email protected]
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
