[..]
Nikos:
Some messy rough timings:
1) i7, 8 cores, 32GB RAM, Base OS: CentOS -> Three r.in.gdal processes
for "p2.tif", each stuck at 3% for almost 14h
2) Xeon, 24 Cores, 32GB RAM, Base OS: Windows -> Three gdal_translate
processes with -projwin, the VRT file as an input and GeoTIFF as output,
at 40% since yesterday afternoon
3) Xeon, 12 Cores, ? RAM, Base OS: CentOS.jpg -> Same processes as in
1), stuck at 0% of progress for more than 16h.
SSD can be seen as a "necessity".
Markus M:
Hmm, not really.
Nikos:
In a laptop (i7-4600U CPU @ 2.10GHz with 8GB of RAM with SSD) it was
progressing, in a quite acceptable manner.
Markus M:
What is the gdal version you used? I use gdal 2.1.3.
Well, yes! 2.1.3 in the laptop, 1.11.4 for the rest.
I had to break the process,
unfortunately, because I don't have a lot of free space :-/
maybe because you forgot the enable compression ;-)
I should!
With the p1 tif and GRASS db on the same spinning HDD, and
6 other heavy processes constantly reading from and writing to that same
HDD, r.in.gdal took 2h 13min to import the p1 tif. 360 MB as input and 1.5
GB as output is not that heavy on disk IO. Most of the time is spent
decompressing input and compressing output.
Is it an 10000rpm disk?
p2 is a harder one!
export GDAL_CACHEMAX=10000
gdal_translate -co "COMPRESS=LZW"
GHS_BUILT_LDS1990_GLOBE_R2016A_3857_38_v1_0_p2.tif p2_test.tif
I did not emphasize it enough, but cache size was among my questions
initially. I wrongly assumed that it can't be more than 2047 due to the
reference in <https://grass.osgeo.org/grass72/manuals/r.in.gdal.html>:
--%<---
memory=integer
..
Options: 0-2047
..
--->%--
I admit I did not head over to
https://trac.osgeo.org/gdal/wiki/ConfigOptions from where it is implied
that it can be much higher than 2047MB.
Can't r.in.gdal deal with memory=4096 for example (will try)? If yes,
can we update the manual(s)?
Also related? GTIFF_DIRECT_IO, GTIFF_VIRTUAL_MEM_IO
finishes in 28 minutes.
Impressive!
you could try gdal 2.1.3, maybe 2.1.3 has a more efficient cache regarding
block-wise reading than gdal 1.11.4
Yes, I have to.
Kudos, Nikos
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
