Hi, (subject renamed for clarity, original thread: part 1: https://lists.osgeo.org/pipermail/grass-dev/2017-October/thread.html#86395 part 2: https://lists.osgeo.org/pipermail/grass-dev/2017-December/086738.html )
Here the benchmarking with SRTM 30m global with 5.4 billion pixels (run on SSD disk): Welcome to GRASS GIS 7.5.svn (r71892) GRASS 7.5.svn (latlong):~ > export GRASS_COMPRESSOR=ZSTD time -p r.in.gdal /vsicurl/ https://www.datenatlas.de/geodata/public/srtmgl1/srtmgl1.003.tif output=srtmgl1_v003_30m_ZSTD memory=3000 360 degree EW extent is exceeded by 0.0395247 cells 360 degree EW extent is exceeded by 1 cells Importing raster map <srtmgl1_v003_30m_ZSTD>... 100% real 17697.11 user 17149.13 sys 195.68 g.region raster=srtmgl1_v003_30m_ZSTD -p 360 degree EW extent is exceeded by 0.0395247 cells 360 degree EW extent is exceeded by 1 cells 360 degree EW extent is exceeded by 1 cells 360 degree EW extent is exceeded by 1 cells projection: 3 (Latitude-Longitude) zone: 0 datum: wgs84 ellipsoid: wgs84 north: 60:00:00.5N south: 56:00:00.5S west: 180:00:00.5W east: 180:00:00.5E nsres: 0:00:01 ewres: 0:00:01 rows: 417601 cols: 1296001 cells: 541211313601 360 degree EW extent is exceeded by 1 cells r.colors srtmgl1_v003_30m_ZSTD color=srtm_plus Color table for raster map <srtmgl1_v003_30m_ZSTD> set to 'srtm_plus' r.compress -p srtmgl1_v003_30m_ZSTD 360 degree EW extent is exceeded by 1 cells <srtmgl1_v003_30m_ZSTD> is compressed (method 5: ZSTD). Data type: CELL <srtmgl1_v003_30m_ZSTD> has a compressed NULL file File size comparison: # ZLIB compressed, human readable: mundialis:~/grassdata/latlong/srtmgl1_30m$ find . -name srtmgl1_v003_30m | sort | xargs du -h 4,0K ./cats/srtmgl1_v003_30m 4,0K ./cellhd/srtmgl1_v003_30m 293M ./cell_misc/srtmgl1_v003_30m 157G ./cell/srtmgl1_v003_30m 4,0K ./colr/srtmgl1_v003_30m 4,0K ./hist/srtmgl1_v003_30m # ZLIB compressed, kB: mundialis:~/grassdata/latlong/srtmgl1_30m$ find . -name srtmgl1_v003_30m | sort | xargs du -k 4 ./cats/srtmgl1_v003_30m 4 ./cellhd/srtmgl1_v003_30m 299696 ./cell_misc/srtmgl1_v003_30m 163988608 ./cell/srtmgl1_v003_30m 4 ./colr/srtmgl1_v003_30m 4 ./hist/srtmgl1_v003_30m ls -la cell_misc/srtmgl1_v003_30m/ total 299696 -rw-rw-r-- 1 mundialis mundialis 306883360 Okt 25 21:19 nullcmpr -rw-rw-r-- 1 mundialis mundialis 13 Okt 25 21:19 range # ZSTD compressed, kB: mundialis:/scratch/grassdata/latlong/srtmgl1_30m$ find . -name srtmgl1_v003_30m_ZSTD | sort | xargs du -k 4 ./cats/srtmgl1_v003_30m_ZSTD 4 ./cellhd/srtmgl1_v003_30m_ZSTD 299708 ./cell_misc/srtmgl1_v003_30m_ZSTD 145278592 ./cell/srtmgl1_v003_30m_ZSTD 4 ./colr/srtmgl1_v003_30m_ZSTD 4 ./hist/srtmgl1_v003_30m_ZSTD ls -la cell_misc/srtmgl1_v003_30m_ZSTD/ total 299704 -rw-rw-r-- 1 mundialis mundialis 306883360 Dez 5 21:40 nullcmpr -rw-rw-r-- 1 mundialis mundialis 22 Dez 5 21:40 stats -rw-rw-r-- 1 mundialis mundialis 13 Dez 5 21:40 range Ratio: # CELL file > 145278592 / 163988608 [1] 0.8859066 # null file identical Curiosity: would it makes sense to compress the "nullcmpr" file according to the selected compression? Probably Markus M had already explained it but I don't remember... Best, markusN -- Markus Neteler, PhD http://www.mundialis.de - free data with free software http://grass.osgeo.org http://courses.neteler.org/blog
_______________________________________________ grass-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/grass-dev
