On 6 September 2016 at 15:23, Markus Metz <[email protected]> wrote:
> On Tue, Sep 6, 2016 at 1:44 PM, Markus Neteler <[email protected]> wrote: > > Hi, > > > > since my local drive was filled up again :) I checked raster how data are > > currently compressed in GRASS GIS 7.2.svn. > > According to > > > > https://grass.osgeo.org/grass72/manuals/r.compress. > html#used-compression-algorithms > > > > all maps are DEFLATE compressed by default: > > > > "Raster maps are by default ZLIB compressed. > > ... > > > > Floating point (FCELL, DCELL) raster maps never use RLE compression; they > > are either compressed with ZLIB, LZ4, BZIP2 or are uncompressed. > > " > > > > Ehm, now how *are* FCELL, DCELL compressed by default? Not quite clear to > > me! This document needs improvements. > > The manual says, as you cited: > "Raster maps are by default ZLIB compressed." > What exactly is unclear about this? Should it say "All raster maps ..." ? > > > > > Reality check with Sentinel-2 data (3 different bands, same regional > > extent): > > > > GRASS 7.2.svn (utm37n):~ > r.info -r s2_20151225_B02_10m > > min=0 > > max=22937 > > GRASS 7.2.svn (utm37n):~ > r.info -r s2_20151225_B04_10m > > min=0 > > max=18849 > > GRASS 7.2.svn (utm37n):~ > r.info -r s2_20151225_B8A_20m > > min=0 > > max=17210 > > > > GRASS 7.2.svn (utm37n):~ >:~ > r.compress s2_20151225_B02_10m -p > > <s2_20151225_B02_10m> is compressed (method 2: ZLIB). Data type: > > CELL > > > > GRASS 7.2.svn (utm37n):~ >:~ > r.compress s2_20151225_B04_10m -p > > <s2_20151225_B04_10m> is compressed (method 2: ZLIB). Data type: > > CELL > > > > GRASS 7.2.svn (utm37n):~ >:~ > r.compress s2_20151225_B8A_20m -p > > <s2_20151225_B8A_20m> is compressed (method 2: ZLIB). Data type: > > CELL > > > > So far so nice. Now the suprising part, while the channels are not > identical > > (obviously, since covering different spectral parts), the map sizes are > > identical! > > > > GRASS 7.2.svn (utm37n):~ > ls -la > > ... > > -rw-r--r-- 1 neteler neteler 2539235691 Jun 14 17:14 s2_20151225_B02_10m > > -rw-r--r-- 1 neteler neteler 2539235691 Jun 14 17:19 s2_20151225_B03_10m > > -rw-r--r-- 1 neteler neteler 2539235691 Jun 14 17:25 s2_20151225_B04_10m > > -rw-r--r-- 1 neteler neteler 634878630 Jun 14 20:36 s2_20151225_B05_20m > > -rw-r--r-- 1 neteler neteler 634878630 Jun 14 20:37 s2_20151225_B06_20m > > -rw-r--r-- 1 neteler neteler 634878630 Jun 14 20:39 s2_20151225_B07_20m > > -rw-r--r-- 1 neteler neteler 634878630 Jun 14 20:40 s2_20151225_B11_20m > > -rw-r--r-- 1 neteler neteler 634878630 Jun 14 20:42 s2_20151225_B12_20m > > -rw-r--r-- 1 neteler neteler 634878630 Jun 14 20:43 s2_20151225_B8A_20m > > > > I would expect different sizes, compression can hardly lead to identical > > file sizes. > > The default ZLIB compression level was invalid, causing ZLIB to not > compress at all. Fixed in r69387,8. > > Markus M > > > > > Next test: gzip the file > > > > GRASS 7.2.svn (utm37n):~/grassdata/utm37n/PERMANENT/cell > ls -la > > s2_20151225_B03_10m > > -rw-r--r-- 1 mneteler mneteler 2539235691 Jun 14 17:19 > s2_20151225_B03_10m > > > > GRASS 7.2.svn (utm37n):~/grassdata/utm37n/PERMANENT/cell > gzip > > s2_20151225_B03_10m > > > > GRASS 7.2.svn (utm37n):~/grassdata/utm37n/PERMANENT/cell > ls -la > > s2_20151225_B03_10m.gz > > -rw-r--r-- 1 mneteler mneteler 1456248453 Jun 14 17:19 > > s2_20151225_B03_10m.gz > > > > R > >> 1456248453/2539235691 > > [1] 0.5734987 > > > > Quite smaller! So I am not at all convinced that these CELL files are > > currently ZLIB compressed. > > Compressing a whole file instead of compressing each row separately > (GRASS reads and writes raster data row by row) can lead to higher > compression ratios. > This brings to a long time amount of discussions about parallelization speed-ups of raster functions being limited by the row-based I/O of GRASS. Maybe we should look into this for GRASS8... > > > > > From this ticket I would expect something else: > > https://trac.osgeo.org/grass/ticket/2349 > > > > Ah, and no specific environment variables are set: > > > > GRASS 7.2.svn (utm37n):~ > echo $GRASS_<tab> > > $GRASS_ADDON_BASE $GRASS_GNUPLOT $GRASS_HTML_BROWSER $GRASS_PAGER > > $GRASS_PROJSHARE $GRASS_PYTHON $GRASS_VERSION > > > > > > A bug? > > > > Markus > > > > _______________________________________________ > > grass-dev mailing list > > [email protected] > > http://lists.osgeo.org/mailman/listinfo/grass-dev > _______________________________________________ > grass-dev mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/grass-dev > -- Yann Chemin Skype/FB: yann.chemin
_______________________________________________ grass-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-dev
