Hi, On Mon, Nov 11, 2019 at 8:57 AM Stefan Blumentrath <stefan.blumentr...@nina.no> wrote: > > Hi, > > Currently, I am working on a system (GPFS parallel file system) with quotas > not only on the amount of data (5 TB), but also on the number of files stored > in my project area (100,000). > > I did hit the number of files limit before i even got close to the quota on > the amount of data (in fact at ~ 1TB).
While I can kind of understand a size limit, do they motivate the limit on open files? Say, when explaining to the administrators how GRASS GIS maps work, might they increase the quota? > I saw that some changes in the GRASS raster format are planned for GRASS 8. > So, my question is, would it be possible and of interest to reduce the number > of files per map in GRASS 8 file format? E.g. writing color table, metadata, > NoData mask, and similar into one file? > > A large number of files can also reduce performance when working against > NFS... It will be a major work to change that. An option might be to aggregate the metadata files into one or at least less file (JSON format?) incl. color table, range, history etc. To embed NULL into the raster files was also discussed in the past. However, all ideas which can only be implemented in a major release (so, GRASS GIS 8, as you already said [1]). Markus PS: Funny idea: could you add "user land" file systems on top of GPFS to let GPSF just see a few blobs while you having access to the data on FUSE level? [1] https://trac.osgeo.org/grass/wiki/Grass8Planning _______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
