Hello, Ignite compresses each page individually. The result of whole file compression will always be better than the result of each individual page compression. Moreover, Ignite stores compressed pages only if the page size shrunk by one or more filesystem blocks. So, for example, if you have fs block size 4K, page size 16Kb and after compression your page size is 13Kb, then the page will be stored without compression.
BTW, how do you check file size? Ignite compression uses sparse files. "ls -l" reports allocated file size and doesn't utilize information about "holes" in a sparse file. To see the real amount of disk space occupied by the file you should use "du" or "ls -s". вт, 17 нояб. 2020 г. в 06:18, David Tinker <[email protected]>: > I have enabled compression > (pageSize=16384, diskPageCompression=ZSTD, diskPageCompressionLevel=18) but > the partition files don't appear to be very compressed. I tested by adding > approx 16000 data items to my cache and looking at the partition files on > disk. > > Example: part-96.bin is 339M in size. If I compress that file with zstd > (default settings) it goes down to 106M. > > Is it possible to do better than this with Ignite? I need to be able to > store a lot of data. > > Thanks > David > > Relevant parts of my ignite config: > > <bean id="grid.cfg" > class="org.apache.ignite.configuration.IgniteConfiguration"> > <property name="consistentId" value=""/> > > <property name="dataStorageConfiguration"> > <bean > class="org.apache.ignite.configuration.DataStorageConfiguration"> > <property name="pageSize" value="16384"/> > ... > </bean> > </property> > > <property name="cacheConfiguration"> > <bean > class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="activity-stream-data"/> > <property name="atomicityMode" value="ATOMIC"/> > <property name="diskPageCompression" value="ZSTD"/> > <property name="diskPageCompressionLevel" value="18"/> > <property name="backups" value="1"/> > </bean> > </property> > </bean> > >
