Hello! Did you add `ignite-compres` module to your classpath?
Have you tried WAL compression instead? Please check https://apacheignite.readme.io/docs/write-ahead-log#section-wal-records-compression Regards, -- Ilya Kasnacheev пт, 28 авг. 2020 г. в 06:52, 38797715 <[email protected]>: > Hi, > > create table statement are as follows: > CREATE TABLE PI_COM_DAY > (COM_ID VARCHAR(30) NOT NULL , > ITEM_ID VARCHAR(30) NOT NULL , > DATE1 VARCHAR(8) NOT NULL , > KIND VARCHAR(1), > QTY_IOD DECIMAL(18, 6) , > AMT_IOD DECIMAL(18, 6) , > QTY_PURCH DECIMAL(18, 6) , > AMT_PURCH DECIMAL(18,6) , > QTY_SOLD DECIMAL(18,6) , > AMT_SOLD DECIMAL(18, 6) , > AMT_SOLD_NO_TAX DECIMAL(18, 6) , > QTY_PROFIT DECIMAL(18, 6) , > AMT_PROFIT DECIMAL(18, 6) , > QTY_LOSS DECIMAL(18,6) , > AMT_LOSS DECIMAL(18, 6) , > QTY_EOD DECIMAL(18, 6) , > AMT_EOD DECIMAL(18,6) , > UNIT_COST DECIMAL(18,8) , > SUMCOST_SOLD DECIMAL(18,6) , > GROSS_PROFIT DECIMAL(18, 6) , > QTY_ALLOCATION DECIMAL(18,6) , > AMT_ALLOCATION DECIMAL(18,2) , > AMT_ALLOCATION_NO_TAX DECIMAL(18, 2) , > GROSS_PROFIT_ALLOCATION DECIMAL(18,6) , > SUMCOST_SOLD_ALLOCATION DECIMAL(18,6) , > PRIMARY KEY (COM_ID,ITEM_ID,DATE1)) WITH > "template=cache-partitioned,CACHE_NAME=PI_COM_DAY"; > CREATE INDEX IDX_PI_COM_DAY_ITEM_DATE ON PI_COM_DAY(ITEM_ID,DATE1); > > I don't think there's anything special about it. > Then we imported 10 million data using the COPY command.Data is basically > the actual production data, I think the dispersion is OK, not artificial > data with high similarity. > I would like to know if there are test results for the function of disk > compression? Most of the other memory databases also have the function of > data compression, but it doesn't look like it is now, or what's wrong with > me? > > 在 2020/8/28 上午12:39, Michael Cherkasov 写道: > > Could you please share your benchmark code? I believe compression might > depend on data you write, if it full random, it's difficult to compress the > data. > > On Wed, Aug 26, 2020, 8:26 PM 38797715 <[email protected]> wrote: > >> Hi, >> >> We turn on disk compression to see the trend of execution time and disk >> space. >> >> Our expectation is that after disk compression is turned on, although >> more CPU is used, the disk space is less occupied. Because more data is >> written per unit time, the overall execution time will be shortened in the >> case of insufficient memory. >> >> However, it is found that the execution time and disk consumption do not >> change significantly. We tested the diskPageCompressionLevel values as 0, >> 10 and 17 respectively. >> >> Our test method is as follows: >> The ignite-compress module has been introduced. >> >> The configuration of ignite is as follows: >> <?xml version="1.0" encoding="UTF-8"?> >> <beans xmlns="http://www.springframework.org/schema/beans"; >> <http://www.springframework.org/schema/beans> >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> <http://www.w3.org/2001/XMLSchema-instance> xsi:schemaLocation=" >> http://www.springframework.org/schema/beans >> http://www.springframework.org/schema/beans/spring-beans.xsd";> >> <bean id="ignite.cfg" class= >> "org.apache.ignite.configuration.IgniteConfiguration"> >> <property name="peerClassLoadingEnabled" value="true"/> >> <property name="consistentId" value="b"/> >> <property name="igniteInstanceName" value="ClusterName1"/> >> <property name="workDirectory" value="/home/ignite"/> >> <property name="gridLogger"> >> <bean class="org.apache.ignite.logger.log4j2.Log4J2Logger"> >> <constructor-arg type="java.lang.String" value="config/ignite-log4j2.xml" >> /> >> </bean> >> </property> >> <property name="cacheConfiguration"> >> <list> >> <bean id="partitioned-cache-template" abstract="true" class= >> "org.apache.ignite.configuration.CacheConfiguration"> >> <property name="name" value="cache-partitioned*"/> >> <property name="cacheMode" value="PARTITIONED" /> >> <property name="queryParallelism" value="2"/> >> <property name="diskPageCompression" value="LZ4"/> >> <property name="diskPageCompressionLevel" value="17"/> >> </bean> >> </list> >> </property> >> <!-- Enabling Apache Ignite Persistent Store. --> >> <property name="dataStorageConfiguration"> >> <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> >> <property name="pageSize" value="#{4096 * 2}"/> >> <property name="defaultDataRegionConfiguration"> >> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> >> <property name="persistenceEnabled" value="true"/> >> <property name="maxSize" value="#{1L * 1024 * 1024 * 1024}"/> >> </bean> >> </property> >> </bean> >> </property> >> </bean> >> </beans> >> >
