Hi,
create table statement are as follows:
CREATETABLEPI_COM_DAY
(COM_ID VARCHAR(30) NOTNULL,
ITEM_ID VARCHAR(30) NOTNULL,
DATE1 VARCHAR(8) NOTNULL,
KIND VARCHAR(1),
QTY_IOD DECIMAL(18, 6) ,
AMT_IOD DECIMAL(18, 6) ,
QTY_PURCH DECIMAL(18, 6) ,
AMT_PURCH DECIMAL(18,6) ,
QTY_SOLD DECIMAL(18,6) ,
AMT_SOLD DECIMAL(18, 6) ,
AMT_SOLD_NO_TAX DECIMAL(18, 6) ,
QTY_PROFIT DECIMAL(18, 6) ,
AMT_PROFIT DECIMAL(18, 6) ,
QTY_LOSS DECIMAL(18,6) ,
AMT_LOSS DECIMAL(18, 6) ,
QTY_EOD DECIMAL(18, 6) ,
AMT_EOD DECIMAL(18,6) ,
UNIT_COST DECIMAL(18,8) ,
SUMCOST_SOLD DECIMAL(18,6) ,
GROSS_PROFIT DECIMAL(18, 6) ,
QTY_ALLOCATION DECIMAL(18,6) ,
AMT_ALLOCATION DECIMAL(18,2) ,
AMT_ALLOCATION_NO_TAX DECIMAL(18, 2) ,
GROSS_PROFIT_ALLOCATION DECIMAL(18,6) ,
SUMCOST_SOLD_ALLOCATION DECIMAL(18,6) ,
PRIMARYKEY(COM_ID,ITEM_ID,DATE1))
WITH"template=cache-partitioned,CACHE_NAME=PI_COM_DAY";
CREATEINDEX IDX_PI_COM_DAY_ITEM_DATE ONPI_COM_DAY(ITEM_ID,DATE1);
I don't think there's anything special about it.
Then we imported 10 million data using the COPY command.Data is
basically the actual production data, I think the dispersion is OK, not
artificial data with high similarity.
I would like to know if there are test results for the function of disk
compression? Most of the other memory databases also have the function
of data compression, but it doesn't look like it is now, or what's wrong
with me?
在 2020/8/28 上午12:39, Michael Cherkasov 写道:
Could you please share your benchmark code? I believe compression
might depend on data you write, if it full random, it's difficult to
compress the data.
On Wed, Aug 26, 2020, 8:26 PM 38797715 <[email protected]
<mailto:[email protected]>> wrote:
Hi,
We turn on disk compression to see the trend of execution time and
disk space.
Our expectation is that after disk compression is turned on,
although more CPU is used, the disk space is less occupied.
Because more data is written per unit time, the overall execution
time will be shortened in the case of insufficient memory.
However, it is found that the execution time and disk consumption
do not change significantly. We tested the
diskPageCompressionLevel values as 0, 10 and 17 respectively.
Our test method is as follows:
The ignite-compress module has been introduced.
The configuration of ignite is as follows:
<?xmlversion="1.0"encoding="UTF-8"?>
<beansxmlns="http://www.springframework.org/schema/beans";
<http://www.springframework.org/schema/beans>
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
<http://www.w3.org/2001/XMLSchema-instance>xsi:schemaLocation="http://www.springframework.org/schema/beans
<http://www.springframework.org/schema/beans>
http://www.springframework.org/schema/beans/spring-beans.xsd
<http://www.springframework.org/schema/beans/spring-beans.xsd>">
<beanid="ignite.cfg"class="org.apache.ignite.configuration.IgniteConfiguration">
<propertyname="peerClassLoadingEnabled"value="true"/>
<propertyname="consistentId"value="b"/>
<propertyname="igniteInstanceName"value="ClusterName1"/>
<propertyname="workDirectory"value="/home/ignite"/>
<propertyname="gridLogger">
<beanclass="org.apache.ignite.logger.log4j2.Log4J2Logger">
<constructor-argtype="java.lang.String"value="config/ignite-log4j2.xml"/>
</bean>
</property>
<propertyname="cacheConfiguration">
<list>
<beanid="partitioned-cache-template"abstract="true"class="org.apache.ignite.configuration.CacheConfiguration">
<propertyname="name"value="cache-partitioned*"/>
<propertyname="cacheMode"value="PARTITIONED"/>
<propertyname="queryParallelism"value="2"/>
<propertyname="diskPageCompression"value="LZ4"/>
<propertyname="diskPageCompressionLevel"value="17"/>
</bean>
</list>
</property>
<!-- Enabling Apache Ignite Persistent Store. -->
<propertyname="dataStorageConfiguration">
<beanclass="org.apache.ignite.configuration.DataStorageConfiguration">
<propertyname="pageSize"value="#{4096 * 2}"/>
<propertyname="defaultDataRegionConfiguration">
<beanclass="org.apache.ignite.configuration.DataRegionConfiguration">
<propertyname="persistenceEnabled"value="true"/>
<propertyname="maxSize"value="#{1L * 1024 * 1024 * 1024}"/>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
