Hey,
Guys is this normal…? So many binary readers are opened without reuse? I saw an improvement issue on this https://jira.apache.org/jira/browse/IGNITE-5721 I'm running a simple compute tasks that split around 4K ids to compute job that each one takes from cache bulk of 500 ids. Each Object size return is ~1.8MB. When I run 3-4 iterations it is super fasts completing the tasks under 100 ms. But then memory rises and it becomes really slow, fully occupied by GC and after a while it caves and get OOME. I'm evaluating ignite for a POC and this is a real showstopper. I'm sure that I'm missing something because I do nothing out of the ordinary. I tried different GC tuning G1 and CMS, different evection policies, different region sizes. Nothing helped. I'm using the latest ignite 2.7, my machine is windows server 12 R2 with 8 cores and 32GB RAM, this is my node config: <beans xmlns="http://www.springframework.org/schema/beans"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xmlns:context="http://www.springframework.org/schema/context"; xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd "> <context:annotation-config/> <context:property-placeholder location="classpath:application.properties"/> <bean id="similarityConfiguration" class="com.cla.filecluster.service.files.pv.SimilarityConfiguration"/> <bean id="similarityCalculator" class="com.cla.filecluster.service.files.pv.SimilarityCalculator" factory-method="create"> <constructor-arg ref="similarityConfiguration" /> </bean> <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"> <property name="userAttributes"> <map key-type="java.lang.String" value-type="java.lang.Boolean"> <entry key="data.node" value="true"/> </map> </property> <property name="binaryConfiguration"> <bean class="org.apache.ignite.configuration.BinaryConfiguration"> <property name="compactFooter" value="false"/> </bean> </property> <!-- property name="asyncCallbackPoolSize" value="2"/ --> <property name="dataStorageConfiguration"> <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> <property name="pageSize" value="#{4 * 1024}"/> <property name="defaultDataRegionConfiguration"> <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> <property name="name" value="analysis_region"/> <property name="persistenceEnabled" value="true"/> <property name="initialSize" value="#{100 * 1024 * 1024}"/> <property name="maxSize" value="8589934592"/> </bean> </property> <property name="storagePath" value="E:\asm-repo\db"/> <property name="walPath" value="E:\asm-repo\wal"/> <property name="walArchivePath" value="E:\asm-repo\wal\archive"/> </bean> </property> <property name="cacheConfiguration"> <list> <!-- Partitioned cache example configuration (Atomic mode). --> <bean class="org.apache.ignite.configuration.CacheConfiguration"> <property name="storeKeepBinary" value="true"/> <property name="eagerTtl" value="true"/> <property name="name" value="PvAnalysisData"/> <property name="dataRegionName" value="analysis_region"/> <property name="atomicityMode" value="ATOMIC"/> <property name="backups" value="0"/> <property name="onheapCacheEnabled" value="true"/> <property name="expiryPolicyFactory"> <bean class="javax.cache.expiry.CreatedExpiryPolicy" factory-method="factoryOf"> <constructor-arg> <bean class="javax.cache.expiry.Duration"> <constructor-arg value="SECONDS"/> <constructor-arg value="5"/> </bean> </constructor-arg> </bean> </property> <property name="evictionPolicy"> <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy"> <!-- property name="maxMemorySize" value="104857600"/ --> <property name="maxSize" value="1000"/> </bean> </property> <property name="nodeFilter"> <bean class="com.cla.filecluster.common.DataNodeFilter"/> </property> </bean> </list> </property> <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. --> <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="ipFinder"> <!-- Ignite provides several options for automatic discovery that can be used instead os static IP based discovery. For information on all options refer to our documentation: http://apacheignite.readme.io/docs/cluster-config --> <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. --> <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">--> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder"> <property name="addresses"> <list> <!-- In distributed environment, replace with actual host IP address. --> <value>127.0.0.1:47500..47509</value> </list> </property> </bean> </property> </bean> </property> </bean> </beans> Thanks, Nadav System Architect +972-544821606 <http://www.docauthority.com/> DocAuthority.com
