Hello!

Since we don't seem to have much experience in running YARN here, can you
please elaborate on this problem? Is the number of consumed resources wrong
or are resources actually get consumed by Ignite?

BTW, if you plan to store data inside Ignite, you should also set
IGNITE_MEMORY_OVERHEAD_PER_NODE, see discussion at
http://apache-ignite-developers.2346864.n4.nabble.com/YARN-parameters-and-default-values-td29899.html

Regards,

-- 
Ilya Kasnacheev

2018-05-17 14:31 GMT+03:00 Murugar <gnithianant...@gmail.com>:

> Hi,
>
> Ignite-yarn[Ignition] uses almost all memory and vcores available in YARN
> cluster. The allocated memory to the Ignition server was 40GB and 16
> VCores.
>
> *When we checked the actual usage with yarn top command the Ignition server
> is using 380 GB and 140 VCores!!! * When we checked the cache, it is having
> only less number of records - 3000 records.
>
> *Information about the application:*
> We load data into Ignite Cache, process them periodically and delete
> processed records periodically.
>
> We are running Ignite-yarn in Cloudera cluster with 5 data nodes with below
> cluster.properties
>
> IGNITE_NODE_COUNT=4
> IGNITE_RUN_CPU_PER_NODE=4
> IGNITE_MEMORY_PER_NODE=10096
> IGNITE_VERSION=2.3.0
> IGNITE_PATH=/apache-ignite-fabric-2.3.0-bin.zip
>
> IGNITE_RELEASES_DIR=/releases
> IGNITE_WORKING_DIR=/work
> IGNITE_XML_CONFIG=/ignite-config.xml
> IGNITE_USERS_LIBS=/libs
> IGNITE_LOCAL_WORK_DIR=/home/user
> IGNITE_JVM_OPTS="-server -Xms10g -Xmx10g -XX:+AlwaysPreTouch -XX:+UseG1GC
> -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC"
>
>
> Please find our Ignite-config.xml below
>
> <?xml version="1.0" encoding="UTF-8"?>
> <beans xmlns="http://www.springframework.org/schema/beans";
>            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>            xmlns:util="http://www.springframework.org/schema/util";
>            xsi:schemaLocation="
>         http://www.springframework.org/schema/beans
>         http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
>         http://www.springframework.org/schema/util
>         http://www.springframework.org/schema/util/spring-util-2.0.xsd";>
> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
>                 <property name="clientMode" value="false"/>
>                 <property name="peerClassLoadingEnabled" value="true"/>
>                 <property name="networkTimeout" value="100000"/>
>                 <property name="networkSendRetryCount" value="5"/>
>                 <property name="discoverySpi">
>             <bean
> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>                 <property name="ipFinder">
>                     <bean
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.
> TcpDiscoveryVmIpFinder">
>                         <property name="addresses">
>                           <list>
>
> <value>ipaddress:47500</value>
>                             </list>
>                         </property>
>                     </bean>
>                 </property>
>                                 <property name="networkTimeout"
> value="10000000"/>
>                                 <property name="joinTimeout"
> value="10000000"/>
>                                 <property name="maxAckTimeout"
> value="10000000"/>
>                                 <property name="reconnectCount"
> value="50"/>
>                                 <property name="socketTimeout"
> value="10000000"/>
>             </bean>
>         </property>
>         </bean>
> </beans>
>
> Cache Configuration:
>
> var userGridConf: CacheConfiguration[Long, User] = new
> CacheConfiguration[Long, User](userCacheName)
>    userGridConf.setCacheMode(CacheMode.REPLICATED)
>     userGridConf.setEagerTtl(false)
>     userGridConf.setOnheapCacheEnabled(false)
>     userGridConf.setBackups(0)
>     userGridConf.setIndexedTypes(classOf[Long], classOf[User])
> val cache = ignite.getOrCreateCache(userGridConf)
> cache.size(CachePeekMode.OFFHEAP)
>
> Could you please let me know what is wrong with the above configurations?
>
> Thanks in Advance!!!
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Reply via email to