Hi, The fact that you can work with 29G cluster with only 8G of memory might be caused by the following things: 1) Your job doesn't use all data form cluster and hence caches only part of it. This is the most likely case. 2) You have eviction policy configured for IGFS data cache. 3) Or may be you use offheap. Please provide the full XML configuration and we will be able to understand it.
Anyways, your initial question was about out-of-memory. Could you provide exact error message? Is it about heap memory or may be permgen? As per execution time, this depends on your workload. If there are lots map tasks and very active work with data, you will see improvement in speed. If there are lots operations on file system (e.g. mkdirs, move, etc.) and very little amount of map jobs, chances there will be no speedup at all. Provide more details on the job you test and type of data you use and we will be able to give you more ideas on what to do. Vladimir. On Wed, Mar 16, 2016 at 10:53 AM, [email protected] <[email protected]> wrote: > Hi, > I have been present a problem in the forum,the link is: > http://apache-ignite-users.70518.x6.nabble.com/about-mr-accelerator-question-tc3502.html > I will follow you suggestions on the ignite cluster, but i am so sorry > that there are some confusions. > > on the single node,with 29G data, my ignite configrution is default > almost.just modify the ignite.sh > as follow: > if [ -z "$JVM_OPTS" ] ; then > JVM_OPTS="-Xms8g -Xmx8g -server -XX:+AggressiveOpts > -XX:MaxPermSize=8g" > fi > > and modify the default-config.xml > > <property name="fileSystemConfiguration"> > <list> > <bean > class="org.apache.ignite.configuration.FileSystemConfiguration" > parent="igfsCfgBase"> > <property name="name" value="igfs"/> > > > <property name="metaCacheName" value="igfs-meta"/> > <property name="dataCacheName" value="igfs-data"/> > > > <property name="ipcEndpointConfiguration"> > <bean > class="org.apache.ignite.igfs.IgfsIpcEndpointConfiguration"> > <property name="type" value="TCP" /> > <property name="host" value="127.0.0.1" /> > <property name="port" value="10500" /> > </bean> > </property> > > > <property name="secondaryFileSystem"> > <bean > class="org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem"> > <constructor-arg name="uri" > value="hdfs://localhost:9000"/> > <constructor-arg > name="cfgPath"><null/></constructor-arg> > <constructor-arg name="userName" > value="client-user-name"/> > </bean> > </property> > </bean> > </list> > </property> > > the result is very cool. > > > > the on-heap just 8G but the data have 29G.It work well.why? > > > > another problem is I start up another ignite node on the same server.I > have 16g heap with two ignite node.My job is finished after 7 minutes, the > effect is the same as on the single node.why the performance is not improve > when add the memory? > > thanks > ------------------------------ > >
