Hi, How I see from your logs, server node (which probably deployed by YARN) consumes 1Gb memory (per node). But client node consumes more memory how I see 4Gb. You can decrease memory consumption using -Xmx and -Xms jvm options for client nodes.
On Wed, Aug 24, 2016 at 5:28 PM, percent620 <[email protected]> wrote: > Here is my detailed steps > 1)root@sparkup1 config]# cat default-config.xml > <?xml version="1.0" encoding="UTF-8"?> > <beans xmlns="http://www.springframework.org/schema/beans"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:util="http://www.springframework.org/schema/util"; > xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans-4.1.xsd";> > <bean class="org.apache.ignite.configuration.IgniteConfiguration"> > <property name="discoverySpi"> > <bean > class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > <property name="ipFinder"> > <bean > class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm. > TcpDiscoveryVmIpFinder"> > <property name="addresses"> > <list> > > <value>172.16.186.200:47500..47509</value> > > <value>172.16.186.201:47500..47509</value> > > <value>172.16.186.202:47500..47509</value> > </list> > </property> > </bean> > </property> > </bean> > </property> > </bean> > </beans> > [root@sparkup1 config]# > > 2)yarn contains log as below > > i started 3 ignite node and for every is 1024G > > > $cat cluster.properties > # The number of nodes in the cluster. > IGNITE_NODE_COUNT=3 > # The number of CPU Cores for each Apache Ignite node. > IGNITE_RUN_CPU_PER_NODE=1 > # The number of Megabytes of RAM for each Apache Ignite node. > IGNITE_MEMORY_PER_NODE=1024 > # The version ignite which will be run on nodes. > IGNITE_VERSION=1.0.6 > IGNITE_WORK_DIR=/u01/yueyi/apache-ignite-hadoop-1.6.0-bin > IGNITE_XML_CONFIG=/ignite/releases/apache-ignite-hadoop- > 1.6.0-bin/config/default-config.xml > IGNITE_RELEASES_DIR=/ignite/releases/ > IGNITE_USERS_LIBS=/ignite/releases/apache-ignite-hadoop-1.6.0-bin/libs/ > #IGNITE_HOSTNAME_CONSTRAINT=vmsecdomain010194070026.cm10 > IGNITE_PATH=/ignite/releases/ > > > [root@sparkup3 config]# tail -f > /usr/hadoop-2.4.1/logs/userlogs/application_1472047995043_0001/container_ > 1472047995043_0001_01_000003/stdout > [07:13:45] Configured plugins: > [07:13:45] ^-- None > [07:13:45] > [07:13:46] Security status [authentication=off, tls/ssl=off] > [07:13:47] To start Console Management & Monitoring run > ignitevisorcmd.{sh|bat} > [07:13:47] > [07:13:47] Ignite node started OK (id=20fb73be) > [07:13:47] Topology snapshot [ver=1, servers=1, clients=0, CPUs=1, > heap=1.0GB] > [07:13:48] Topology snapshot [ver=2, servers=2, clients=0, CPUs=2, > heap=2.0GB] > [07:13:50] Topology snapshot [ver=3, servers=3, clients=0, CPUs=3, > heap=3.0GB] > ========================================== > Thanks is ok for above steps > > > > 3) > spark-submit *--driver-memory 4G* --class com.ignite.testIgniteSharedRDD > --master yarn --executor-cores 2 --executor-memory 1000m --num-executors 2 > --conf spark.rdd.compress=false --conf spark.shuffle.compress=false --conf > spark.broadcast.compress=false > /root/limu/ignite/spark-project-jar-with-dependencies.jar > > > 4)Yarn logs become is > [07:13:46] Security status [authentication=off, tls/ssl=off] > [07:13:47] To start Console Management & Monitoring run > ignitevisorcmd.{sh|bat} > [07:13:47] > [07:13:47] Ignite node started OK (id=20fb73be) > [07:13:47] Topology snapshot [ver=1, servers=1, clients=0, CPUs=1, > heap=1.0GB] > [07:13:48] Topology snapshot [ver=2, servers=2, clients=0, CPUs=2, > heap=2.0GB] > [07:13:50] Topology snapshot [ver=3, servers=3, clients=0, CPUs=3, > heap=3.0GB] > *[07:16:54] Topology snapshot [ver=4, servers=3, clients=1, CPUs=3, > heap=7.0GB] correct > > * > /[07:17:06] Topology snapshot [ver=5, servers=4, clients=1, CPUs=3, > heap=8.0GB] > [07:17:07] Topology snapshot [ver=6, servers=5, clients=1, CPUs=3, > heap=9.0GB]/ > > is not correct, why become 5 servers and 9GB memory > > > > details log > nohup: ignoring input > 16/08/24 07:16:17 INFO spark.SparkContext: Running Spark version 1.6.1 > 16/08/24 07:16:18 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 16/08/24 07:16:18 INFO spark.SecurityManager: Changing view acls to: root > 16/08/24 07:16:18 INFO spark.SecurityManager: Changing modify acls to: root > 16/08/24 07:16:18 INFO spark.SecurityManager: SecurityManager: > authentication disabled; ui acls disabled; users with view permissions: > Set(root); users with modify permissions: Set(root) > 16/08/24 07:16:19 INFO util.Utils: Successfully started service > 'sparkDriver' on port 56368. > 16/08/24 07:16:20 INFO slf4j.Slf4jLogger: Slf4jLogger started > 16/08/24 07:16:20 INFO Remoting: Starting remoting > 16/08/24 07:16:20 INFO Remoting: Remoting started; listening on addresses > :[akka.tcp://[email protected]:45375] > 16/08/24 07:16:20 INFO util.Utils: Successfully started service > 'sparkDriverActorSystem' on port 45375. > 16/08/24 07:16:20 INFO spark.SparkEnv: Registering MapOutputTracker > 16/08/24 07:16:20 INFO spark.SparkEnv: Registering BlockManagerMaster > 16/08/24 07:16:20 INFO storage.DiskBlockManager: Created local directory at > /tmp/blockmgr-8739bc15-9e06-4496-91fc-beea794a57d7 > 16/08/24 07:16:20 INFO storage.MemoryStore: MemoryStore started with > capacity 2.7 GB > 16/08/24 07:16:21 INFO spark.SparkEnv: Registering OutputCommitCoordinator > 16/08/24 07:16:21 INFO server.Server: jetty-8.y.z-SNAPSHOT > 16/08/24 07:16:21 INFO server.AbstractConnector: Started > [email protected]:4040 > 16/08/24 07:16:21 INFO util.Utils: Successfully started service 'SparkUI' > on > port 4040. > 16/08/24 07:16:21 INFO ui.SparkUI: Started SparkUI at > http://172.16.186.200:4040 > 16/08/24 07:16:21 INFO spark.HttpFileServer: HTTP File server directory is > /tmp/spark-07a3f373-5d80-42b0-8544-33583e9fff7d/httpd- > 4d4119d2-ac26-4f49-90a9-03b88e24f9d9 > 16/08/24 07:16:21 INFO spark.HttpServer: Starting HTTP Server > 16/08/24 07:16:21 INFO server.Server: jetty-8.y.z-SNAPSHOT > 16/08/24 07:16:21 INFO server.AbstractConnector: Started > [email protected]:38541 > 16/08/24 07:16:21 INFO util.Utils: Successfully started service 'HTTP file > server' on port 38541. > 16/08/24 07:16:22 INFO spark.SparkContext: Added JAR > file:/root/limu/ignite/spark-project-jar-with-dependencies.jar at > http://172.16.186.200:38541/jars/spark-project-jar-with-dependencies.jar > with timestamp 1472048182938 > 16/08/24 07:16:23 INFO client.RMProxy: Connecting to ResourceManager at > sparkup1/172.16.186.200:8032 > 16/08/24 07:16:23 INFO yarn.Client: Requesting a new application from > cluster with 3 NodeManagers > 16/08/24 07:16:23 INFO yarn.Client: Verifying our application has not > requested more than the maximum memory capability of the cluster (8192 MB > per container) > 16/08/24 07:16:23 INFO yarn.Client: Will allocate AM container, with 896 MB > memory including 384 MB overhead > 16/08/24 07:16:23 INFO yarn.Client: Setting up container launch context for > our AM > 16/08/24 07:16:23 INFO yarn.Client: Setting up the launch environment for > our AM container > 16/08/24 07:16:23 INFO yarn.Client: Preparing resources for our AM > container > 16/08/24 07:16:24 INFO yarn.Client: Uploading resource > file:/usr/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly- > 1.6.1-hadoop2.6.0.jar > -> > hdfs://sparkup1:9000/user/root/.sparkStaging/application_1472047995043_ > 0002/spark-assembly-1.6.1-hadoop2.6.0.jar > 16/08/24 07:16:31 INFO yarn.Client: Uploading resource > file:/tmp/spark-07a3f373-5d80-42b0-8544-33583e9fff7d/__spark_conf__ > 9140885938827716798.zip > -> > hdfs://sparkup1:9000/user/root/.sparkStaging/application_1472047995043_ > 0002/__spark_conf__9140885938827716798.zip > 16/08/24 07:16:31 INFO spark.SecurityManager: Changing view acls to: root > 16/08/24 07:16:31 INFO spark.SecurityManager: Changing modify acls to: root > 16/08/24 07:16:31 INFO spark.SecurityManager: SecurityManager: > authentication disabled; ui acls disabled; users with view permissions: > Set(root); users with modify permissions: Set(root) > 16/08/24 07:16:31 INFO yarn.Client: Submitting application 2 to > ResourceManager > 16/08/24 07:16:31 INFO impl.YarnClientImpl: Submitted application > application_1472047995043_0002 > 16/08/24 07:16:32 INFO yarn.Client: Application report for > application_1472047995043_0002 (state: ACCEPTED) > 16/08/24 07:16:32 INFO yarn.Client: > client token: N/A > diagnostics: N/A > ApplicationMaster host: N/A > ApplicationMaster RPC port: -1 > queue: default > start time: 1472048191273 > final status: UNDEFINED > tracking URL: > http://sparkup1:8088/proxy/application_1472047995043_0002/ > user: root > 16/08/24 07:16:33 INFO yarn.Client: Application report for > application_1472047995043_0002 (state: ACCEPTED) > 16/08/24 07:16:34 INFO yarn.Client: Application report for > application_1472047995043_0002 (state: ACCEPTED) > 16/08/24 07:16:35 INFO yarn.Client: Application report for > application_1472047995043_0002 (state: ACCEPTED) > 16/08/24 07:16:36 INFO yarn.Client: Application report for > application_1472047995043_0002 (state: ACCEPTED) > 16/08/24 07:16:37 INFO yarn.Client: Application report for > application_1472047995043_0002 (state: ACCEPTED) > 16/08/24 07:16:38 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: > ApplicationMaster registered as NettyRpcEndpointRef(null) > 16/08/24 07:16:38 INFO cluster.YarnClientSchedulerBackend: Add WebUI > Filter. > org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, > Map(PROXY_HOSTS > -> sparkup1, PROXY_URI_BASES -> > http://sparkup1:8088/proxy/application_1472047995043_0002), > /proxy/application_1472047995043_0002 > 16/08/24 07:16:38 INFO ui.JettyUtils: Adding filter: > org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter > 16/08/24 07:16:38 INFO yarn.Client: Application report for > application_1472047995043_0002 (state: RUNNING) > 16/08/24 07:16:38 INFO yarn.Client: > client token: N/A > diagnostics: N/A > ApplicationMaster host: 172.16.186.201 > ApplicationMaster RPC port: 0 > queue: default > start time: 1472048191273 > final status: UNDEFINED > tracking URL: > http://sparkup1:8088/proxy/application_1472047995043_0002/ > user: root > 16/08/24 07:16:38 INFO cluster.YarnClientSchedulerBackend: Application > application_1472047995043_0002 has started running. > 16/08/24 07:16:38 INFO util.Utils: Successfully started service > 'org.apache.spark.network.netty.NettyBlockTransferService' on port 41610. > 16/08/24 07:16:38 INFO netty.NettyBlockTransferService: Server created on > 41610 > 16/08/24 07:16:38 INFO storage.BlockManagerMaster: Trying to register > BlockManager > 16/08/24 07:16:38 INFO storage.BlockManagerMasterEndpoint: Registering > block > manager 172.16.186.200:41610 with 2.7 GB RAM, BlockManagerId(driver, > 172.16.186.200, 41610) > 16/08/24 07:16:38 INFO storage.BlockManagerMaster: Registered BlockManager > 16/08/24 07:16:46 INFO cluster.YarnClientSchedulerBackend: Registered > executor NettyRpcEndpointRef(null) (sparkup2:56516) with ID 1 > 16/08/24 07:16:46 INFO storage.BlockManagerMasterEndpoint: Registering > block > manager sparkup2:43778 with 500.0 MB RAM, BlockManagerId(1, sparkup2, > 43778) > 16/08/24 07:16:47 INFO cluster.YarnClientSchedulerBackend: Registered > executor NettyRpcEndpointRef(null) (sparkup1:46131) with ID 2 > 16/08/24 07:16:47 INFO cluster.YarnClientSchedulerBackend: > SchedulerBackend > is ready for scheduling beginning after reached > minRegisteredResourcesRatio: > 0.8 > 16/08/24 07:16:47 INFO storage.BlockManagerMasterEndpoint: Registering > block > manager sparkup1:35739 with 500.0 MB RAM, BlockManagerId(2, sparkup1, > 35739) > 16/08/24 07:16:48 INFO xml.XmlBeanDefinitionReader: Loading XML bean > definitions from URL > [file:/usr/apache-ignite-fabric-1.6.0-bin/config/default-config.xml] > 16/08/24 07:16:52 INFO support.GenericApplicationContext: Refreshing > org.springframework.context.support.GenericApplicationContext@467048e: > startup date [Wed Aug 24 07:16:52 PDT 2016]; root of context hierarchy > 16/08/24 07:16:53 INFO internal.IgniteKernal: > > >>> __________ ________________ > >>> / _/ ___/ |/ / _/_ __/ __/ > >>> _/ // (7 7 // / / / / _/ > >>> /___/\___/_/|_/___/ /_/ /___/ > >>> > >>> ver. 1.6.0#20160518-sha1:0b22c45b > >>> 2016 Copyright(C) Apache Software Foundation > >>> > >>> Ignite documentation: http://ignite.apache.org > > 16/08/24 07:16:53 INFO internal.IgniteKernal: Config URL: n/a > 16/08/24 07:16:53 INFO internal.IgniteKernal: Daemon mode: off > 16/08/24 07:16:53 INFO internal.IgniteKernal: OS: Linux > 2.6.32-431.el6.x86_64 amd64 > 16/08/24 07:16:53 INFO internal.IgniteKernal: OS user: root > 16/08/24 07:16:53 INFO internal.IgniteKernal: Language runtime: Java > Platform API Specification ver. 1.7 > 16/08/24 07:16:53 INFO internal.IgniteKernal: VM information: Java(TM) SE > Runtime Environment 1.7.0_71-b14 Oracle Corporation Java HotSpot(TM) 64-Bit > Server VM 24.71-b01 > 16/08/24 07:16:53 INFO internal.IgniteKernal: VM total memory: 3.9GB > 16/08/24 07:16:53 INFO internal.IgniteKernal: Remote Management [restart: > off, REST: on, JMX (remote: off)] > 16/08/24 07:16:53 INFO internal.IgniteKernal: > IGNITE_HOME=/usr/apache-ignite-fabric-1.6.0-bin > 16/08/24 07:16:53 INFO internal.IgniteKernal: VM arguments: [-Xms4G, > -Xmx4G, > -XX:MaxPermSize=256m] > 16/08/24 07:16:53 INFO internal.IgniteKernal: Configured caches > ['ignite-marshaller-sys-cache', 'ignite-sys-cache', > 'ignite-atomics-sys-cache'] > 16/08/24 07:16:53 INFO internal.IgniteKernal: 3-rd party licenses can be > found at: /usr/apache-ignite-fabric-1.6.0-bin/libs/licenses > 16/08/24 07:16:53 INFO internal.IgniteKernal: Non-loopback local IPs: > 172.16.186.200, fe80:0:0:0:20c:29ff:fecd:2f25%2 > 16/08/24 07:16:53 INFO internal.IgniteKernal: Enabled local MACs: > 000C29CD2F25 > 16/08/24 07:16:53 INFO plugin.IgnitePluginProcessor: Configured plugins: > 16/08/24 07:16:53 INFO plugin.IgnitePluginProcessor: ^-- None > 16/08/24 07:16:53 INFO plugin.IgnitePluginProcessor: > 16/08/24 07:16:53 INFO tcp.TcpCommunicationSpi: IPC shared memory server > endpoint started [port=48101, > tokDir=/usr/apache-ignite-fabric-1.6.0-bin/work/ipc/ > shmem/529a5fe1-5f95-4aeb-a0dc-a8f814f63369-25100] > 16/08/24 07:16:53 INFO tcp.TcpCommunicationSpi: Successfully bound shared > memory communication to TCP port [port=48101, locHost=0.0.0.0/0.0.0.0] > 16/08/24 07:16:53 INFO tcp.TcpCommunicationSpi: Successfully bound to TCP > port [port=47101, locHost=0.0.0.0/0.0.0.0] > 16/08/24 07:16:53 WARN noop.NoopCheckpointSpi: Checkpoints are disabled (to > enable configure any GridCheckpointSpi implementation) > 16/08/24 07:16:53 WARN collision.GridCollisionManager: Collision > resolution > is disabled (all jobs will be activated upon arrival). > 16/08/24 07:16:53 WARN noop.NoopSwapSpaceSpi: Swap space is disabled. To > enable use FileSwapSpaceSpi. > 16/08/24 07:16:53 INFO internal.IgniteKernal: Security status > [authentication=off, tls/ssl=off] > 16/08/24 07:16:54 INFO tcp.GridTcpRestProtocol: Command protocol > successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11212] > 16/08/24 07:16:54 WARN internal.IgniteKernal: Attempting to start more > nodes > than physical RAM available on current host (this can cause significant > slowdown) > 16/08/24 07:16:54 INFO cache.GridCacheProcessor: Started cache > [name=ignite-marshaller-sys-cache, mode=REPLICATED] > 16/08/24 07:16:54 INFO cache.GridCacheProcessor: Started cache > [name=ignite-atomics-sys-cache, mode=PARTITIONED] > 16/08/24 07:16:54 INFO cache.GridCacheProcessor: Started cache > [name=ignite-sys-cache, mode=REPLICATED] > 16/08/24 07:16:55 INFO internal.IgniteKernal: To start Console Management & > Monitoring run ignitevisorcmd.{sh|bat} > 16/08/24 07:16:55 INFO internal.IgniteKernal: > 16/08/24 07:16:55 INFO internal.IgniteKernal: > > >>> +----------------------------------------------------------- > -----------+ > >>> Ignite ver. 1.6.0#20160518-sha1:0b22c45bb9b97692208fd0705ddf80 > 45ff34a031 > >>> +----------------------------------------------------------- > -----------+ > >>> OS name: Linux 2.6.32-431.el6.x86_64 amd64 > >>> CPU(s): 1 > >>> Heap: 4.0GB > >>> VM name: 25100@sparkup1 > >>> Local node [ID=529A5FE1-5F95-4AEB-A0DC-A8F814F63369, order=4, > >>> clientMode=true] > >>> Local node addresses: [sparkup1/0:0:0:0:0:0:0:1%1, /127.0.0.1, > >>> /172.16.186.200] > >>> Local ports: TCP:11212 TCP:47101 TCP:48101 > > 16/08/24 07:16:55 INFO discovery.GridDiscoveryManager: Topology snapshot > [ver=4, servers=3, clients=1, CPUs=3, heap=7.0GB] > 16/08/24 07:16:56 INFO cache.GridCacheProcessor: Started cache > [name=sharedIgniteRDD-ling-sha111o, mode=PARTITIONED] > 16/08/24 07:16:57 INFO spark.SparkContext: Starting job: count at > testIgniteSharedRDD.scala:19 > 16/08/24 07:16:57 INFO scheduler.DAGScheduler: Got job 0 (count at > testIgniteSharedRDD.scala:19) with 1024 output partitions > 16/08/24 07:16:57 INFO scheduler.DAGScheduler: Final stage: ResultStage 0 > (count at testIgniteSharedRDD.scala:19) > 16/08/24 07:16:57 INFO scheduler.DAGScheduler: Parents of final stage: > List() > 16/08/24 07:16:57 INFO scheduler.DAGScheduler: Missing parents: List() > 16/08/24 07:16:57 INFO scheduler.DAGScheduler: Submitting ResultStage 0 > (IgniteRDD[0] at RDD at IgniteAbstractRDD.scala:31), which has no missing > parents > 16/08/24 07:16:57 INFO storage.MemoryStore: Block broadcast_0 stored as > values in memory (estimated size 1872.0 B, free 1872.0 B) > 16/08/24 07:16:57 INFO storage.MemoryStore: Block broadcast_0_piece0 stored > as bytes in memory (estimated size 1853.0 B, free 3.6 KB) > 16/08/24 07:16:57 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 > in > memory on 172.16.186.200:41610 (size: 1853.0 B, free: 2.7 GB) > 16/08/24 07:16:57 INFO spark.SparkContext: Created broadcast 0 from > broadcast at DAGScheduler.scala:1006 > 16/08/24 07:16:57 INFO scheduler.DAGScheduler: Submitting 1024 missing > tasks > from ResultStage 0 (IgniteRDD[0] at RDD at IgniteAbstractRDD.scala:31) > 16/08/24 07:16:57 INFO cluster.YarnScheduler: Adding task set 0.0 with 1024 > tasks > 16/08/24 07:16:57 INFO scheduler.TaskSetManager: Starting task 7.0 in stage > 0.0 (TID 0, sparkup2, partition 7,NODE_LOCAL, 1967 bytes) > 16/08/24 07:16:57 INFO scheduler.TaskSetManager: Starting task 1.0 in stage > 0.0 (TID 1, sparkup1, partition 1,NODE_LOCAL, 1967 bytes) > 16/08/24 07:16:57 INFO scheduler.TaskSetManager: Starting task 8.0 in stage > 0.0 (TID 2, sparkup2, partition 8,NODE_LOCAL, 1967 bytes) > 16/08/24 07:16:57 INFO scheduler.TaskSetManager: Starting task 3.0 in stage > 0.0 (TID 3, sparkup1, partition 3,NODE_LOCAL, 1967 bytes) > 16/08/24 07:17:00 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 > in > memory on sparkup1:35739 (size: 1853.0 B, free: 500.0 MB) > 16/08/24 07:17:01 INFO storage.BlockManagerInfo: Added broadcast_0_piece0 > in > memory on sparkup2:43778 (size: 1853.0 B, free: 500.0 MB) > [07:17:04] New version is available at ignite.apache.org: 1.7.0 > 16/08/24 07:17:04 WARN cluster.GridUpdateNotifier: New version is available > at ignite.apache.org: 1.7.0 > *16/08/24 07:17:06 INFO discovery.GridDiscoveryManager: Added new node to > topology: TcpDiscoveryNode [id=18575d1b-d764-43ca-9fcb-2f130b159557, > addrs=[0:0:0:0:0:0:0:1%1, 127.0.0.1, 172.16.186.201], > sockAddrs=[sparkup2/172.16.186.201:47501, /0:0:0:0:0:0:0:1%1:47501, > /127.0.0.1:47501, /172.16.186.201:47501], discPort=47501, order=5, > intOrder=5, lastExchangeTime=1472048225977, loc=false, > ver=1.6.0#20160518-sha1:0b22c45b, isClient=false] > 16/08/24 07:17:06 INFO discovery.GridDiscoveryManager: Topology snapshot > [ver=5, servers=4, clients=1, CPUs=3, heap=8.0GB] > 16/08/24 07:17:07 INFO discovery.GridDiscoveryManager: Added new node to > topology: TcpDiscoveryNode [id=a8fbf4cd-7fe3-40d6-ad64-b1c53479785f, > addrs=[0:0:0:0:0:0:0:1%1, 127.0.0.1, 172.16.186.200], > sockAddrs=[sparkup1/172.16.186.200:47501, /0:0:0:0:0:0:0:1%1:47501, > /127.0.0.1:47501, /172.16.186.200:47501], discPort=47501, order=6, > intOrder=6, lastExchangeTime=1472048226549, loc=false, > ver=1.6.0#20160518-sha1:0b22c45b, isClient=false] > 16/08/24 07:17:07 INFO discovery.GridDiscoveryManager: Topology snapshot > [ver=6, servers=5, clients=1, CPUs=3, heap=9.0GB]* > > *why local port 127.0.0.1 join yarn ignite topology, is this correct? I > think that this is not correct. > * > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Starting task 17.0 in > stage > 0.0 (TID 4, sparkup2, partition 17,NODE_LOCAL, 1967 bytes) > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Starting task 18.0 in > stage > 0.0 (TID 5, sparkup2, partition 18,NODE_LOCAL, 1967 bytes) > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Finished task 8.0 in stage > 0.0 (TID 2) in 13294 ms on sparkup2 (1/1024) > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Finished task 7.0 in stage > 0.0 (TID 0) in 13364 ms on sparkup2 (2/1024) > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Starting task 4.0 in stage > 0.0 (TID 6, sparkup1, partition 4,NODE_LOCAL, 1967 bytes) > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Starting task 6.0 in stage > 0.0 (TID 7, sparkup1, partition 6,NODE_LOCAL, 1967 bytes) > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Finished task 1.0 in stage > 0.0 (TID 1) in 13763 ms on sparkup1 (3/1024) > 16/08/24 07:17:11 INFO scheduler.TaskSetManager: Finished task 3.0 in stage > 0.0 (TID 3) in 13759 ms on sparkup1 (4/1024) > 16/08/24 07:17:12 INFO scheduler.TaskSetManager: Starting task 20.0 in > stage > 0.0 (TID 8, sparkup2, partition 20,NODE_LOCAL, > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Yarn-deployment-for-memory- > capacity-make-a-bigger-than-before-Urgent-tp7275.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
