One interesting message here , *No plan file found: * 15/11/01 23:55:36 INFO exec.Utilities: No plan file found: hdfs:// sandbox.hortonworks.com:8020/tmp/hive/root/119652ff-3158-4cce-b32d-b300bfead1bc/hive_2015-11-01_23-54-47_767_5715642849033319370-1/-mr-10003/40878ced-7985-40d9-9b1d-27f06acb1bef/map.xml <https://contactmonkey.com/api/v1/tracker?cm_session=027d853f-0e03-41b4-ad1e-a017b45dcb74&cm_type=link&cm_link=3fd1ebf2-8ac7-4140-a457-8f9e0dd4cb69&cm_destination=http://sandbox.hortonworks.com:8020/tmp/hive/root/119652ff-3158-4cce-b32d-b300bfead1bc/hive_2015-11-01_23-54-47_767_5715642849033319370-1/-mr-10003/40878ced-7985-40d9-9b1d-27f06acb1bef/map.xml>
Similar error message was here https://issues.apache.org/jira/browse/HIVE-7210 <https://contactmonkey.com/api/v1/tracker?cm_session=027d853f-0e03-41b4-ad1e-a017b45dcb74&cm_type=link&cm_link=fe471ebf-79d4-48cf-a5fc-7b724d5ac331&cm_destination=https://issues.apache.org/jira/browse/HIVE-7210>
[root@sandbox ~]# yarn logs -applicationId application_1446420063056_0009 15/11/02 01:27:03 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/ 15/11/02 01:27:03 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.15:8050 15/11/02 01:27:04 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library 15/11/02 01:27:04 INFO compress.CodecPool: Got brand-new decompressor [.deflate] Container: container_e24_1446420063056_0009_01_000002 on sandbox.hortonworks.com_45454 ======================================================================================== LogType:stderr Log Upload Time:Mon Nov 02 00:19:42 +0000 2015 LogLength:16487 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/yarn/local/usercache/root/filecache/20/spark-assembly-1.3.1.2.3.0.0-2557-hadoop2.7.1.2.3.0.0-2557.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.3.0.0-2557/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 15/11/01 23:55:14 INFO executor.CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT] 15/11/01 23:55:15 INFO spark.SecurityManager: Changing view acls to: yarn,root 15/11/01 23:55:15 INFO spark.SecurityManager: Changing modify acls to: yarn,root 15/11/01 23:55:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, root); users with modify permissions: Set(yarn, root) 15/11/01 23:55:15 INFO slf4j.Slf4jLogger: Slf4jLogger started 15/11/01 23:55:15 INFO Remoting: Starting remoting 15/11/01 23:55:15 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:50249] 15/11/01 23:55:15 INFO util.Utils: Successfully started service 'driverPropsFetcher' on port 50249. 15/11/01 23:55:16 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon. 15/11/01 23:55:16 INFO spark.SecurityManager: Changing view acls to: yarn,root 15/11/01 23:55:16 INFO spark.SecurityManager: Changing modify acls to: yarn,root 15/11/01 23:55:16 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, root); users with modify permissions: Set(yarn, root) 15/11/01 23:55:16 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports. 15/11/01 23:55:16 INFO slf4j.Slf4jLogger: Slf4jLogger started 15/11/01 23:55:16 INFO Remoting: Starting remoting 15/11/01 23:55:16 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down. 15/11/01 23:55:16 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:47798] 15/11/01 23:55:16 INFO util.Utils: Successfully started service 'sparkExecutor' on port 47798. 15/11/01 23:55:16 INFO util.AkkaUtils: Connecting to MapOutputTracker: akka.tcp://[email protected]:34108/user/MapOutputTracker 15/11/01 23:55:16 INFO util.AkkaUtils: Connecting to BlockManagerMaster: akka.tcp://[email protected]:34108/user/BlockManagerMaster 15/11/01 23:55:16 INFO storage.DiskBlockManager: Created local directory at /hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/blockmgr-46d0d58f-72e8-4820-890b-92091f9f2e0b 15/11/01 23:55:16 INFO storage.MemoryStore: MemoryStore started with capacity 132.8 MB 15/11/01 23:55:16 INFO util.AkkaUtils: Connecting to OutputCommitCoordinator: akka.tcp://[email protected]:34108/user/OutputCommitCoordinator 15/11/01 23:55:16 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: akka.tcp://[email protected]:34108/user/CoarseGrainedScheduler 15/11/01 23:55:16 INFO executor.CoarseGrainedExecutorBackend: Successfully registered with driver 15/11/01 23:55:16 INFO executor.Executor: Starting executor ID 1 on host sandbox.hortonworks.com 15/11/01 23:55:16 INFO netty.NettyBlockTransferService: Server created on 45010 15/11/01 23:55:16 INFO storage.BlockManagerMaster: Trying to register BlockManager 15/11/01 23:55:16 INFO storage.BlockManagerMaster: Registered BlockManager 15/11/01 23:55:16 INFO util.AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://[email protected]:34108/user/HeartbeatReceiver 15/11/01 23:55:35 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 0 15/11/01 23:55:35 INFO executor.Executor: Running task 0.0 in stage 0.0 (TID 0) 15/11/01 23:55:35 INFO executor.Executor: Fetching http://10.0.2.15:36949/jars/hive-hcatalog-core.jar with timestamp 1446422133376 15/11/01 23:55:35 INFO util.Utils: Fetching http://10.0.2.15:36949/jars/hive-hcatalog-core.jar to /hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/fetchFileTemp7123832568053136399.tmp 15/11/01 23:55:35 INFO util.Utils: Copying /hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/-4947024171446422133376_cache to /hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/container_e24_1446420063056_0009_01_000002/./hive-hcatalog-core.jar 15/11/01 23:55:35 INFO executor.Executor: Adding file:/hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/container_e24_1446420063056_0009_01_000002/./hive-hcatalog-core.jar to class loader 15/11/01 23:55:35 INFO executor.Executor: Fetching http://10.0.2.15:36949/jars/hive-exec-1.2.1.2.3.0.0-2557.jar with timestamp 1446422133353 15/11/01 23:55:35 INFO util.Utils: Fetching http://10.0.2.15:36949/jars/hive-exec-1.2.1.2.3.0.0-2557.jar to /hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/fetchFileTemp1820438727975347160.tmp 15/11/01 23:55:35 INFO util.Utils: Copying /hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/20116347171446422133353_cache to /hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/container_e24_1446420063056_0009_01_000002/./hive-exec-1.2.1.2.3.0.0-2557.jar 15/11/01 23:55:35 INFO executor.Executor: Adding file:/hadoop/yarn/local/usercache/root/appcache/application_1446420063056_0009/container_e24_1446420063056_0009_01_000002/./hive-exec-1.2.1.2.3.0.0-2557.jar to class loader 15/11/01 23:55:35 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 1 15/11/01 23:55:35 INFO storage.MemoryStore: ensureFreeSpace(53475) called with curMem=0, maxMem=139292835 15/11/01 23:55:35 INFO storage.MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 52.2 KB, free 132.8 MB) 15/11/01 23:55:35 INFO storage.BlockManagerMaster: Updated info of block broadcast_1_piece0 15/11/01 23:55:35 INFO broadcast.TorrentBroadcast: Reading broadcast variable 1 took 187 ms 15/11/01 23:55:35 INFO storage.MemoryStore: ensureFreeSpace(164728) called with curMem=53475, maxMem=139292835 15/11/01 23:55:35 INFO storage.MemoryStore: Block broadcast_1 stored as values in memory (estimated size 160.9 KB, free 132.6 MB) 15/11/01 23:55:35 INFO rdd.HadoopRDD: Input split: Paths:/apps/hive/warehouse/sample_07/sample_07:0+46055InputFormatClass: org.apache.hadoop.mapred.TextInputFormat 15/11/01 23:55:35 INFO broadcast.TorrentBroadcast: Started reading broadcast variable 0 15/11/01 23:55:35 INFO storage.MemoryStore: ensureFreeSpace(49944) called with curMem=218203, maxMem=139292835 15/11/01 23:55:35 INFO storage.MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 48.8 KB, free 132.6 MB) 15/11/01 23:55:35 INFO storage.BlockManagerMaster: Updated info of block broadcast_0_piece0 15/11/01 23:55:35 INFO broadcast.TorrentBroadcast: Reading broadcast variable 0 took 15 ms 15/11/01 23:55:36 INFO storage.MemoryStore: ensureFreeSpace(695756) called with curMem=268147, maxMem=139292835 15/11/01 23:55:36 INFO storage.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 679.4 KB, free 131.9 MB) 15/11/01 23:55:36 INFO exec.Utilities: No plan file found: hdfs://sandbox.hortonworks.com:8020/tmp/hive/root/119652ff-3158-4cce-b32d-b300bfead1bc/hive_2015-11-01_23-54-47_767_5715642849033319370-1/-mr-10003/40878ced-7985-40d9-9b1d-27f06acb1bef/map.xml 15/11/01 23:55:36 ERROR executor.Executor: Exception in task 0.0 in stage 0.0 (TID 0) java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587) at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:64) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 15/11/01 23:55:36 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 1 15/11/01 23:55:36 INFO executor.Executor: Running task 0.1 in stage 0.0 (TID 1) 15/11/01 23:55:36 INFO rdd.HadoopRDD: Input split: Paths:/apps/hive/warehouse/sample_07/sample_07:0+46055InputFormatClass: org.apache.hadoop.mapred.TextInputFormat 15/11/01 23:55:36 INFO exec.Utilities: No plan file found: hdfs://sandbox.hortonworks.com:8020/tmp/hive/root/119652ff-3158-4cce-b32d-b300bfead1bc/hive_2015-11-01_23-54-47_767_5715642849033319370-1/-mr-10003/40878ced-7985-40d9-9b1d-27f06acb1bef/map.xml 15/11/01 23:55:36 ERROR executor.Executor: Exception in task 0.1 in stage 0.0 (TID 1) java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587) at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:64) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 15/11/01 23:55:36 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 2 15/11/01 23:55:36 INFO executor.Executor: Running task 0.2 in stage 0.0 (TID 2) 15/11/01 23:55:36 INFO rdd.HadoopRDD: Input split: Paths:/apps/hive/warehouse/sample_07/sample_07:0+46055InputFormatClass: org.apache.hadoop.mapred.TextInputFormat 15/11/01 23:55:36 INFO exec.Utilities: No plan file found: hdfs://sandbox.hortonworks.com:8020/tmp/hive/root/119652ff-3158-4cce-b32d-b300bfead1bc/hive_2015-11-01_23-54-47_767_5715642849033319370-1/-mr-10003/40878ced-7985-40d9-9b1d-27f06acb1bef/map.xml 15/11/01 23:55:36 ERROR executor.Executor: Exception in task 0.2 in stage 0.0 (TID 2) java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587) at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:64) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 15/11/01 23:55:36 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 3 15/11/01 23:55:36 INFO executor.Executor: Running task 0.3 in stage 0.0 (TID 3) 15/11/01 23:55:36 INFO rdd.HadoopRDD: Input split: Paths:/apps/hive/warehouse/sample_07/sample_07:0+46055InputFormatClass: org.apache.hadoop.mapred.TextInputFormat 15/11/01 23:55:36 INFO exec.Utilities: No plan file found: hdfs://sandbox.hortonworks.com:8020/tmp/hive/root/119652ff-3158-4cce-b32d-b300bfead1bc/hive_2015-11-01_23-54-47_767_5715642849033319370-1/-mr-10003/40878ced-7985-40d9-9b1d-27f06acb1bef/map.xml 15/11/01 23:55:36 ERROR executor.Executor: Exception in task 0.3 in stage 0.0 (TID 3) java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437) at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587) at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:236) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:212) at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) at org.apache.spark.scheduler.Task.run(Task.scala:64) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 15/11/02 00:07:48 INFO storage.BlockManager: Removing broadcast 1 15/11/02 00:07:48 INFO storage.BlockManager: Removing block broadcast_1_piece0 15/11/02 00:07:48 INFO storage.MemoryStore: Block broadcast_1_piece0 of size 53475 dropped from memory (free 138382407) 15/11/02 00:07:48 INFO storage.BlockManagerMaster: Updated info of block broadcast_1_piece0 15/11/02 00:07:48 INFO storage.BlockManager: Removing block broadcast_1 15/11/02 00:07:48 INFO storage.MemoryStore: Block broadcast_1 of size 164728 dropped from memory (free 138547135) 15/11/02 00:19:41 ERROR executor.CoarseGrainedExecutorBackend: Driver Disassociated [akka.tcp://[email protected]:47798] -> [akka.tcp://[email protected]:34108] disassociated! Shutting down. 15/11/02 00:19:41 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://[email protected]:34108] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. End of LogType:stderr LogType:stdout Log Upload Time:Mon Nov 02 00:19:42 +0000 2015 LogLength:0 Log Contents: End of LogType:stdout Container: container_e24_1446420063056_0009_01_000001 on sandbox.hortonworks.com_45454 ======================================================================================== LogType:stderr Log Upload Time:Mon Nov 02 00:19:42 +0000 2015 LogLength:7515 Log Contents: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/hadoop/yarn/local/usercache/root/filecache/20/spark-assembly-1.3.1.2.3.0.0-2557-hadoop2.7.1.2.3.0.0-2557.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/hdp/2.3.0.0-2557/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 15/11/01 23:55:07 INFO yarn.ApplicationMaster: Registered signal handlers for [TERM, HUP, INT] 15/11/01 23:55:07 INFO yarn.ApplicationMaster: ApplicationAttemptId: appattempt_1446420063056_0009_000001 15/11/01 23:55:08 INFO spark.SecurityManager: Changing view acls to: yarn,root 15/11/01 23:55:08 INFO spark.SecurityManager: Changing modify acls to: yarn,root 15/11/01 23:55:08 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, root); users with modify permissions: Set(yarn, root) 15/11/01 23:55:08 INFO slf4j.Slf4jLogger: Slf4jLogger started 15/11/01 23:55:08 INFO Remoting: Starting remoting 15/11/01 23:55:08 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:44389] 15/11/01 23:55:08 INFO util.Utils: Successfully started service 'sparkYarnAM' on port 44389. 15/11/01 23:55:08 INFO yarn.ApplicationMaster: Waiting for Spark driver to be reachable. 15/11/01 23:55:08 INFO yarn.ApplicationMaster: Driver now available: sandbox.hortonworks.com:34108 15/11/01 23:55:08 INFO yarn.ApplicationMaster: Listen to driver: akka.tcp://[email protected]:34108/user/YarnScheduler 15/11/01 23:55:08 INFO yarn.ApplicationMaster: Add WebUI Filter. AddWebUIFilter(org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter,Map(PROXY_HOSTS -> sandbox.hortonworks.com, PROXY_URI_BASES -> http://sandbox.hortonworks.com:8088/proxy/application_1446420063056_0009),/proxy/application_1446420063056_0009) 15/11/01 23:55:08 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.15:8030 15/11/01 23:55:08 INFO yarn.YarnRMClient: Registering the ApplicationMaster 15/11/01 23:55:09 INFO yarn.YarnAllocator: Will request 2 executor containers, each with 1 cores and 640 MB memory including 384 MB overhead 15/11/01 23:55:09 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:640, vCores:1>) 15/11/01 23:55:09 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:640, vCores:1>) 15/11/01 23:55:09 INFO yarn.ApplicationMaster: Started progress reporter thread - sleep time : 5000 15/11/01 23:55:14 INFO impl.AMRMClientImpl: Received new token for : sandbox.hortonworks.com:45454 15/11/01 23:55:14 INFO yarn.YarnAllocator: Launching container container_e24_1446420063056_0009_01_000002 for on host sandbox.hortonworks.com 15/11/01 23:55:14 INFO yarn.YarnAllocator: Launching ExecutorRunnable. driverUrl: akka.tcp://[email protected]:34108/user/CoarseGrainedScheduler, executorHostname: sandbox.hortonworks.com 15/11/01 23:55:14 INFO yarn.YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them. 15/11/01 23:55:14 INFO yarn.ExecutorRunnable: Starting Executor Container 15/11/01 23:55:14 INFO impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0 15/11/01 23:55:14 INFO yarn.ExecutorRunnable: Setting up ContainerLaunchContext 15/11/01 23:55:14 INFO yarn.ExecutorRunnable: Preparing Local resources 15/11/01 23:55:14 INFO yarn.ExecutorRunnable: Prepared Local resources Map(__spark__.jar -> resource { scheme: "hdfs" host: "sandbox.hortonworks.com" port: 8020 file: "/user/root/.sparkStaging/application_1446420063056_0009/spark-assembly-1.3.1.2.3.0.0-2557-hadoop2.7.1.2.3.0.0-2557.jar" } size: 169244777 timestamp: 1446422105626 type: FILE visibility: PRIVATE) 15/11/01 23:55:14 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>/usr/hdp/current/hadoop-client/*<CPS>/usr/hdp/current/hadoop-client/lib/*<CPS>/usr/hdp/current/hadoop-hdfs-client/*<CPS>/usr/hdp/current/hadoop-hdfs-client/lib/*<CPS>/usr/hdp/current/hadoop-yarn-client/*<CPS>/usr/hdp/current/hadoop-yarn-client/lib/*<CPS>$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/2.3.0.0-2557/hadoop/lib/hadoop-lzo-0.6.0.2.3.0.0-2557.jar:/etc/hadoop/conf/secure, SPARK_LOG_URL_STDERR -> http://sandbox.hortonworks.com:8042/node/containerlogs/container_e24_1446420063056_0009_01_000002/root/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1446420063056_0009, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 169244777, SPARK_USER -> root, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS -> 1446422105626, SPARK_LOG_URL_STDOUT -> http://sandbox.hortonworks.com:8042/node/containerlogs/container_e24_1446420063056_0009_01_000002/root/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://sandbox.hortonworks.com:8020/user/root/.sparkStaging/application_1446420063056_0009/spark-assembly-1.3.1.2.3.0.0-2557-hadoop2.7.1.2.3.0.0-2557.jar#__spark__.jar) 15/11/01 23:55:14 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms256m, -Xmx256m, '-Dhive.spark.log.dir=/usr/hdp/2.3.0.0-2557/spark/logs/', -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.driver.port=34108', -Dspark.yarn.app.container.log.dir=<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://[email protected]:34108/user/CoarseGrainedScheduler, --executor-id, 1, --hostname, sandbox.hortonworks.com, --cores, 1, --app-id, application_1446420063056_0009, --user-class-path, file:$PWD/__app__.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr) 15/11/01 23:55:14 INFO impl.ContainerManagementProtocolProxy: Opening proxy : sandbox.hortonworks.com:45454 15/11/02 00:19:41 INFO yarn.ApplicationMaster: Driver terminated or disconnected! Shutting down. Disassociated [akka.tcp://[email protected]:44389] -> [akka.tcp://[email protected]:34108] 15/11/02 00:19:41 INFO yarn.ApplicationMaster: Final app status: SUCCEEDED, exitCode: 0 15/11/02 00:19:41 WARN remote.ReliableDeliverySupervisor: Association with remote system [akka.tcp://[email protected]:34108] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. 15/11/02 00:19:41 INFO yarn.ApplicationMaster: Unregistering ApplicationMaster with SUCCEEDED 15/11/02 00:19:41 INFO yarn.ApplicationMaster: Driver terminated or disconnected! Shutting down. Disassociated [akka.tcp://[email protected]:44389] -> [akka.tcp://[email protected]:34108] 15/11/02 00:19:41 INFO impl.AMRMClientImpl: Waiting for application to be successfully unregistered. 15/11/02 00:19:41 INFO yarn.ApplicationMaster: Deleting staging directory .sparkStaging/application_1446420063056_0009 End of LogType:stderr LogType:stdout Log Upload Time:Mon Nov 02 00:19:42 +0000 2015 LogLength:0 Log Contents: End of LogType:stdout [root@sandbox ~]#
