[ https://issues.apache.org/jira/browse/MAPREDUCE-6736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ha, Hun Cheol updated MAPREDUCE-6736: ------------------------------------- Environment: Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) Hadoop 2.6.0-cdh5.7.0 was: Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) Hadoop 2.6.0-cdh5.7.0 Subversion http://github.com/cloudera/hadoop -r c00978c67b0d3fe9f3b896b5030741bd40bf541a Compiled by jenkins on 2016-03-23T18:36Z Compiled with protoc 2.5.0 >From source with checksum b2eabfa328e763c88cb14168f9b372 This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-cdh5.7.0.jar > import hive table from parquet files, there is no 'job.splitmetainfo' file > message > ---------------------------------------------------------------------------------- > > Key: MAPREDUCE-6736 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6736 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: applicationmaster, mrv2 > Affects Versions: 2.6.0 > Environment: Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) > Hadoop 2.6.0-cdh5.7.0 > Reporter: Ha, Hun Cheol > Priority: Blocker > > same issue : https://issues.apache.org/jira/browse/MAPREDUCE-3056 > which is created on 2011.09.21 and fixed 2011.10.04 > there is user(Sergey) who has same issue on 2015.05.13 too! (look last > comment of above link) > on beeline prompt, try import hive table from parquet files that are exported > from another hive table, there is no 'job.splitmetainfo' file on staging > directory : FileNotFoundException occurs > full log messages below > ================== > Log Type: syslog > Log Upload Time: Wed Jul 20 17:57:36 +0900 2016 > Log Length: 21439 > 2016-07-20 17:57:26,139 INFO [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for > application appattempt_1468834620182_0036_000001 > 2016-07-20 17:57:26,417 WARN [main] org.apache.hadoop.util.NativeCodeLoader: > Unable to load native-hadoop library for your platform... using builtin-java > classes where applicable > 2016-07-20 17:57:26,463 INFO [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens: > 2016-07-20 17:57:26,463 INFO [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, > Service: , Ident: > (org.apache.hadoop.yarn.security.AMRMTokenIdentifier@52af26ee) > 2016-07-20 17:57:26,510 INFO [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter. > 2016-07-20 17:57:26,991 WARN [main] > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: The short-circuit > local reads feature cannot be used because libhadoop cannot be loaded. > 2016-07-20 17:57:27,091 INFO [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config > null > 2016-07-20 17:57:27,154 INFO [main] > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output > Committer Algorithm version is 1 > 2016-07-20 17:57:27,159 INFO [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is > org.apache.hadoop.tools.mapred.CopyCommitter > 2016-07-20 17:57:27,231 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.jobhistory.EventType for class > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler > 2016-07-20 17:57:27,232 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher > 2016-07-20 17:57:27,233 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher > 2016-07-20 17:57:27,234 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher > 2016-07-20 17:57:27,235 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler > 2016-07-20 17:57:27,241 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher > 2016-07-20 17:57:27,242 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter > 2016-07-20 17:57:27,243 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for > class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter > 2016-07-20 17:57:27,292 INFO [main] > org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file > system [hdfs://da74:8020] > 2016-07-20 17:57:27,322 INFO [main] > org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file > system [hdfs://da74:8020] > 2016-07-20 17:57:27,352 INFO [main] > org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file > system [hdfs://da74:8020] > 2016-07-20 17:57:27,382 INFO [main] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Perms after > creating 448, Expected: 448 > 2016-07-20 17:57:27,387 INFO [main] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job > history data to the timeline server is not enabled > 2016-07-20 17:57:27,428 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler > 2016-07-20 17:57:27,645 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 2016-07-20 17:57:27,703 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period > at 10 second(s). > 2016-07-20 17:57:27,704 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system > started > 2016-07-20 17:57:27,712 INFO [main] > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for > job_1468834620182_0036 to jobTokenSecretManager > 2016-07-20 17:57:27,723 WARN [main] > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Job init failed > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > java.io.FileNotFoundException: File does not exist: > hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1579) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1443) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1401) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385) > at > org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) > at > org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) > at > org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:997) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:139) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1333) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1101) > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1544) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1540) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1473) > Caused by: java.io.FileNotFoundException: File does not exist: > hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo > at > org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1219) > at > org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1211) > at > org.apache.hadoop.mapreduce.split.SplitMetaInfoReader.readSplitMetaInfo(SplitMetaInfoReader.java:51) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1574) > ... 17 more > 2016-07-20 17:57:27,726 INFO [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, > non-uberized, multi-container job job_1468834620182_0036. > 2016-07-20 17:57:27,753 INFO [main] org.apache.hadoop.ipc.CallQueueManager: > Using callQueue class java.util.concurrent.LinkedBlockingQueue > 2016-07-20 17:57:27,761 INFO [Socket Reader #1 for port 35855] > org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 35855 > 2016-07-20 17:57:27,778 INFO [main] > org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding > protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server > 2016-07-20 17:57:27,779 INFO [IPC Server Responder] > org.apache.hadoop.ipc.Server: IPC Server Responder: starting > 2016-07-20 17:57:27,779 INFO [IPC Server listener on 35855] > org.apache.hadoop.ipc.Server: IPC Server listener on 35855: starting > 2016-07-20 17:57:27,780 INFO [main] > org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated > MRClientService at da74/115.68.67.98:35855 > 2016-07-20 17:57:27,837 INFO [main] org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2016-07-20 17:57:27,842 INFO [main] > org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable > to initialize FileSignerSecretProvider, falling back to use random secrets. > 2016-07-20 17:57:27,846 INFO [main] org.apache.hadoop.http.HttpRequestLog: > Http request log for http.requests.mapreduce is not defined > 2016-07-20 17:57:27,855 INFO [main] org.apache.hadoop.http.HttpServer2: Added > global filter 'safety' > (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) > 2016-07-20 17:57:27,860 INFO [main] org.apache.hadoop.http.HttpServer2: Added > filter AM_PROXY_FILTER > (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context > mapreduce > 2016-07-20 17:57:27,860 INFO [main] org.apache.hadoop.http.HttpServer2: Added > filter AM_PROXY_FILTER > (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context > static > 2016-07-20 17:57:27,862 INFO [main] org.apache.hadoop.http.HttpServer2: > adding path spec: /mapreduce/* > 2016-07-20 17:57:27,862 INFO [main] org.apache.hadoop.http.HttpServer2: > adding path spec: /ws/* > 2016-07-20 17:57:27,870 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty > bound to port 42206 > 2016-07-20 17:57:27,870 INFO [main] org.mortbay.log: jetty-6.1.26.cloudera.4 > 2016-07-20 17:57:27,899 INFO [main] org.mortbay.log: Extract > jar:file:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.6.0-cdh5.7.0.jar!/webapps/mapreduce > to /tmp/Jetty_0_0_0_0_42206_mapreduce____p5x30f/webapp > 2016-07-20 17:57:28,159 INFO [main] org.mortbay.log: Started > HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:42206 > 2016-07-20 17:57:28,160 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: > Web app /mapreduce started at 42206 > 2016-07-20 17:57:28,493 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: > Registered webapp guice modules > 2016-07-20 17:57:28,497 INFO [AsyncDispatcher event handler] > org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE > job_1468834620182_0036 > 2016-07-20 17:57:28,499 INFO [main] org.apache.hadoop.ipc.CallQueueManager: > Using callQueue class java.util.concurrent.LinkedBlockingQueue > 2016-07-20 17:57:28,499 INFO [Socket Reader #1 for port 41561] > org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 41561 > 2016-07-20 17:57:28,506 INFO [IPC Server listener on 41561] > org.apache.hadoop.ipc.Server: IPC Server listener on 41561: starting > 2016-07-20 17:57:28,502 INFO [IPC Server Responder] > org.apache.hadoop.ipc.Server: IPC Server Responder: starting > 2016-07-20 17:57:28,540 INFO [main] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: > nodeBlacklistingEnabled:true > 2016-07-20 17:57:28,540 INFO [main] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: > maxTaskFailuresPerNode is 3 > 2016-07-20 17:57:28,540 INFO [main] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: > blacklistDisablePercent is 33 > 2016-07-20 17:57:28,605 INFO [main] org.apache.hadoop.yarn.client.RMProxy: > Connecting to ResourceManager at da74/115.68.67.98:8030 > 2016-07-20 17:57:28,687 INFO [main] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: > maxContainerCapability: <memory:8192, vCores:4> > 2016-07-20 17:57:28,687 INFO [main] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: root.hdfs > 2016-07-20 17:57:28,692 INFO [main] > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper > limit on the thread pool size is 500 > 2016-07-20 17:57:28,692 INFO [main] > org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread > pool initial size is 10 > 2016-07-20 17:57:28,694 INFO [main] > org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: > yarn.client.max-cached-nodemanagers-proxies : 0 > 2016-07-20 17:57:28,699 INFO [main] > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: > job_1468834620182_0036Job Transitioned from NEW to FAIL_ABORT > 2016-07-20 17:57:28,700 INFO [CommitterEvent Processor #0] > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing > the event EventType: JOB_ABORT > 2016-07-20 17:57:28,727 INFO [CommitterEvent Processor #0] > org.apache.hadoop.tools.mapred.CopyCommitter: Cleaning up temporary work > folder: /user/hive/staging/hdfs/.staging/_distcp70301392 > 2016-07-20 17:57:28,758 INFO [AsyncDispatcher event handler] > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: > job_1468834620182_0036Job Transitioned from FAIL_ABORT to FAILED > 2016-07-20 17:57:28,759 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so > this is the last retry > 2016-07-20 17:57:28,759 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator > isAMLastRetry: true > 2016-07-20 17:57:28,759 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator > notified that shouldUnregistered is: true > 2016-07-20 17:57:28,760 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: > true > 2016-07-20 17:57:28,760 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: > JobHistoryEventHandler notified that forceJobCompletion is true > 2016-07-20 17:57:28,760 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the > services > 2016-07-20 17:57:28,760 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping > JobHistoryEventHandler. Size of the outstanding queue size is 3 > 2016-07-20 17:57:28,850 INFO [eventHandlingThread] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer > setup for JobId: job_1468834620182_0036, File: > hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job_1468834620182_0036_1.jhist > 2016-07-20 17:57:29,188 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, > writing event JOB_SUBMITTED > 2016-07-20 17:57:29,223 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file > system [hdfs://da74:8020] > 2016-07-20 17:57:29,253 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, > writing event JOB_QUEUE_CHANGED > 2016-07-20 17:57:29,253 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, > writing event JOB_FAILED > 2016-07-20 17:57:29,376 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying > hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job_1468834620182_0036_1.jhist > to > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist_tmp > 2016-07-20 17:57:29,513 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done > location: > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist_tmp > 2016-07-20 17:57:29,520 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying > hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job_1468834620182_0036_1_conf.xml > to > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml_tmp > 2016-07-20 17:57:29,561 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done > location: > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml_tmp > 2016-07-20 17:57:29,580 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to > done: > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036.summary_tmp > to > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036.summary > 2016-07-20 17:57:29,586 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to > done: > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml_tmp > to > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036_conf.xml > 2016-07-20 17:57:29,594 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to > done: > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist_tmp > to > hdfs://da74:8020/user/yarn/staging/history/done_intermediate/hdfs/job_1468834620182_0036-1469005044142-hdfs-distcp-1469005048754-0-0-FAILED-root.hdfs-1469005048754.jhist > 2016-07-20 17:57:29,594 INFO [Thread-55] > org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped > JobHistoryEventHandler. super.stop() > 2016-07-20 17:57:29,596 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job > diagnostics to Job init failed : > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > java.io.FileNotFoundException: File does not exist: > hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1579) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1443) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1401) > at > org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385) > at > org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) > at > org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) > at > org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:997) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:139) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1333) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1101) > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1544) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1540) > at > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1473) > Caused by: java.io.FileNotFoundException: File does not exist: > hdfs://da74:8020/user/yarn/staging/hdfs/.staging/job_1468834620182_0036/job.splitmetainfo > at > org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1219) > at > org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1211) > at > org.apache.hadoop.mapreduce.split.SplitMetaInfoReader.readSplitMetaInfo(SplitMetaInfoReader.java:51) > at > org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1574) > ... 17 more > 2016-07-20 17:57:29,597 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is > http://da74:19888/jobhistory/job/job_1468834620182_0036 > 2016-07-20 17:57:29,604 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for > application to be successfully unregistered. > 2016-07-20 17:57:30,605 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: > PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 > CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0 > 2016-07-20 17:57:30,607 INFO [Thread-55] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory > hdfs://da74:8020/ /user/hive/staging/hdfs/.staging/job_1468834620182_0036 > 2016-07-20 17:57:30,628 INFO [Thread-55] org.apache.hadoop.ipc.Server: > Stopping server on 41561 > 2016-07-20 17:57:30,629 INFO [IPC Server listener on 41561] > org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 41561 > 2016-07-20 17:57:30,629 INFO [IPC Server Responder] > org.apache.hadoop.ipc.Server: Stopping IPC Server Responder > 2016-07-20 17:57:30,629 INFO [TaskHeartbeatHandler PingChecker] > org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler > thread interrupted > 2016-07-20 17:57:30,629 INFO [Ping Checker] > org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: > TaskAttemptFinishingMonitor thread interrupted -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org