My guess for this issue is that the MR input initialize does not wait for all inputs to be ready which can trigger occasional NPE. If I can get a local repro, I will create a patch for this and confirm if its the actual issue.
Thanks Prasanth > On Jun 30, 2015, at 4:15 PM, Prasanth Jayachandran > <[email protected]> wrote: > > Hi > > Few fixes went in after hive 0.14 release. Is it possible to try in the > recently released 1.2.1 version? Also can you send yarn application logs to > see whats going on. > > Thanks > Prasanth > >> On Jun 30, 2015, at 1:44 AM, patcharee <[email protected]> wrote: >> >> Hi, >> >> I am using hive 0.14 + tez 0.5. It fails to alter table concatenate >> occasionally (see the exception below). It is strange that it fails from >> time to time not predictable. However, it works on mr. >> Is there any suggestion/clue? >> >> hive> alter table 4dim partition(zone=2,z=15,year=2005,month=4) CONCATENATE; >> -------------------------------------------------------------------------------- >> >> VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED >> -------------------------------------------------------------------------------- >> >> File Merge FAILED -1 0 0 -1 0 0 >> -------------------------------------------------------------------------------- >> >> VERTICES: 00/01 [>>--------------------------] 0% ELAPSED TIME: >> 1435651968.00 s >> -------------------------------------------------------------------------------- >> >> Status: Failed >> Vertex failed, vertexName=File Merge, >> vertexId=vertex_1435307579867_0041_1_00, diagnostics=[Vertex >> vertex_1435307579867_0041_1_00 [File Merge] killed/failed due >> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: >> [hdfs://service-10-0.local:8020/apps/hive/warehouse/wrf_tables/4dim/zone=2/z=15/year=2005/month=4] >> initializer failed, vertex=vertex_1435307579867_0041_1_00 [File Merge], >> java.lang.NullPointerException >> at >> org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:265) >> at >> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:452) >> at >> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:441) >> at >> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:295) >> at >> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:124) >> at >> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245) >> at >> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) >> at >> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239) >> at >> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226) >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> ] >> DAG failed due to vertex failure. failedVertices:1 killedVertices:0 >> FAILED: Execution Error, return code 2 from >> org.apache.hadoop.hive.ql.exec.DDLTask >> >> BR, >> Patcharee >
