[jira] [Commented] (HIVE-11651) Index creation is failing from beeline when execution engine is set to Tez

[Venkata Srinivasa Rao Kolla (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-11651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712892#comment-14712892
 ] 

Venkata Srinivasa Rao Kolla  commented on HIVE-11651:
-

Also tried to run the same set of scripts on Hive 1.2 + Tez 0.7, i saw some 
errors in the job log. 

2015-08-26 09:48:31,178 INFO [Dispatcher thread: Central] 
history.HistoryEventHandler: 
[HISTORY][DAG:dag_1440569395691_0214_1][Event:DAG_FINISHED]: 
dagId=dag_1440569395691_0214_1, startTime=1440582480900, 
finishTime=1440582511086, timeTaken=30186, status=FAILED, diagnostics=Vertex 
failed, vertexName=Map 1, vertexId=vertex_1440569395691_0214_1_00, 
diagnostics=[Task failed, taskId=task_1440569395691_0214_1_00_00, 
diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running 
task:java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:192)
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.init(TezGroupedSplitsInputFormat.java:131)
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:97)
at 
org.apache.tez.mapreduce.lib.MRReaderMapred.setupOldRecordReader(MRReaderMapred.java:149)
at 
org.apache.tez.mapreduce.lib.MRReaderMapred.setSplit(MRReaderMapred.java:80)
at 
org.apache.tez.mapreduce.input.MRInput.initFromEventInternal(MRInput.java:614)
at 
org.apache.tez.mapreduce.input.MRInput.initFromEvent(MRInput.java:593)
at 
org.apache.tez.mapreduce.input.MRInputLegacy.checkAndAwaitRecordReaderInitialization(MRInputLegacy.java:141)
at 
org.apache.tez.mapreduce.input.MRInputLegacy.init(MRInputLegacy.java:109)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.getMRInput(MapRecordProcessor.java:370)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:127)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:147)
... 14 more
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:270)
at 
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:234)
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:189)
... 25 more
], TaskAttempt 1 failed, info=[Error: Failure while running 
task:java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 

[jira] [Commented] (HIVE-11651) Index creation is failing from beeline when execution engine is set to Tez

[Venkata Srinivasa Rao Kolla (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-11651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712891#comment-14712891
 ] 

Venkata Srinivasa Rao Kolla  commented on HIVE-11651:
-

I have tried this on Hive 0.13 version and Tez 0.4

 Index creation is failing from beeline when execution engine is set to Tez
 --

 Key: HIVE-11651
 URL: https://issues.apache.org/jira/browse/HIVE-11651
 Project: Hive
  Issue Type: Bug
  Components: Beeline, Hive, Tez
Reporter: Venkata Srinivasa Rao Kolla 

 In Hive, with execution engine is set to Tez and when we tried to rebuild 
 index on a table (which is not empty) using beeline, index creation is not 
 happening. It is not throwing any error too on to console. But in the 
 corresponding tez job logs we found below error:
 2015-08-24 18:08:43,999 WARN [AMShutdownThread] 
 org.apache.tez.dag.history.recovery.RecoveryService: Error when closing 
 summary stream 
 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
  No lease on 
 /tmp/hive-alti-test-01/_tez_session_dir/573df26e-4a22-4bf5-ad5c-a0d72cdacec6/application_1439396922313_1732/recovery/1/application_1439396922313_1732.summary:
  File does not exist. Holder DFSClient_NONMAPREDUCE_-117450461_1 does not 
 have any open files. at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2956)
  at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3027)
  at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3007)
  at 
 org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:641)
  at 
 org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:484)
  at 
 org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
  at 
 org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928) at 
 org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013) at 
 org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009) at 
 java.security.AccessController.doPrivileged(Native Method) at 
 javax.security.auth.Subject.doAs(Subject.java:415) at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
  at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
 Below are the set of commands we have used: CREATE TABLE table02(column1 
 String, column2 string, column3 int, column4 string) ROW FORMAT DELIMITED 
 FIELDS TERMINATED BY ','; LOAD DATA LOCAL INPATH 'posts_us' OVERWRITE INTO 
 TABLE table02; CREATE INDEX table02_index ON TABLE table02 (column3) AS 
 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler' WITH DEFERRED 
 REBUILD; ALTER INDEX table02_index ON table02 REBUILD;
 Does any one seen this problem?
 Regards, Srinivas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)