[
https://issues.apache.org/jira/browse/HIVE-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-15889:
------------------------------------
Attachment: HIVE-15889.3.patch
Thanks [~sershe]. Attaching .3 patch to address review comments. Used
setAccessible and also caching it statically.
Reflection is used since there is an option to have tez optionally in earlier
clusters. Not sure this is still valid given it is the default engine now. But
didn't want to change that code for backward compatibility.
> LLAP: Some tasks still run after hive cli is shutdown
> -----------------------------------------------------
>
> Key: HIVE-15889
> URL: https://issues.apache.org/jira/browse/HIVE-15889
> Project: Hive
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Attachments: HIVE-15889.1.patch, HIVE-15889.2.patch,
> HIVE-15889.3.patch
>
>
> E.g: In cross product case, the tight loop in merge join operator ignores any
> interrupt or abort flag checks, causing the tasks to be in running state even
> when the client cli is quit.
> intensionally written cross product query to simulate this.
> {noformat}
> hive> select count(1) from lineitem, orders;;
> {noformat}
> Even when the cli is quit, LLAP would continue executing the task for quite
> sometime. E.g stack trace
> {noformat}
> "TezTaskRunner" #1945 daemon prio=5 os_prio=0 tid=0x00007fe9e43a5000
> nid=0x4c8 runnable [0x00007fc8d881b000]
> java.lang.Thread.State: RUNNABLE
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:453)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.mergeJoinComputeKeys(CommonMergeJoinOperator.java:603)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:207)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:351)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:282)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:410)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchNextGroup(CommonMergeJoinOperator.java:381)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.doFirstFetchIfNeeded(CommonMergeJoinOperator.java:491)
> at
> org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:209)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:351)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:282)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:319)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
