Zhiyuan Yang created HIVE-17641:
-----------------------------------
Summary: Visibility issue of Task.done cause Driver skip stages in
parallel execution
Key: HIVE-17641
URL: https://issues.apache.org/jira/browse/HIVE-17641
Project: Hive
Issue Type: Bug
Affects Versions: 1.2.1
Reporter: Zhiyuan Yang
Assignee: Zhiyuan Yang
Task.done is not volatile. In case of parallel execution, TaskRunner thread set
this value, and Driver thread read this value when it determines whether a
child task is runnable
DriverContext.java
{code}
public static boolean isLaunchable(Task<? extends Serializable> tsk) {
return !tsk.getQueued() && !tsk.getInitialized() && tsk.isRunnable();
{code}
Task.java
{code}
public boolean isRunnable() {
boolean isrunnable = true;
if (parentTasks != null) {
for (Task<? extends Serializable> parent : parentTasks) {
if (!parent.done()) {
{code}
This happens without any synchronization, so a child can be not runnable even
all parents finish.
To make it worse, Driver think query is successful when there is no running
task or runnable task, so query may finish without executing some stages.
Driver.java
{code}
while (!destroyed && driverCxt.isRunning()) {
{code}
DriverContext.java
{code}
public synchronized boolean isRunning() {
return !shutdown && (!running.isEmpty() || !runnable.isEmpty());
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
