Zhiyuan Yang created HIVE-17641: ----------------------------------- Summary: Visibility issue of Task.done cause Driver skip stages in parallel execution Key: HIVE-17641 URL: https://issues.apache.org/jira/browse/HIVE-17641 Project: Hive Issue Type: Bug Affects Versions: 1.2.1 Reporter: Zhiyuan Yang Assignee: Zhiyuan Yang
Task.done is not volatile. In case of parallel execution, TaskRunner thread set this value, and Driver thread read this value when it determines whether a child task is runnable DriverContext.java {code} public static boolean isLaunchable(Task<? extends Serializable> tsk) { return !tsk.getQueued() && !tsk.getInitialized() && tsk.isRunnable(); {code} Task.java {code} public boolean isRunnable() { boolean isrunnable = true; if (parentTasks != null) { for (Task<? extends Serializable> parent : parentTasks) { if (!parent.done()) { {code} This happens without any synchronization, so a child can be not runnable even all parents finish. To make it worse, Driver think query is successful when there is no running task or runnable task, so query may finish without executing some stages. Driver.java {code} while (!destroyed && driverCxt.isRunning()) { {code} DriverContext.java {code} public synchronized boolean isRunning() { return !shutdown && (!running.isEmpty() || !runnable.isEmpty()); {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)