----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33115/ -----------------------------------------------------------
(Updated April 15, 2015, 5:22 a.m.) Review request for drill, abdelhakim deneche, Chris Westin, and Steven Phillips. Changes ------- Updates to clarify FragmentExecutor and FragmentContext state. See updated description for explanation of changeset (as some things were removed from previous patch). Repository: drill-git Description (updated) ------- DRILL-2762: Update Fragment state reporting and error collection DeferredException - Add new throwAndClear operation on to allow checking for exceptions preClose in FragmentContext - Add new getAndClear operation BufferManager - Ensure close() can be called multiple times by clearing managed buffer list on close(). FragmentContext/FragmentExecutor - Update FragmentContext to have a preClose so that we can check closure state before doing final close. - Update so that there is only a single state maintained between FragmentContext and FragmentExecutor - Clean up FragmentExecutor run() method to better manage error states and have only single terminal point (avoiding multiple messages to Foreman). - Add new CANCELLATION_REQUESTED state for FragmentState. - Move all users of isCancelled or isFailed in main code to use shouldContinue() - Update receivingFragmentFinished message to not cancel fragment (only inform root operator of cancellation) WorkManager Updates - Add new afterExecute command to the WorkManager ExecutorService so that we get log entries if a thread leaks an exception. (Otherwise logs don't show these exceptions and they only go to standard out.) Foreman/QueryManager - Extract listenable interfaces into anonymous inner classes from body of Foreman QueryManager - Update QueryManager to track completed nodes rather than completed fragments using NodeTracker - Update DrillbitStatusListener to decrement expected completion messages on Nodes that have died to avoid query hang when a node dies FragmentData/MinorFragmentProfile - Add ability to track last status update as well as last time fragment made progress AbstractRecordBatch - Update awareness of current cancellation state to avoid cancellation delays Misc. Other changes - Move ByteCode optimization code to only record assembly and code as trace messages - Update SimpleRootExec to create fake ExecutorState to make existing tests work. - Update sort to exit prematurely in the case that the fragment was asked to cancel. - Add finals to all edited files. - Modify control handler and FragmentManager to directly support receivingFragmentFinished Diffs (updated) ----- common/src/main/java/org/apache/drill/common/DeferredException.java 99f18f1 exec/java-exec/src/main/java/org/apache/drill/exec/ops/BufferManager.java 2d22d84 exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java da2229c exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java 2bb29e5 exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java e2bcec3 exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 23ef0d3 exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/FragmentData.java 433ab26 exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java 8626d5b exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java a7e6c46 protocol/src/main/java/org/apache/drill/exec/proto/SchemaUserBitShared.java f72d5e1 protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java ac1bcbb protocol/src/main/java/org/apache/drill/exec/proto/beans/MinorFragmentProfile.java 5cd71f9 protocol/src/main/protobuf/UserBitShared.proto 2938114 Diff: https://reviews.apache.org/r/33115/diff/ Testing ------- Regression & Unit, more manual testing planned before final patch. Thanks, Jacques Nadeau
