----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33115/ -----------------------------------------------------------
(Updated April 15, 2015, 7 p.m.) Review request for drill, abdelhakim deneche, Chris Westin, and Steven Phillips. Changes ------- Actually update patch Repository: drill-git Description ------- DRILL-2762: Update Fragment state reporting and error collection DeferredException - Add new throwAndClear operation on to allow checking for exceptions preClose in FragmentContext - Add new getAndClear operation BufferManager - Ensure close() can be called multiple times by clearing managed buffer list on close(). FragmentContext/FragmentExecutor - Update FragmentContext to have a preClose so that we can check closure state before doing final close. - Update so that there is only a single state maintained between FragmentContext and FragmentExecutor - Clean up FragmentExecutor run() method to better manage error states and have only single terminal point (avoiding multiple messages to Foreman). - Add new CANCELLATION_REQUESTED state for FragmentState. - Move all users of isCancelled or isFailed in main code to use shouldContinue() - Update receivingFragmentFinished message to not cancel fragment (only inform root operator of cancellation) WorkManager Updates - Add new afterExecute command to the WorkManager ExecutorService so that we get log entries if a thread leaks an exception. (Otherwise logs don't show these exceptions and they only go to standard out.) Foreman/QueryManager - Extract listenable interfaces into anonymous inner classes from body of Foreman QueryManager - Update QueryManager to track completed nodes rather than completed fragments using NodeTracker - Update DrillbitStatusListener to decrement expected completion messages on Nodes that have died to avoid query hang when a node dies FragmentData/MinorFragmentProfile - Add ability to track last status update as well as last time fragment made progress AbstractRecordBatch - Update awareness of current cancellation state to avoid cancellation delays Misc. Other changes - Move ByteCode optimization code to only record assembly and code as trace messages - Update SimpleRootExec to create fake ExecutorState to make existing tests work. - Update sort to exit prematurely in the case that the fragment was asked to cancel. - Add finals to all edited files. - Modify control handler and FragmentManager to directly support receivingFragmentFinished Diffs (updated) ----- common/src/main/java/org/apache/drill/common/DeferredException.java 99f18f1 exec/java-exec/src/main/java/org/apache/drill/exec/compile/AsmUtil.java 81904df exec/java-exec/src/main/java/org/apache/drill/exec/compile/bytecode/InstructionModifier.java 6c0292e exec/java-exec/src/main/java/org/apache/drill/exec/compile/bytecode/ScalarReplacementNode.java 6e981bc exec/java-exec/src/main/java/org/apache/drill/exec/ops/BufferManager.java 2d22d84 exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java da2229c exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/BaseRootExec.java 5b7ca66 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java e230fd2 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java c50cb8a exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java 4b317e0 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java 389d668 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java a23780e exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java 94bc3a3 exec/java-exec/src/main/java/org/apache/drill/exec/proto/helper/QueryIdHelper.java 1aadaa2 exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java 2bb29e5 exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java c15bb7c exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java e2bcec3 exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ControlHandlerImpl.java 3a7123d exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 2430e64 exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 23ef0d3 exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/FragmentData.java 433ab26 exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java 8626d5b exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/AbstractStatusReporter.java 1b0885d exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java a7e6c46 exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentManager.java 7a819c4 exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/NonRootFragmentManager.java 41e87cd exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java 84071c3 exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/StateTransitionException.java 7155d43 exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/StatusReporter.java 26b5d68 exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/SimpleRootExec.java 933417e protocol/src/main/java/org/apache/drill/exec/proto/SchemaUserBitShared.java f72d5e1 protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java 96a921b protocol/src/main/java/org/apache/drill/exec/proto/beans/FragmentState.java ba536fc protocol/src/main/java/org/apache/drill/exec/proto/beans/MinorFragmentProfile.java 5cd71f9 protocol/src/main/protobuf/UserBitShared.proto 2938114 Diff: https://reviews.apache.org/r/33115/diff/ Testing ------- Regression & Unit, more manual testing planned before final patch. Thanks, Jacques Nadeau
