Sailesh Mukil has posted comments on this change. Change subject: IMPALA-2550 Introduce query-wide execution context. ......................................................................
Patch Set 9: (3 comments) In this patch the following is addressed: - DescriptorTbl is not shared anymore. I've moved it back to RuntimeState as we still need to change how the serialization/deserialization is done before we can make it shared. It will be done in a separate patch. - QueryExecMgr now starts fragment threads so that refcount management stays in the scope of QueryExecMgr and is not passed to a lower level. - FragmentInstanceExecState refcount mechanism removed. Now these objects have the same lifetime as QueryExecState and are destroyed when QueryExecState is destroyed. - fragment_inst_exec_state_map_ is now indexed by fragment instance index and not fragment instance IDs. (Rebased on top of IMPALA-3988) http://gerrit.cloudera.org:8080/#/c/3817/7/be/src/service/query-exec-mgr.cc File be/src/service/query-exec-mgr.cc: Line 85: > how is this possible? it looks like an invariant that anything in the map w No that's not how it works. Everytime a new fragment comes in, FindOrCreateQueryExecState() is called (in RegisterFragmentInstanceWithQuery()) which increases the refcount by 1. Everytime a fragment completes (in QEM->ExecInstance()), ReleaseQES() is called which decrements the refcount by 1. This means that this refcount will ultimately reach 0. Alternatively CancelPlanFragment() and PublishFilter() use QESGuard() which calls a GetQES() at the beginning of its scope and a ReleaseQES() when it goes out of scope which will effectively balance the refcount. http://gerrit.cloudera.org:8080/#/c/3817/8/be/src/service/query-exec-mgr.cc File be/src/service/query-exec-mgr.cc: Line 54: QueryExecState* query_exec_state = FindOrCreateQueryExecState(exec_params.query_ctx); > comment seems redundant with the code (especially if you rename the method Done PS8, Line 100: > FindOrCreateQueryExecState() given that it creates (as opposed to inserting Done -- To view, visit http://gerrit.cloudera.org:8080/3817 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I892091d6401acb2ea91ccb1623af54c6f9635e6c Gerrit-PatchSet: 9 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Sailesh Mukil <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: David Knupp <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]> Gerrit-HasComments: Yes
