Matthew Jacobs has posted comments on this change. Change subject: IMPALA-2550 Introduce query-wide execution context. ......................................................................
Patch Set 7: (9 comments) http://gerrit.cloudera.org:8080/#/c/3817/7//COMMIT_MSG Commit Message: PS7, Line 17: passes this fragment along to the QueryExecState which fragment? PS7, Line 26: Once scheduled for destruction, a thread in the QueryExecMgr will : destroy the QueryExecState. Why do we need a special thread for this? PS7, Line 28: Every user of the QueryExecState must access it within the scope of : ScopedQueryExecStateRef which guarantees ref count incrementing and : decrementing at entry/exit of the scope. It looks like we're implementing something that looks like a shared_ptr. I know we don't like shared_ptrs, but what we're doing looks a bit error prone. Also, I'm wondering if we can't avoid reference counting to begin with, if the ownership and lifecycle of the QES is more explicitly defined from the beginning. I don't see enough context to reason about that though. http://gerrit.cloudera.org:8080/#/c/3817/7/be/src/service/query-exec-mgr.h File be/src/service/query-exec-mgr.h: PS7, Line 54: and that the fragment instance must either run to completion : /// or be cancelled or it can encounter an error? I assume this is executed on another thread? PS7, Line 60: RegisterFragmentInstanceWithQuery How about just 'RegisterFragment'? PS7, Line 68: QueryExecState* GetQueryExecState(const TUniqueId& query_id); : : /// Decrements query_exec_state->num_current_references and schedules it for destruction : /// if the reference count hits zero. : void ReleaseQueryExecState(QueryExecState* query_exec_state); Who are the consumers of these APIs? I'm wondering if ref counting is really necessary. Maybe I'll be able to answer this as I read more through the code. PS7, Line 74: right query's PS7, Line 74: a PS7, Line 127: /// The thread pool that is in charge of destroying a QueryExecState when it is no : /// longer in use. : ThreadPool<QueryExecState*> destroy_thread_; why does the QES destruction need to happen on a separate thread? -- To view, visit http://gerrit.cloudera.org:8080/3817 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I892091d6401acb2ea91ccb1623af54c6f9635e6c Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Sailesh Mukil <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: David Knupp <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]> Gerrit-HasComments: Yes
