John Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/17144 )
Change subject: IMPALA-10551: Add result sink support for external frontends ...................................................................... Patch Set 2: (6 comments) http://gerrit.cloudera.org:8080/#/c/17144/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17144/1//COMMIT_MSG@14 PS1, Line 14: - External frontends are responsible for managing the files after the : query is completed. > I wonder how this would interact with the "retry_failed_queries" query opti I think the newest version of the patch doesn't try to retry failed queries when there is a result sink. http://gerrit.cloudera.org:8080/#/c/17144/1/be/src/runtime/coordinator.cc File be/src/runtime/coordinator.cc: http://gerrit.cloudera.org:8080/#/c/17144/1/be/src/runtime/coordinator.cc@798 PS1, Line 798: Coordinator::FinalizeResultSink() > I might be missing something, but where is this called? Great catch - I think some code got lost in a rebase. Will show up in next patch (in Wait()). http://gerrit.cloudera.org:8080/#/c/17144/1/be/src/runtime/coordinator.cc@907 PS1, Line 907: } else if (stmt_type_ == TStmtType::QUERY) { : DCHECK(coord_instance_ != nullptr); > I'm thinking through the implication of calling this after WaitForBackends( I think a delay in reporting an error is okay for now (As long as that is the only negative in thee way I am doing this) I will also take some time to think it through a bit since I haven't thought about this code for awhile. http://gerrit.cloudera.org:8080/#/c/17144/2/be/src/runtime/coordinator.cc File be/src/runtime/coordinator.cc: http://gerrit.cloudera.org:8080/#/c/17144/2/be/src/runtime/coordinator.cc@897 PS2, Line 897: // than streaming them to a client directly. For these types of queries we need to wait > line too long (91 > 90) I'll catch in next round of fixes. http://gerrit.cloudera.org:8080/#/c/17144/1/be/src/runtime/fragment-instance-state.h File be/src/runtime/fragment-instance-state.h: http://gerrit.cloudera.org:8080/#/c/17144/1/be/src/runtime/fragment-instance-state.h@121 PS1, Line 121: JoinBuilder* GetJoinBuildSink() const; > Add CR Done http://gerrit.cloudera.org:8080/#/c/17144/1/fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java File fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java: http://gerrit.cloudera.org:8080/#/c/17144/1/fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java@245 PS1, Line 245: exprs.addAll(outputExprs_.subList(0, > line too long (91 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/17144 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I024bf41d77bb81f1ab0debdbd31ec3687c83f072 Gerrit-Change-Number: 17144 Gerrit-PatchSet: 2 Gerrit-Owner: John Sherman <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: John Sherman <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Comment-Date: Tue, 02 Mar 2021 00:23:26 +0000 Gerrit-HasComments: Yes
