Marcel Kornacker has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. ......................................................................
Patch Set 2: (10 comments) http://gerrit.cloudera.org:8080/#/c/4677/2/be/src/exec/exec-node.cc File be/src/exec/exec-node.cc: Line 272: DCHECK_EQ(state->query_options().num_scanner_threads, 1); roll into a single dcheck with a compound predicate http://gerrit.cloudera.org:8080/#/c/4677/2/common/thrift/PlanNodes.thrift File common/thrift/PlanNodes.thrift: Line 205: trailing Line 207: // If this is true then the MT_DOP query option must be > 0. leave todo that it should be removed later http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: Line 314: * Retrurns a set of file formats being scanned. spelling the set Line 562: msg.hdfs_scan_node.setUse_mt_scan_nodeIsSet(useMtScanNode_); ..IsSet? it doesn't look like you're calling the actual setter http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/Planner.java File fe/src/main/java/org/apache/impala/planner/Planner.java: Line 206: if (ctx_.getQueryOptions().mt_dop > 0) { the if is redundant, but you can make it a checkstate at the start of the function, if you want. also, the planner only creates parallel plans for query stmts. http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/PlannerContext.java File fe/src/main/java/org/apache/impala/planner/PlannerContext.java: Line 83: public TQueryOptions getQueryOptions() { return getRootAnalyzer().getQueryOptions(); } why the change? http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: Line 155: * - MT_DOP > 0 is not supported for plans with distributed joins or table sinks. it's only supported for QueryStmts. take a look at Frontend.createExecRequest() http://gerrit.cloudera.org:8080/#/c/4677/1/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 5: ---- PLAN > The purpose is to validate that this plan is executable with mt_dop>0 and n but it's not executable, because the join sink isn't executable. sorry, i overlooked that earlier: for the upcoming release we need to turn this off even for non-distributed joins. http://gerrit.cloudera.org:8080/#/c/4677/2/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 51: | tuple-ids=0,1 row-size=8B cardinality=unavailable what happened here? -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-HasComments: Yes
