[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. MT_DOP > 0 is only supported for plans without distributed joins or table sinks. Adds validation to fail unsupported queries gracefully in planning. For scans in queries that are executable with MT_DOP > 0 we either use the optimized MT scan node BE implementation (only Parquet), or we use the conventional scan node with num_scanner_threads=1. TODO: Still need to add end-to-end tests. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Reviewed-on: http://gerrit.cloudera.org:8080/4677 Reviewed-by: Alex BehmTested-by: Internal Jenkins --- M be/src/exec/exec-node.cc M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/PlannerContext.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java A testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 10 files changed, 450 insertions(+), 34 deletions(-) Approvals: Internal Jenkins: Verified Alex Behm: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm Gerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 6: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 6: Code-Review+2 Rebase + trival fix. -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Hello Marcel Kornacker, Internal Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/4677 to look at the new patch set (#6). Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. MT_DOP > 0 is only supported for plans without distributed joins or table sinks. Adds validation to fail unsupported queries gracefully in planning. For scans in queries that are executable with MT_DOP > 0 we either use the optimized MT scan node BE implementation (only Parquet), or we use the conventional scan node with num_scanner_threads=1. TODO: Still need to add end-to-end tests. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d --- M be/src/exec/exec-node.cc M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/PlannerContext.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java A testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 10 files changed, 450 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/4677/6 -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Internal Jenkins has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 5: Verified-1 Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/298/ -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Hello Marcel Kornacker, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/4677 to look at the new patch set (#5). Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. MT_DOP > 0 is only supported for plans without distributed joins or table sinks. Adds validation to fail unsupported queries gracefully in planning. For scans in queries that are executable with MT_DOP > 0 we either use the optimized MT scan node BE implementation (only Parquet), or we use the conventional scan node with num_scanner_threads=1. TODO: Still need to add end-to-end tests. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d --- M be/src/exec/exec-node.cc M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/PlannerContext.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java A testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 10 files changed, 431 insertions(+), 33 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/4677/5 -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/4677/4/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 41: PARALLELPLANS > let's also check 'plan' here, because these should also run in single-node Done Line 115: PARALLELPLANS > same here Done Line 170: PARALLELPLANS > and here Done -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Marcel Kornacker has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 4: Code-Review+2 (3 comments) http://gerrit.cloudera.org:8080/#/c/4677/4/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 41: PARALLELPLANS let's also check 'plan' here, because these should also run in single-node mode Line 115: PARALLELPLANS same here Line 170: PARALLELPLANS and here -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/4677/1/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 5: PLAN > the build registration, etc., isn't expected to get in before code freeze, Ahh right. Done. -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has uploaded a new patch set (#4). Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. MT_DOP > 0 is only supported for plans without distributed joins or table sinks. Adds validation to fail unsupported queries gracefully in planning. For scans in queries that are executable with MT_DOP > 0 we either use the optimized MT scan node BE implementation (only Parquet), or we use the conventional scan node with num_scanner_threads=1. TODO: Still need to add end-to-end tests. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d --- M be/src/exec/exec-node.cc M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/PlannerContext.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java A testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 10 files changed, 308 insertions(+), 33 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/4677/4 -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/4677/1/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 5: PLAN > but it's not executable, because the join sink isn't executable. We need the current mode for executing joins in subplans where the build will hopefully never be in a separate fragment. -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has uploaded a new patch set (#3). Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. MT_DOP > 0 is only supported for plans without distributed joins or table sinks. Adds validation to fail unsupported queries gracefully in planning. For scans in queries that are executable with MT_DOP > 0 we either use the optimized MT scan node BE implementation (only Parquet), or we use the conventional scan node with num_scanner_threads=1. TODO: Still need to add end-to-end tests. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d --- M be/src/exec/exec-node.cc M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/PlannerContext.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java A testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 10 files changed, 367 insertions(+), 33 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/4677/3 -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 2: (9 comments) http://gerrit.cloudera.org:8080/#/c/4677/2/be/src/exec/exec-node.cc File be/src/exec/exec-node.cc: Line 272: DCHECK_EQ(state->query_options().num_scanner_threads, 1); > roll into a single dcheck with a compound predicate Done http://gerrit.cloudera.org:8080/#/c/4677/2/common/thrift/PlanNodes.thrift File common/thrift/PlanNodes.thrift: Line 205: > trailing Done Line 207: // If this is true then the MT_DOP query option must be > 0. > leave todo that it should be removed later Done http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: Line 314:* Retrurns a set of file formats being scanned. > spelling Done Line 562: msg.hdfs_scan_node.setUse_mt_scan_nodeIsSet(useMtScanNode_); > ..IsSet? it doesn't look like you're calling the actual setter Weird. That seemed to work also. Fixed. http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/Planner.java File fe/src/main/java/org/apache/impala/planner/Planner.java: Line 206: if (ctx_.getQueryOptions().mt_dop > 0) { > the if is redundant, but you can make it a checkstate at the start of the f Added Preconditions check. http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/PlannerContext.java File fe/src/main/java/org/apache/impala/planner/PlannerContext.java: Line 83: public TQueryOptions getQueryOptions() { return getRootAnalyzer().getQueryOptions(); } > why the change? I added a convenience function Analyzer.getQueryOptions() and thought it would make sense to use it here. http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: Line 155:* - MT_DOP > 0 is not supported for plans with distributed joins or table sinks. > it's only supported for QueryStmts. take a look at Frontend.createExecReque My thinking was that from a user's perspective running a statement with mt_dop > 0 should have one of two outcomes: 1. The statement runs fine in multi-threaded mode. 2. The statement returns an error indicating that mt_dop is not supported for that statement. If that's the behavior we want, then it's irrelevant whether we can actually generate parallel plans for inserts/ctas. http://gerrit.cloudera.org:8080/#/c/4677/2/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 51: | tuple-ids=0,1 row-size=8B cardinality=unavailable > what happened here? functional_parquet has no stats. I preferred to write the tests on Parquet because that's the preferred file format, happy to change it back or add more tests? -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Marcel Kornacker has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 2: (10 comments) http://gerrit.cloudera.org:8080/#/c/4677/2/be/src/exec/exec-node.cc File be/src/exec/exec-node.cc: Line 272: DCHECK_EQ(state->query_options().num_scanner_threads, 1); roll into a single dcheck with a compound predicate http://gerrit.cloudera.org:8080/#/c/4677/2/common/thrift/PlanNodes.thrift File common/thrift/PlanNodes.thrift: Line 205: trailing Line 207: // If this is true then the MT_DOP query option must be > 0. leave todo that it should be removed later http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: Line 314:* Retrurns a set of file formats being scanned. spelling the set Line 562: msg.hdfs_scan_node.setUse_mt_scan_nodeIsSet(useMtScanNode_); ..IsSet? it doesn't look like you're calling the actual setter http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/Planner.java File fe/src/main/java/org/apache/impala/planner/Planner.java: Line 206: if (ctx_.getQueryOptions().mt_dop > 0) { the if is redundant, but you can make it a checkstate at the start of the function, if you want. also, the planner only creates parallel plans for query stmts. http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/PlannerContext.java File fe/src/main/java/org/apache/impala/planner/PlannerContext.java: Line 83: public TQueryOptions getQueryOptions() { return getRootAnalyzer().getQueryOptions(); } why the change? http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: Line 155:* - MT_DOP > 0 is not supported for plans with distributed joins or table sinks. it's only supported for QueryStmts. take a look at Frontend.createExecRequest() http://gerrit.cloudera.org:8080/#/c/4677/1/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 5: PLAN > The purpose is to validate that this plan is executable with mt_dop>0 and n but it's not executable, because the join sink isn't executable. sorry, i overlooked that earlier: for the upcoming release we need to turn this off even for non-distributed joins. http://gerrit.cloudera.org:8080/#/c/4677/2/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 51: | tuple-ids=0,1 row-size=8B cardinality=unavailable what happened here? -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has uploaded a new patch set (#2). Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. MT_DOP > 0 is only supported for plans without distributed joins or table sinks. Adds validation to fail unsupported queries gracefully in planning. For scans in queries that are executable with MT_DOP > 0 we either use the optimized MT scan node BE implementation (only Parquet), or we use the conventional scan node with num_scanner_threads=1. TODO: Still need to add end-to-end tests. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d --- M be/src/exec/exec-node.cc M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/PlannerContext.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java A testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 10 files changed, 368 insertions(+), 33 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/4677/2 -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Alex Behm Gerrit-Reviewer: Marcel Kornacker
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Marcel Kornacker has posted comments on this change. Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/4677/1/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test: Line 5: PLAN do we need this section? or isn't this already validated somewhere else? Line 32: DISTRIBUTEDPLAN we don't need this section Line 121: select row_number() over(partition by int_col order by id) have you tried to run this? Line 156: from tpch_nested_parquet.customer c, c.c_orders o, o.o_lineitems have you tried this? -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex BehmGerrit-Reviewer: Marcel Kornacker Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Alex Behm has uploaded a new change for review. http://gerrit.cloudera.org:8080/4677 Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. .. IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0. Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d --- M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java A testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test 5 files changed, 305 insertions(+), 24 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/4677/1 -- To view, visit http://gerrit.cloudera.org:8080/4677 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm