Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/8400 )
Change subject: IMPALA-4167: Support insert plan hints for CREATE TABLE AS SELECT ...................................................................... Patch Set 7: (7 comments) I have overwritten the wrong test in patch set 7, please ignore it and compare patch 6 to 8. http://gerrit.cloudera.org:8080/#/c/8400/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8400/6//COMMIT_MSG@19 PS6, Line 19: nly > add "more memory efficient"? That makes it more obvious why it is more effi I hope it is correct and understandable enough this way. http://gerrit.cloudera.org:8080/#/c/8400/6//COMMIT_MSG@27 PS6, Line 27: shuffle: > I don't think this is true, is it? For unpartitioned tables, what would the Done http://gerrit.cloudera.org:8080/#/c/8400/6//COMMIT_MSG@35 PS6, Line 35: > Mention how you tested it in the commit message. I wrote about the current state of tests, but I became unsure during the process - should I add tests that actually execute the CTASs with hints? The reason why not to add them is that the non-DDL part of query execution is really the same as in INSERT tests and can be quite slow. http://gerrit.cloudera.org:8080/#/c/8400/6/fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java File fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java: http://gerrit.cloudera.org:8080/#/c/8400/6/fe/src/main/java/org/apache/impala/analysis/CreateTableAsSelectStmt.java@69 PS6, Line 69: * Helper class for parsing. > Is this class still needed or a left-over from earlier patch-sets? Can you It is still needed, as it is the result value of non-terminal create_tbl_as_select_stmt_inner. I am not sure that this is the correct place for this explanation, maybe it should be moved to the .cup. http://gerrit.cloudera.org:8080/#/c/8400/6/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java: http://gerrit.cloudera.org:8080/#/c/8400/6/fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java@1701 PS6, Line 1701: Multiple non-conflicting > Can you add one that has multiple distinct hints, like shuffle, clustered? Done http://gerrit.cloudera.org:8080/#/c/8400/6/testdata/workloads/functional-planner/queries/PlannerTest/ddl.test File testdata/workloads/functional-planner/queries/PlannerTest/ddl.test: http://gerrit.cloudera.org:8080/#/c/8400/6/testdata/workloads/functional-planner/queries/PlannerTest/ddl.test@233 PS6, Line 233: shuffle > Is that true? Without specifying a hint there should be cases where the cos Thanks for spotting this, I have relied too much on experimentation instead of checking the code... I have split this test into two to test both outcomes. http://gerrit.cloudera.org:8080/#/c/8400/6/testdata/workloads/functional-planner/queries/PlannerTest/ddl.test@344 PS6, Line 344: | order by: year ASC NULLS > At first I didn't understand what this one does (I saw that insert.test:574 I should add a comment about this, but I am unsure whether it should be a simple comment, or a todo about a strange behavior. This looks intentional (see https://gerrit.cloudera.org/#/c/4162/ ), but it is not mentioned explicitly in the documentation. It is true that this reduces the number of writes overall, but as a side effect, it causes the whole table to be written to one node. Writing the whole table to the coordinator may be useful in some cases (like a temp table?), but I think that shuffle has a different intention. I would ask Greg / Alex about their opinion. -- To view, visit http://gerrit.cloudera.org:8080/8400 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8d74bca999da8ae1bb89427c70841f33e3c56ab0 Gerrit-Change-Number: 8400 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Comment-Date: Mon, 15 Jan 2018 18:06:38 +0000 Gerrit-HasComments: Yes
