Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16590 )
Change subject: IMPALA-10233: zorder sort node should output rows in lexical order of partition keys ...................................................................... IMPALA-10233: zorder sort node should output rows in lexical order of partition keys When inserting to a partitioned hdfs table, the planner will add a sort node on top of the plan, depending on the clustered/noclustered plan hint and on the 'sort.columns' table property. If clustering is enabled in insertStmt or additional columns are specified in the 'sort.columns' table property, then the ordering columns will start with the clustering columns, so that partitions can be written sequentially in the table sink. Any additional non-clustering columns specified by the 'sort.columns' property will be added to the ordering columns and after any clustering columns. For Z-order sort type, we should deal with these ordering columns separately. The clustering columns should still be sorted lexically, and only the remaining ordering columns be sorted in Z-order. So we can still insert partitions one by one and avoid hitting the DCHECK as described in the JIRA. Tests - Add tests for inserting to a partitioned table with zorder. Change-Id: I30cbad711167b8b63c81837e497b36fd41be9b54 Reviewed-on: http://gerrit.cloudera.org:8080/16590 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/runtime/dml-exec-state.cc M be/src/util/tuple-row-compare.cc M be/src/util/tuple-row-compare.h M common/thrift/PlanNodes.thrift M fe/src/main/java/org/apache/impala/analysis/SortInfo.java M fe/src/main/java/org/apache/impala/planner/ExchangeNode.java M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M testdata/workloads/functional-planner/queries/PlannerTest/insert-sort-by-zorder.test M testdata/workloads/functional-query/queries/QueryTest/insert.test 11 files changed, 220 insertions(+), 36 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16590 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I30cbad711167b8b63c81837e497b36fd41be9b54 Gerrit-Change-Number: 16590 Gerrit-PatchSet: 8 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Norbert Luksa <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
