Norbert Luksa has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/13955 )
Change subject: IMPALA-8755: Frontend support for Z-ordering ...................................................................... IMPALA-8755: Frontend support for Z-ordering Extended the SQL grammar with an optional flag for SORT BY, ZORDER. If set, the new 'sort.algorithm' table property will be set to ZORDER and the information will sink down to the backend. The default order is indicated by LEXICAL and can be omitted. Examples are: CREATE TABLE t (a INT, b INT) PARTITIONED BY (c INT) SORT BY ZORDER (a, b); CREATE TABLE t SORT BY ZORDER (int_col,id) LIKE u; CREATE TABLE t LIKE PARQUET '/foo' SORT BY ZORDER (id,zip); ALTER TABLE t SORT BY ZORDER (int_col,id); The following two are the same statements: CREATE TABLE t (a INT, b INT) SORT BY (a, b); CREATE TABLE t (a INT, b INT) SORT BY LEXICAL (a, b); For strings, varchars, floats and doubles Z-ordering is currently not supported. It's not suitable for strings and varchars, but support can be added for floats and doubles later. Currently ZORDER has the same functionality as a simple SORT BY clause, therefore hidden behind a feature flag: unlock_zorder. The custom sorting with Z-ordering will be in a different commit later. Testing: * Added tests for the ZORDER option for every SORT BY test. * Modified some tests by adding the LEXICAL option. * The .test workloads are temporarily put in separate test files in order to set up the feature flag. These tests are run from tests/custom_cluster/test_zorder.py which is a duplication of the relevant tests, but with CustomClusterTestSuite decorator. Change-Id: Ie122002ca8f52ca2c1e1ec8ff1d476ae1f4f875d --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/DataSinks.thrift M common/thrift/JniCatalog.thrift M common/thrift/PlanNodes.thrift M common/thrift/Types.thrift M fe/src/main/cup/sql-parser.cup M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java M fe/src/main/java/org/apache/impala/analysis/AlterTableSortByStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeFileStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java M fe/src/main/java/org/apache/impala/analysis/DeleteStmt.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/SortInfo.java M fe/src/main/java/org/apache/impala/analysis/TableDef.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/analysis/UpdateStmt.java M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M fe/src/main/java/org/apache/impala/planner/ExchangeNode.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/main/java/org/apache/impala/planner/Planner.java M fe/src/main/java/org/apache/impala/planner/SortNode.java M fe/src/main/java/org/apache/impala/planner/TableSink.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/jflex/sql-scanner.flex M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlUtilsTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A testdata/workloads/functional-planner/queries/PlannerTest/insert-sort-by-zorder.test A testdata/workloads/functional-query/queries/QueryTest/alter-table-zorder.test A testdata/workloads/functional-query/queries/QueryTest/create-table-as-select-zorder.test A testdata/workloads/functional-query/queries/QueryTest/create-table-like-file-zorder.test A testdata/workloads/functional-query/queries/QueryTest/create-table-like-table-zorder.test A testdata/workloads/functional-query/queries/QueryTest/create-table-zorder.test A testdata/workloads/functional-query/queries/QueryTest/show-create-table-zorder.test M testdata/workloads/functional-query/queries/QueryTest/show-create-table.test A tests/custom_cluster/test_zorder.py 44 files changed, 1,796 insertions(+), 159 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/13955/9 -- To view, visit http://gerrit.cloudera.org:8080/13955 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie122002ca8f52ca2c1e1ec8ff1d476ae1f4f875d Gerrit-Change-Number: 13955 Gerrit-PatchSet: 9 Gerrit-Owner: Norbert Luksa <norbert.lu...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Norbert Luksa <norbert.lu...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
