Daniel Becker has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/19660 )
Change subject: IMPALA-12019: Support ORDER BY for arrays of fixed length types in select list ...................................................................... IMPALA-12019: Support ORDER BY for arrays of fixed length types in select list As a first stage of IMPALA-10939, this change implements support for including in the sorting tuple top-level collections that only contain fixed length types (including fixed length structs). For these types the implementation is almost the same as the existing handling of strings. Another limitation is that structs that contain any type of collection are not yet allowed in the sorting tuple. Also refactored the RawValue::Write*() functions to have a clearer interface. Testing: - Added a new test table that contains many rows with arrays. This is queried in a new test added in test_sort.py, to ensure that we handle spilling correctly. - Added tests in test_nested_types.py that have arrays and maps in the sorting tuple - TODO: We should also test MAPs on a big table to test spilling but with the data generator expects an AVRO schema and in AVRO all struct keys are strings which are var-len. Change-Id: Ic7974ef392c1412e8c60231e3420367bd189677a --- M be/src/exec/hash-table.cc M be/src/runtime/collection-value.cc M be/src/runtime/collection-value.h M be/src/runtime/raw-value.cc M be/src/runtime/raw-value.h M be/src/runtime/sorter-internal.h M be/src/runtime/sorter.cc M be/src/runtime/tuple.cc M be/src/runtime/tuple.h M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/QueryStmt.java M fe/src/main/java/org/apache/impala/analysis/SortInfo.java A testdata/ComplexTypesTbl/simple_arrays_big.parq M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/nested-array-in-select-list.test M testdata/workloads/functional-query/queries/QueryTest/nested-map-in-select-list.test M tests/query_test/test_nested_types.py M tests/query_test/test_sort.py 20 files changed, 856 insertions(+), 221 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/19660/7 -- To view, visit http://gerrit.cloudera.org:8080/19660 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic7974ef392c1412e8c60231e3420367bd189677a Gerrit-Change-Number: 19660 Gerrit-PatchSet: 7 Gerrit-Owner: Daniel Becker <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]>
