Hello Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24543
to look at the new patch set (#5).
Change subject: IMPALA-15054: Set EXPAND_COMPLEX_TYPES to true by default
......................................................................
IMPALA-15054: Set EXPAND_COMPLEX_TYPES to true by default
Flips the default of the EXPAND_COMPLEX_TYPES query option from false to
true so that '*' expressions include complex-typed (ARRAY/MAP/STRUCT)
columns by default.
- common/thrift/Query.thrift: change the default to true.
- Docs: update the EXPAND_COMPLEX_TYPES default and rationale, rework
the complex-types SELECT * example, add an Impala 5.0.x
incompatible-changes entry, and clarify the Iceberg metadata-tables
note.
- SelectStmt.java: refresh stale comments (the empty-select-list
invariant in analyzeStarItem still holds under the new default).
Test handling for the behaviour change. Where a query specifically uses
the collection-join/unnest syntax, expands a struct via '.*', or is a
dedicated/paired complex-type test, EXPAND_COMPLEX_TYPES is pinned to
false so the query keeps its previous meaning; incidental top-level
'SELECT *' tests adopt the new default and their expected results are
regenerated.
- AnalyzeStmtsTest.TestInsert: pin the affected query via a no-expand
AnalysisContext ('select *' on a text-format table with complex
columns would otherwise fail before the intended error).
- AnalyzeStmtsTest: move the cases that exercise the no-expand behaviour
into a dedicated TestDisableExpandComplexTypes test that sets
EXPAND_COMPLEX_TYPES=false.
- AnalyzerTest.TestVirtualColumnInputFileName and empty.test: alias the
self-unnested collection ref so the column expanded by '*' does not
collide with the collection alias, keeping EXPAND_COMPLEX_TYPES at its
new default.
- .test files: pin EXPAND_COMPLEX_TYPES=false where we need the
old behavior
- Incidental 'SELECT *' goldens regenerated for the new default
- test_lineage.py: run QueryTest/lineage with EXPAND_COMPLEX_TYPES=false
as a session option so the expected lineage graphs (and their query
hashes) stay valid without altering the recorded query text.
Change-Id: I6cf01f0b6bb3b5fa2aa770c77bc772abf9cc84b2
Assisted-by: Claude Opus 4.8 (1M context) <[email protected]>
---
M common/thrift/Query.thrift
M docs/topics/impala_complex_types.xml
M docs/topics/impala_expand_complex_types.xml
M docs/topics/impala_iceberg.xml
M docs/topics/impala_incompatible_changes.xml
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/workloads/functional-planner/queries/PlannerTest/empty.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-metadata-tables.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-migrated-table-field-id-resolution-complex.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-query.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-negative.test
M testdata/workloads/functional-query/queries/QueryTest/max-nesting-depth.test
M
testdata/workloads/functional-query/queries/QueryTest/nested-struct-in-select-list.test
M
testdata/workloads/functional-query/queries/QueryTest/nested-types-star-expansion.test
M
testdata/workloads/functional-query/queries/QueryTest/nested-types-subplan.test
M
testdata/workloads/functional-query/queries/QueryTest/nested-types-tpch-mem-limit.test
M testdata/workloads/functional-query/queries/QueryTest/paimon-query.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_row_filtering.test
M
testdata/workloads/functional-query/queries/QueryTest/virtual-column-file-position-parquet.test
M tests/custom_cluster/test_lineage.py
M tests/query_test/test_kudu.py
25 files changed, 201 insertions(+), 86 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24543/5
--
To view, visit http://gerrit.cloudera.org:8080/24543
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6cf01f0b6bb3b5fa2aa770c77bc772abf9cc84b2
Gerrit-Change-Number: 24543
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
