kokila-19 opened a new pull request, #6558: URL: https://github.com/apache/hive/pull/6558
### What changes were proposed in this pull request? - Iceberg tables can now use CLUSTERED BY together with sorting (z-order or WRITE LOCALLY ORDERED BY) on write. - SDPO removes the extra enforce-bucketing reduce sink, reuses its reducer count when needed and when buckets exceed reducers, rows carry a Hive bucket id and output files are routed by bucket. - HMS bucketing metadata is passed through jobconf to the SerDe/writer. - Iceberg TABLESAMPLE(BUCKET …) input pruning is fixed. - Tests were added for z-order and linear sort. ### Why are the changes needed? Without this, CLUSTERED BY + Iceberg z-order/linear sort did not work correctly: SDPO and enforce-bucketing conflicted, reducers could be fewer than buckets (wrong file layout) and Iceberg bucket sampling was wrong because it assumed native Hive bucket paths. These changes keep Hive bucket semantics while still applying Iceberg sort order on write. ### Does this PR introduce _any_ user-facing change? Yes. Users can now use CLUSTERED BY + Sorting (WRITE LOCALLY ORDERED BY or WRITE LOCALLY ORDERED BY ZORDER) with Iceberg tables: ``` CREATE TABLE t (...) CLUSTERED BY (col) INTO N BUCKETS WRITE LOCALLY ORDERED BY zorder(...) STORED BY ICEBERG; ``` DESCRIBE FORMATTED now shows bucket metadata and sorting info. ### How was this patch tested? Junit - TestHiveIcebergClusteredByWithZOrder, TestHiveIcebergClusteredByWithLinearSort, qtest - iceberg_clustered_by_with_zorder.q , iceberg_clustered_by_with_linear_sort.q -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
