Sourabh Badhya created HIVE-27991:
-------------------------------------
Summary: Utilise FanoutWriters when inserting records in an
Iceberg table when the records are unsorted
Key: HIVE-27991
URL: https://issues.apache.org/jira/browse/HIVE-27991
Project: Hive
Issue Type: Improvement
Reporter: Sourabh Badhya
Assignee: Sourabh Badhya
FanoutWriter is a writer in Iceberg which can be used to write records in a
table. This writer keeps all the file handles open, until the write is
finished. FanoutWriters is used as the writer when the incoming records are
unsorted. We can by default have some mechanism to switch to using
FanoutWriters instead of ClusteredWriters when custom sort expressions are not
present for the given table/query.
Similar stuff is already implemented in Spark -
[https://github.com/apache/iceberg/pull/8621]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
