aokolnychyi commented on code in PR #8621:
URL: https://github.com/apache/iceberg/pull/8621#discussion_r1334890624
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkWriteConf.java:
##########
@@ -179,7 +179,7 @@ public boolean fanoutWriterEnabled() {
.booleanConf()
.option(SparkWriteOptions.FANOUT_ENABLED)
.tableProperty(TableProperties.SPARK_WRITE_PARTITIONED_FANOUT_ENABLED)
-
.defaultValue(TableProperties.SPARK_WRITE_PARTITIONED_FANOUT_ENABLED_DEFAULT)
+ .defaultValue(true)
Review Comment:
The proposed approach simply enables fanout writers by default. Our
distribution and ordering code checks if the table is unsorted and whether
fanout writers are enabled to skip a local sort. Previously, we only skipped
local sorts if fanout writers were enabled explicitly.
Even though we enable fanout writers by default, it does not mean they will
be used. Our writers later check if the input is ordered. If yes, we will still
use clustered writers cause we know the local sort includes partition columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
