RussellSpitzer commented on code in PR #14948:
URL: https://github.com/apache/iceberg/pull/14948#discussion_r3320703962
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkScan.java:
##########
@@ -366,7 +367,18 @@ public CustomMetric[] supportedCustomMetrics() {
};
}
+ protected boolean isOrderingEnabled() {
+ return false;
+ }
+
protected long adjustSplitSize(List<? extends ScanTask> tasks, long
splitSize) {
+ if (readConf.preserveDataOrdering() && readConf.preserveDataGrouping()) {
Review Comment:
Not sure if you want to check this elsewhere, but currently we will get some
weird behaviors with unpartitioned and unsorted tables when these are true.
Maybe we should have some precondition guards against setting these
properites if the table is not sorted or partitioned to avoid getting to this
point in the code and accidently creating one giant split even when the table
isn't partitioned (or sorted)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
