huaxingao commented on code in PR #18709:
URL: https://github.com/apache/iceberg/pull/18709#discussion_r4534156375
##########
spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkPartitioningAwareScan.java:
##########
@@ -233,11 +233,14 @@ protected synchronized List<T> tasks() {
@Override
protected synchronized List<ScanTaskGroup<T>> taskGroups() {
if (taskGroups == null) {
+ long splitSize = adjustSplitSize(tasks(), scan.targetSplitSize());
+ setEffectiveSplitTargetSize(splitSize);
Review Comment:
When the scan has no files to read, this still records a non-zero target
instead of 0. For example, a filter like `WHERE id > 100` prunes every file, or
a `DELETE` removed all data files. `tasks()` is empty, so the scan size is 0,
and with adaptive sizing on (the default) `adjustSplitSize` returns `min(16 MB,
target)`. I checked both cases locally and each reported 16777216. The
description says a scan that plans no task groups reports 0. Should this record
0 when `tasks()` is empty?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
