JingsongLi commented on code in PR #9368:
URL: https://github.com/apache/paimon/pull/9368#discussion_r3886657469
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/FlinkConnectorOptions.java:
##########
@@ -181,6 +181,16 @@ public class FlinkConnectorOptions {
.withDescription(
"The mode used by StaticFileStoreSplitEnumerator
to assign splits.");
+ public static final ConfigOption<SplitWeightMode>
SCAN_SPLIT_ENUMERATOR_WEIGHT_MODE =
+ key("scan.split-enumerator.weight-mode")
+ .enumType(SplitWeightMode.class)
+ .defaultValue(SplitWeightMode.ROW_COUNT)
+ .withDescription(
+ "The weight metric used by
StaticFileStoreSplitEnumerator. "
Review Comment:
[P2] Apply the weight mode to bounded system-table sources
This option is currently consumed only by FlinkSourceBuilder.
SystemTableSource also constructs StaticFileStoreSource for bounded
system-table reads and already propagates the batch size and assign mode, but
it calls the overload that leaves the weight function null. A query such as a
bounded $ro read with scan.split-enumerator.weight-mode=file-size therefore
silently continues balancing by row count, and file-size with preemptive is
accepted without the documented validation. Please share the parsing/validation
and weight function with SystemTableSource (including copy()), or explicitly
reject/document the unsupported path, and add a bounded system-table regression
test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
