JingsongLi commented on code in PR #7914:
URL: https://github.com/apache/paimon/pull/7914#discussion_r3534354894
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/source/BaseDataTableSource.java:
##########
@@ -206,7 +207,8 @@ public ScanRuntimeProvider
getScanRuntimeProvider(ScanContext scanContext) {
.env(env)
.build(),
tableIdentifier.asSummaryString(),
- table);
+ table,
+ options.get(FlinkConnectorOptions.SCAN_PARALLELISM));
Review Comment:
This only exposes the explicitly configured `scan.parallelism` to
`ParallelismProvider`. The default path still calls
`inferSourceParallelism(env)` inside the producer (for example bucket/split
based inference when `infer-scan.parallelism` is enabled and the env
parallelism is default), but `getParallelism()` remains empty for that inferred
value. In that common case the planner never gets a configured parallelism to
mark with `setParallelism(..., true)`, while the producer still sets the source
parallelism through `DataStream#setParallelism` / transformation parallelism.
That leaves the forward-edge propagation bug unresolved for the default
inferred path. Please compute/reuse the same resolved source parallelism for
the provider, and cover the inferred path with a job-graph assertion, not just
the explicit option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
