zhoulii commented on pull request #19040: URL: https://github.com/apache/flink/pull/19040#issuecomment-1067738895
> Thanks for all the efforts on this PR @zhoulii, but unfortunately this PR doesn't fully solve the problem and introduces new problems. > > Currently, the solution discussed in [FLINK-26548](https://issues.apache.org/jira/browse/FLINK-26548) encounters the following problem: Setting the parallelism of the actual source directly will cause the parallelism of the multiInput to be configured, which will break the forward property and cause the following error: `org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Forward partitioning does not allow change of parallelism. Upstream operation: Calc[1124]-1669 parallelism: -1, downstream operation: MultipleInput[1149]-1721 parallelism: 1 You must use another partitioning strategy, such as broadcast, rebalance, shuffle or global.` > > Considering that the legacy file sources will be replaced by new source(FLIP-27), and there are some known issues: [FLINK-26576](https://issues.apache.org/jira/browse/FLINK-26576), I think we should choose to support new sources only, and add this to limitations of adaptive batch scheduler. BTW, I think we should migrate the `StreamExecutionEnvironment#readFile` `StreamExecutionEnvironment#createInput` to new source as soon as possible. > > WDYT @zhoulii @zhuzhurk ? Thanks for replying, I found this problem too. Considering that the legacy file sources will be replaced by new source and this pr would introduce some new problems, supporting new sources only is a proper way, so I agree with your proposal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
