[
https://issues.apache.org/jira/browse/FLINK-26759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510316#comment-17510316
]
Martijn Visser commented on FLINK-26759:
----------------------------------------
[~fanrui] I totally understand that. In the end, I think it's also honest that
new features don't get backported to older versions. Those can also introduce
instabilities or break API compatibility. If the companies have a large
business value on Flink, I would hope that they also free up engineers to help
contribute improvements to Flink in general and not put the maintenance burden
in the hands of an open source community. With regards to your arguments:
1. This is exactly a reason why companies should perform the upgrade, so that
they can get benefits such as the one one mentioned in this ticket.
2. I would rather focus engineering efforts on improving this instead of going
for the easy solution. This benefits all users and the Flink community.
3. Doesn't this also mean that backporting a new feature wouldn't make sense,
because it still would only be used by new jobs/new SQL jobs. In the end, you
need to perform upgrades and maintenance anyway. Support for Flink 1.13 will be
dropped when Flink 1.15 is released.
> From a business perspective, they prefer to optimize the engine without
> changing the business code.
I agree, but we're not only here for the business perspective, but we need to
make sure that Flink can still be properly maintained by the community. If we
would go this route, it would make maintaining harder, with a risk of losing
contributors which would hurt Flink in the longer run.
The SourceFunction and SinkFunction will be deprecated with FLINK-25852 in the
next release.
> Legacy source support waiting for recordWriter to be available
> --------------------------------------------------------------
>
> Key: FLINK-26759
> URL: https://issues.apache.org/jira/browse/FLINK-26759
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Common, Runtime / Checkpointing
> Affects Versions: 1.13.0, 1.14.0, 1.15.0
> Reporter: fanrui
> Priority: Major
>
> In order for Unaligned Checkpoint not to be blocked, StreamTask#processInput
> will check recordWriter.isAvailable(). If not available, the data will not be
> processed until recordWriter is available.
> The new Source api is compatible with the above logic, but Legacy Source is
> not compatible with the above logic. When using Unaligned Checkpoint, if the
> backpressure of Legacy Source is high, the Checkpoint duration of Legacy
> Source will be very long.
>
> Since legacy sources are often used in production, can we add logic to wait
> for recordWriter to be available for legacy source?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
