Hi Jink, I haven't yet fully reviewed the FLIP document, but I wanted to clarify something.
> Flink Forward Compatibility > Based on the previous clarification, Flink forward compatibility should mean that Flink jobs or ecosystems like external connectors/formats built with newer > Flink version Y like 1.15 can be running on an older Flink version X like 1.14 with no issue. With this definition, we might realize that the original requirement > in the thread [1] was asking for Flink forward compatibility, i.e. iceberg-flink module compiled with Flink 1.13 wanted to be running on Flink 1.12 cluster. And > from Flink user perspective, this is a backward compatibility issue. This is the same case: Flink forward compatibility equals Flink job/ecosystems backward > compatibility. Is it really important to provide this kind of "forward binary" compatibility? You are referring to the iceberg example, but the workaround there was quite obvious: just build the connector using Flink 1.12. As long as the connector is using stable, @Public API, AND assuming we provide binary compatibility, it should work fine with Flink 1.13. So to solve this user request, we could provide either forward OR backward binary compatibility, Right? The question is which one of those is easier to provide and explain to the users. Of course the big missing piece after FLIP-197 is that we haven't agreed on any type of binary compatibility. Best, Piotrek śr., 29 gru 2021 o 14:07 Jing Ge <j...@ververica.com> napisał(a): > Hi everyone, > > with great interest I have read all discussions [1][2][3] w.r.t. the (API?) > compatibility issues. The feedback coming from the Flink user's point of > view is very valuable. Many thanks for it. In these discussions, there were > many explanations that talked about backward and forward compatibility and > broke down to API and ABI compatibility. Since each time when a developer > referred to compatibility, there was an implicit context behind, which > might cause confusion, e.g. the forward compatibility mentioned in the API > compatibility discussion thread[1] was actually the Flink ABI backward > compatibility mentioned in FLIP-196. The original requirement posted in the > API compatibility discussion thread[1] was actually Flink ABI forward > compatibility, afaik. I will explain it in the proposal. They were all > correct because they talked from different perspectives. But it was hard > for audiences to follow it. > > I’d like to start a discussion about backward/forward compatibility from > both users perspective and Flink perspective. I Tried to put myself in > users' shoes and then to see whether we could do anything to reduce users' > effort for upgrading Flink. > > The proposal contains four parts: > - Clarify the definition of API and ABI, backward and forward compatibility > to make sure everyone is on the same page. > - Analyze and classify issues from users' feedback. > - Summarize the definition and the gap between the current status and what > users need. > - Proposed changes > > Please find the details on > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-207%3A+Flink+backward+and+forward+compatibility > > Looking forward to your feedback. Many thanks. > > best regards > Jing >
