Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
Thanks for the follow-up Chesnay, I believe this compatible result is useful, and +1 for applying the FLIP to 1.15. And yes, from the result we are doing a good job keeping the binary compatibility in the 1.15 branch :-) Best Regards, Yu On Thu, 1 Sept 2022 at 19:54, Chesnay Schepler wrote: > I compared 1.15.0 and 1.15.2; we overall did a very good job with one > outlier. > Based on this I'd say we should also apply this FLIP to 1.15. > > Table connectors: > Source-compatible change; relevant if you implement your own > DataStream(Scan/Sink)Provider: > > > org.apache.flink.table.connector.sink.DataStreamSinkProvider.consumeDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.datastream.DataStream):METHOD_ABSTRACT_NOW_DEFAULT > > > org.apache.flink.table.connector.source.DataStreamScanProvider.produceDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.environment.StreamExecutionEnvironment):METHOD_ABSTRACT_NOW_DEFAULT > > Pulsar connector: > These are due to adding @Internal in 1.15.2; no effect on compatibility: > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.getMessageId():METHOD_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.valueOf(java.lang.String):METHOD_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.values():METHOD_REMOVED,java.lang.Comparable[java.lang.Comparable]:INTERFACE_REMOVED,java.io.Serializable[ > java.io.Serializable]:INTERFACE_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:SUPERCLASS_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.TIMESTAMP:FIELD_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.MESSAGE_ID:FIELD_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:CLASS_REMOVED > These are straight-up breaking changes (neither source nor binary > compatible): > > org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.seekPosition(org.apache.pulsar.client.api.Consumer):METHOD_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.fromPublishTime(long):METHOD_NEW_DEFAULT > > org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.seekPosition(java.lang.String,int,org.apache.pulsar.client.api.Consumer):METHOD_REMOVED > > org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterEventTime(long):METHOD_NEW_DEFAULT,org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterPublishTime(long):METHOD_NEW_DEFAULT > > org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.shouldStop(org.apache.pulsar.client.api.Message):METHOD_RETURN_TYPE_CHANGED > > > On 01/09/2022 03:49, Yu Li wrote: > > *bq. a list of all breaking changes between 1.15.0 and the latest 1.15.x > > (or intermediate releases)* > > Yes, this might help users to be more prepared before upgrading, if they > > could know whether need to recompile their applications. Asking about > > 1.14/1.15 since they are the "in service" versions. > > > > But it's totally fine if we're targeting for the future (smile). > > > > Best Regards, > > Yu > > > > > > On Wed, 31 Aug 2022 at 21:41, Chesnay Schepler > wrote: > > > >> Well then someone didnt update the FLIP number as they should! > >> I'll increment the number of this FLIP to 255. > >> > >> On 31/08/2022 15:06, Matthias Pohl wrote: > >>> +1 for bringing this into a consistent state. Thanks, Chesnay. > >>> > >>> nit: There's a conflict between this FLIP-254 and the FLIP-254 on the > >> redis > >>> streams connector. > >>> > >>> On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler > >> wrote: > * backport to 1.14/1.15 > > On 31/08/2022 14:45, Chesnay Schepler wrote: > > @Yu I haven't really considered 1.14/1.15. > > > > What exactly are you interested in; a list of all breaking changes > > between 1.15.0 and the latest 1.15.x (or intermediate releases), > > or are you suggesting to also backport this whole thing to 1.16 > (which > > should be possible)? > > > > On 31/08/2022 13:31, Yu Li wrote: > >> +1 for the FLIP. Thanks for the efforts Chesnay. > >> > >> I believe ensuring binary compatibility for patch releases will also > >> benefit our end users besides the cloud service providers. > >> > >> I'm also wondering about the compatibility checking result after > >> enabling > >> japicmp for all modules with existing patch releases (1.14.x and > >> 1.15.x). > >> Do you already have one with your local customized japicmp or do we > >> need to > >> wait until the works tracked in JIRA are done? (smile) > >> > >> Best Regards, > >> Yu > >> > >> > >> On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf > >> wrote:
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
I compared 1.15.0 and 1.15.2; we overall did a very good job with one outlier. Based on this I'd say we should also apply this FLIP to 1.15. Table connectors: Source-compatible change; relevant if you implement your own DataStream(Scan/Sink)Provider: org.apache.flink.table.connector.sink.DataStreamSinkProvider.consumeDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.datastream.DataStream):METHOD_ABSTRACT_NOW_DEFAULT org.apache.flink.table.connector.source.DataStreamScanProvider.produceDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.environment.StreamExecutionEnvironment):METHOD_ABSTRACT_NOW_DEFAULT Pulsar connector: These are due to adding @Internal in 1.15.2; no effect on compatibility: org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.getMessageId():METHOD_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.valueOf(java.lang.String):METHOD_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.values():METHOD_REMOVED,java.lang.Comparable[java.lang.Comparable]:INTERFACE_REMOVED,java.io.Serializable[java.io.Serializable]:INTERFACE_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:SUPERCLASS_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.TIMESTAMP:FIELD_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.MESSAGE_ID:FIELD_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:CLASS_REMOVED These are straight-up breaking changes (neither source nor binary compatible): org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.seekPosition(org.apache.pulsar.client.api.Consumer):METHOD_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.fromPublishTime(long):METHOD_NEW_DEFAULT org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.seekPosition(java.lang.String,int,org.apache.pulsar.client.api.Consumer):METHOD_REMOVED org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterEventTime(long):METHOD_NEW_DEFAULT,org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterPublishTime(long):METHOD_NEW_DEFAULT org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.shouldStop(org.apache.pulsar.client.api.Message):METHOD_RETURN_TYPE_CHANGED On 01/09/2022 03:49, Yu Li wrote: *bq. a list of all breaking changes between 1.15.0 and the latest 1.15.x (or intermediate releases)* Yes, this might help users to be more prepared before upgrading, if they could know whether need to recompile their applications. Asking about 1.14/1.15 since they are the "in service" versions. But it's totally fine if we're targeting for the future (smile). Best Regards, Yu On Wed, 31 Aug 2022 at 21:41, Chesnay Schepler wrote: Well then someone didnt update the FLIP number as they should! I'll increment the number of this FLIP to 255. On 31/08/2022 15:06, Matthias Pohl wrote: +1 for bringing this into a consistent state. Thanks, Chesnay. nit: There's a conflict between this FLIP-254 and the FLIP-254 on the redis streams connector. On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler wrote: * backport to 1.14/1.15 On 31/08/2022 14:45, Chesnay Schepler wrote: @Yu I haven't really considered 1.14/1.15. What exactly are you interested in; a list of all breaking changes between 1.15.0 and the latest 1.15.x (or intermediate releases), or are you suggesting to also backport this whole thing to 1.16 (which should be possible)? On 31/08/2022 13:31, Yu Li wrote: +1 for the FLIP. Thanks for the efforts Chesnay. I believe ensuring binary compatibility for patch releases will also benefit our end users besides the cloud service providers. I'm also wondering about the compatibility checking result after enabling japicmp for all modules with existing patch releases (1.14.x and 1.15.x). Do you already have one with your local customized japicmp or do we need to wait until the works tracked in JIRA are done? (smile) Best Regards, Yu On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf wrote: Hi Chesnay, thanks for bringing this up and for your research and fixes to japicmd. +1 for the proposal. For Immerok as an Apache Flink cloud service provider it is very valuable to know that our users don't need to upgrade their Jobs when the Flink patch version changes. I am sure the same is true for internal platform teams as well as end users of Apache Flink. Cheers, Konstantin Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < ches...@apache.org>: Hello, I just published a FLIP to guarantee binary compatibility for patch releases. I don't think our current guarantees of source-compatibility are sufficient for patch releases.
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
*bq. a list of all breaking changes between 1.15.0 and the latest 1.15.x (or intermediate releases)* Yes, this might help users to be more prepared before upgrading, if they could know whether need to recompile their applications. Asking about 1.14/1.15 since they are the "in service" versions. But it's totally fine if we're targeting for the future (smile). Best Regards, Yu On Wed, 31 Aug 2022 at 21:41, Chesnay Schepler wrote: > Well then someone didnt update the FLIP number as they should! > I'll increment the number of this FLIP to 255. > > On 31/08/2022 15:06, Matthias Pohl wrote: > > +1 for bringing this into a consistent state. Thanks, Chesnay. > > > > nit: There's a conflict between this FLIP-254 and the FLIP-254 on the > redis > > streams connector. > > > > On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler > wrote: > > > >> * backport to 1.14/1.15 > >> > >> On 31/08/2022 14:45, Chesnay Schepler wrote: > >>> @Yu I haven't really considered 1.14/1.15. > >>> > >>> What exactly are you interested in; a list of all breaking changes > >>> between 1.15.0 and the latest 1.15.x (or intermediate releases), > >>> or are you suggesting to also backport this whole thing to 1.16 (which > >>> should be possible)? > >>> > >>> On 31/08/2022 13:31, Yu Li wrote: > +1 for the FLIP. Thanks for the efforts Chesnay. > > I believe ensuring binary compatibility for patch releases will also > benefit our end users besides the cloud service providers. > > I'm also wondering about the compatibility checking result after > enabling > japicmp for all modules with existing patch releases (1.14.x and > 1.15.x). > Do you already have one with your local customized japicmp or do we > need to > wait until the works tracked in JIRA are done? (smile) > > Best Regards, > Yu > > > On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf > wrote: > > > Hi Chesnay, > > > > thanks for bringing this up and for your research and fixes to > japicmd. > > > > +1 for the proposal. For Immerok as an Apache Flink cloud service > > provider > > it is very valuable to know that our users don't need to upgrade > > their Jobs > > when the Flink patch version changes. I am sure the same is true for > > internal platform teams as well as end users of Apache Flink. > > > > Cheers, > > > > Konstantin > > > > > > > > > > > > Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < > > ches...@apache.org>: > > > >> Hello, > >> > >> I just published a FLIP to guarantee binary compatibility for patch > >> releases. I don't think our current guarantees of > source-compatibility > >> are sufficient for patch releases. > >> > >> > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 > >> Let me know what you think. > >> > >> Regards, > >> Chesnay > >> > > -- > > https://twitter.com/snntrable > > https://github.com/knaufk > > > >> > >
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
Well then someone didnt update the FLIP number as they should! I'll increment the number of this FLIP to 255. On 31/08/2022 15:06, Matthias Pohl wrote: +1 for bringing this into a consistent state. Thanks, Chesnay. nit: There's a conflict between this FLIP-254 and the FLIP-254 on the redis streams connector. On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler wrote: * backport to 1.14/1.15 On 31/08/2022 14:45, Chesnay Schepler wrote: @Yu I haven't really considered 1.14/1.15. What exactly are you interested in; a list of all breaking changes between 1.15.0 and the latest 1.15.x (or intermediate releases), or are you suggesting to also backport this whole thing to 1.16 (which should be possible)? On 31/08/2022 13:31, Yu Li wrote: +1 for the FLIP. Thanks for the efforts Chesnay. I believe ensuring binary compatibility for patch releases will also benefit our end users besides the cloud service providers. I'm also wondering about the compatibility checking result after enabling japicmp for all modules with existing patch releases (1.14.x and 1.15.x). Do you already have one with your local customized japicmp or do we need to wait until the works tracked in JIRA are done? (smile) Best Regards, Yu On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf wrote: Hi Chesnay, thanks for bringing this up and for your research and fixes to japicmd. +1 for the proposal. For Immerok as an Apache Flink cloud service provider it is very valuable to know that our users don't need to upgrade their Jobs when the Flink patch version changes. I am sure the same is true for internal platform teams as well as end users of Apache Flink. Cheers, Konstantin Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < ches...@apache.org>: Hello, I just published a FLIP to guarantee binary compatibility for patch releases. I don't think our current guarantees of source-compatibility are sufficient for patch releases. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 Let me know what you think. Regards, Chesnay -- https://twitter.com/snntrable https://github.com/knaufk
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
+1 for bringing this into a consistent state. Thanks, Chesnay. nit: There's a conflict between this FLIP-254 and the FLIP-254 on the redis streams connector. On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler wrote: > * backport to 1.14/1.15 > > On 31/08/2022 14:45, Chesnay Schepler wrote: > > @Yu I haven't really considered 1.14/1.15. > > > > What exactly are you interested in; a list of all breaking changes > > between 1.15.0 and the latest 1.15.x (or intermediate releases), > > or are you suggesting to also backport this whole thing to 1.16 (which > > should be possible)? > > > > On 31/08/2022 13:31, Yu Li wrote: > >> +1 for the FLIP. Thanks for the efforts Chesnay. > >> > >> I believe ensuring binary compatibility for patch releases will also > >> benefit our end users besides the cloud service providers. > >> > >> I'm also wondering about the compatibility checking result after > >> enabling > >> japicmp for all modules with existing patch releases (1.14.x and > >> 1.15.x). > >> Do you already have one with your local customized japicmp or do we > >> need to > >> wait until the works tracked in JIRA are done? (smile) > >> > >> Best Regards, > >> Yu > >> > >> > >> On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf > >> wrote: > >> > >>> Hi Chesnay, > >>> > >>> thanks for bringing this up and for your research and fixes to japicmd. > >>> > >>> +1 for the proposal. For Immerok as an Apache Flink cloud service > >>> provider > >>> it is very valuable to know that our users don't need to upgrade > >>> their Jobs > >>> when the Flink patch version changes. I am sure the same is true for > >>> internal platform teams as well as end users of Apache Flink. > >>> > >>> Cheers, > >>> > >>> Konstantin > >>> > >>> > >>> > >>> > >>> > >>> Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < > >>> ches...@apache.org>: > >>> > Hello, > > I just published a FLIP to guarantee binary compatibility for patch > releases. I don't think our current guarantees of source-compatibility > are sufficient for patch releases. > > > >>> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 > >>> > Let me know what you think. > > Regards, > Chesnay > > >>> > >>> -- > >>> https://twitter.com/snntrable > >>> https://github.com/knaufk > >>> > > > >
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
* backport to 1.14/1.15 On 31/08/2022 14:45, Chesnay Schepler wrote: @Yu I haven't really considered 1.14/1.15. What exactly are you interested in; a list of all breaking changes between 1.15.0 and the latest 1.15.x (or intermediate releases), or are you suggesting to also backport this whole thing to 1.16 (which should be possible)? On 31/08/2022 13:31, Yu Li wrote: +1 for the FLIP. Thanks for the efforts Chesnay. I believe ensuring binary compatibility for patch releases will also benefit our end users besides the cloud service providers. I'm also wondering about the compatibility checking result after enabling japicmp for all modules with existing patch releases (1.14.x and 1.15.x). Do you already have one with your local customized japicmp or do we need to wait until the works tracked in JIRA are done? (smile) Best Regards, Yu On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf wrote: Hi Chesnay, thanks for bringing this up and for your research and fixes to japicmd. +1 for the proposal. For Immerok as an Apache Flink cloud service provider it is very valuable to know that our users don't need to upgrade their Jobs when the Flink patch version changes. I am sure the same is true for internal platform teams as well as end users of Apache Flink. Cheers, Konstantin Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < ches...@apache.org>: Hello, I just published a FLIP to guarantee binary compatibility for patch releases. I don't think our current guarantees of source-compatibility are sufficient for patch releases. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 Let me know what you think. Regards, Chesnay -- https://twitter.com/snntrable https://github.com/knaufk
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
@Yu I haven't really considered 1.14/1.15. What exactly are you interested in; a list of all breaking changes between 1.15.0 and the latest 1.15.x (or intermediate releases), or are you suggesting to also backport this whole thing to 1.16 (which should be possible)? On 31/08/2022 13:31, Yu Li wrote: +1 for the FLIP. Thanks for the efforts Chesnay. I believe ensuring binary compatibility for patch releases will also benefit our end users besides the cloud service providers. I'm also wondering about the compatibility checking result after enabling japicmp for all modules with existing patch releases (1.14.x and 1.15.x). Do you already have one with your local customized japicmp or do we need to wait until the works tracked in JIRA are done? (smile) Best Regards, Yu On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf wrote: Hi Chesnay, thanks for bringing this up and for your research and fixes to japicmd. +1 for the proposal. For Immerok as an Apache Flink cloud service provider it is very valuable to know that our users don't need to upgrade their Jobs when the Flink patch version changes. I am sure the same is true for internal platform teams as well as end users of Apache Flink. Cheers, Konstantin Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < ches...@apache.org>: Hello, I just published a FLIP to guarantee binary compatibility for patch releases. I don't think our current guarantees of source-compatibility are sufficient for patch releases. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 Let me know what you think. Regards, Chesnay -- https://twitter.com/snntrable https://github.com/knaufk
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
+1 for the FLIP. Thanks for the efforts Chesnay. I believe ensuring binary compatibility for patch releases will also benefit our end users besides the cloud service providers. I'm also wondering about the compatibility checking result after enabling japicmp for all modules with existing patch releases (1.14.x and 1.15.x). Do you already have one with your local customized japicmp or do we need to wait until the works tracked in JIRA are done? (smile) Best Regards, Yu On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf wrote: > Hi Chesnay, > > thanks for bringing this up and for your research and fixes to japicmd. > > +1 for the proposal. For Immerok as an Apache Flink cloud service provider > it is very valuable to know that our users don't need to upgrade their Jobs > when the Flink patch version changes. I am sure the same is true for > internal platform teams as well as end users of Apache Flink. > > Cheers, > > Konstantin > > > > > > Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < > ches...@apache.org>: > > > Hello, > > > > I just published a FLIP to guarantee binary compatibility for patch > > releases. I don't think our current guarantees of source-compatibility > > are sufficient for patch releases. > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 > > > > Let me know what you think. > > > > Regards, > > Chesnay > > > > > -- > https://twitter.com/snntrable > https://github.com/knaufk >
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
Hi Chesnay, thanks for bringing this up and for your research and fixes to japicmd. +1 for the proposal. For Immerok as an Apache Flink cloud service provider it is very valuable to know that our users don't need to upgrade their Jobs when the Flink patch version changes. I am sure the same is true for internal platform teams as well as end users of Apache Flink. Cheers, Konstantin Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler < ches...@apache.org>: > Hello, > > I just published a FLIP to guarantee binary compatibility for patch > releases. I don't think our current guarantees of source-compatibility > are sufficient for patch releases. > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 > > Let me know what you think. > > Regards, > Chesnay > -- https://twitter.com/snntrable https://github.com/knaufk
[DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
Hello, I just published a FLIP to guarantee binary compatibility for patch releases. I don't think our current guarantees of source-compatibility are sufficient for patch releases. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857 Let me know what you think. Regards, Chesnay