Hi everyone, I'm restarting this thread [1] with a new subject, given that Flink 1.14.1 was a (cancelled) emergency release for the Log4j update and we've released Flink 1.14.2 as an emergency release for Log4j updates [2].
To give an update, this is the current blocker for Flink 1.14.3: * https://issues.apache.org/jira/browse/FLINK-25132 - KafkaSource cannot work with object-reusing DeserializationSchema -> @renqs...@gmail.com <renqs...@gmail.com> can you provide an ETA for this ticket? There are two critical tickets open for Flink 1.14.3. That means that if the above ticket is resolved, these two will not block the release. If we can merge them in before the above ticket is completed, that's a bonus. * https://issues.apache.org/jira/browse/FLINK-25199 - fromValues does not emit final MAX watermark -> @Marios Trivyzas <mat...@gmail.com> any update or thoughts on this? * https://issues.apache.org/jira/browse/FLINK-25227 - Comparing the equality of the same (boxed) numeric values returns false -> @Caizhi Weng <tsreape...@gmail.com> any update or thoughts on this? Best regards, Martijn [1] https://lists.apache.org/thread/r0xhs9x01k8hnm0hyq2kk4ptrhkzgdw9 [2] https://flink.apache.org/news/2021/12/16/log4j-patch-releases.html On Thu, 9 Dec 2021 at 17:21, David Morávek <d...@apache.org> wrote: > Hi Martijn, I've just opened a backport PR [1] for FLINK-23946 [2]. > > [1] https://github.com/apache/flink/pull/18066 > [2] https://issues.apache.org/jira/browse/FLINK-23946 > > Best, > D. > > On Thu, Dec 9, 2021 at 4:59 PM Fabian Paul <fp...@apache.org> wrote: > >> Actually I meant https://issues.apache.org/jira/browse/FLINK-25126 >> sorry for the confusion. >> >> On Thu, Dec 9, 2021 at 4:55 PM Fabian Paul <fp...@apache.org> wrote: >> > >> > Hi Martijn, >> > >> > I just opened the backport for >> > https://issues.apache.org/jira/browse/FLINK-25132. The changes are >> > already approved I only wait for a green Azure build. >> > >> > Best, >> > Fabian >> > >> > On Thu, Dec 9, 2021 at 4:01 PM Martijn Visser <mart...@ververica.com> >> wrote: >> > > >> > > Hi all, >> > > >> > > Thanks for the fixes Jingsong and Zhu! >> > > >> > > That means that we still have the following tickets open: >> > > >> > > * https://issues.apache.org/jira/browse/FLINK-23946 - Application >> mode >> > > fails fatally when being shut down -> A PR is there, just pending a >> review. >> > > * https://issues.apache.org/jira/browse/FLINK-25126 - Kafka >> connector tries >> > > to commit aborted transaction in batch mode -> I believe this is >> pending a >> > > backport, correct @fp...@apache.org <fp...@apache.org> ? >> > > * https://issues.apache.org/jira/browse/FLINK-25132 - KafkaSource >> cannot >> > > work with object-reusing DeserializationSchema -> @renqs...@gmail.com >> > > <renqs...@gmail.com> can you provide an ETA for this ticket? >> > > * https://issues.apache.org/jira/browse/FLINK-25199 - fromValues >> does not >> > > emit final MAX watermark -> @Marios Trivyzas <mat...@gmail.com> can >> you >> > > provide an ETA for this ticket? >> > > * https://issues.apache.org/jira/browse/FLINK-25227 - Comparing the >> > > equality of the same (boxed) numeric values returns false -> @Caizhi >> Weng >> > > <tsreape...@gmail.com> mentioned that a fix is planned for >> today/tomorrow. >> > > I am wondering if this is indeed a blocker for 1.14.1, but given that >> there >> > > are still some blockers waiting to be merged we could probably >> include it. >> > > >> > > Best regards, >> > > >> > > Martijn >> > > >> > > On Thu, 9 Dec 2021 at 07:31, Caizhi Weng <tsreape...@gmail.com> >> wrote: >> > > >> > > > Hi devs! >> > > > >> > > > Sorry for the interruptions, but I just found an issue [1] (which I >> think >> > > > is a blocking one) in every Flink version, including Flink 1.14.1. >> > > > >> > > > For Flink < 1.15, this issue will cause incorrect result when user >> cast >> > > > two strings to numerics and compare the numerics. >> > > > >> > > > I'm planning for a quick fix today or tomorrow. >> > > > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-25227 >> > > > >> > > > Zhu Zhu <reed...@gmail.com> 于2021年12月9日周四 10:48写道： >> > > > >> > > >> update: backport of FLINK-19142 is done >> > > >> >> > > >> Thanks, >> > > >> Zhu >> > > >> >> > > >> Zhu Zhu <reed...@gmail.com> 于2021年12月8日周三 19:35写道： >> > > >> >> > > >> > Hi Martijn, >> > > >> > >> > > >> > I'd like to backport the fix of FLINK-19142 to 1.14.1. >> > > >> > The backport is in progress. >> > > >> > Will update it here when it is done. >> > > >> > >> > > >> > Thanks, >> > > >> > Zhu >> > > >> > >> > > >> > Jingsong Li <jingsongl...@gmail.com> 于2021年12月8日周三 10:33写道： >> > > >> > >> > > >> >> Hi Martijn, >> > > >> >> >> > > >> >> We just created a cherry-pick pull-request for >> > > >> >> https://issues.apache.org/jira/browse/FLINK-20370 >> > > >> >> We could finish it as soon as possible. >> > > >> >> >> > > >> >> Best, >> > > >> >> Jingsong >> > > >> >> >> > > >> >> On Fri, Dec 3, 2021 at 10:25 PM Fabian Paul <fp...@apache.org> >> wrote: >> > > >> >> > >> > > >> >> > I just opened a PR for >> > > >> >> > https://issues.apache.org/jira/browse/FLINK-25126 I'll >> expect to >> > > >> merge >> > > >> >> > it sometime next week. >> > > >> >> > >> > > >> >> > Best, >> > > >> >> > Fabian >> > > >> >> > >> > > >> >> > On Fri, Dec 3, 2021 at 10:49 AM Martijn Visser < >> > > >> mart...@ververica.com> >> > > >> >> wrote: >> > > >> >> > > >> > > >> >> > > Hi all, >> > > >> >> > > >> > > >> >> > > Just a status update on the open blockers for 1.14.1: >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-22113 - >> UniqueKey >> > > >> >> constraint is lost with multiple sources join in SQL -> I >> believe most >> > > >> >> review comments have been fixed and it's just the final review >> remarks >> > > >> >> before it's ready. >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-23946 - >> Application >> > > >> >> mode fails fatally when being shut down -> @David Morávek can >> you >> > > >> provide >> > > >> >> an update? >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-25022 - >> ClassLoader >> > > >> >> leak with ThreadLocals on the JM when submitting a job through >> the >> > > >> REST API >> > > >> >> -> I think this is just pending on a merge to master and then >> creating >> > > >> a >> > > >> >> backport? >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-25126 - Kafka >> > > >> >> connector tries to commit aborted transaction in batch mode -> >> This is >> > > >> a >> > > >> >> new blocker. @fp...@apache.org can you give an update? >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-25132 - >> KafkaSource >> > > >> >> cannot work with object-reusing DeserializationSchema -> >> There's a PR >> > > >> >> that's being reviewed and then needs a backport. >> > > >> >> > > >> > > >> >> > > It would be great if we can finish all these blockers next >> week to >> > > >> >> start a release. Do the assignees think that's realistic? >> > > >> >> > > >> > > >> >> > > Best regards, >> > > >> >> > > >> > > >> >> > > Martijn >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > On Thu, 2 Dec 2021 at 14:25, Marios Trivyzas < >> mat...@gmail.com> >> > > >> >> wrote: >> > > >> >> > >> >> > > >> >> > >> https://issues.apache.org/jira/browse/FLINK-22113 will >> be merged >> > > >> >> today (most probably) >> > > >> >> > >> >> > > >> >> > >> On Mon, Nov 29, 2021 at 10:15 AM Martijn Visser < >> > > >> >> mart...@ververica.com> wrote: >> > > >> >> > >>> >> > > >> >> > >>> Thanks all for the updates! To summarize, these are open >> tickets >> > > >> >> that are considered blockers for Flink 1.14.1: >> > > >> >> > >>> >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-22113 - >> UniqueKey >> > > >> >> constraint is lost with multiple sources join in SQL -> @Marios >> > > >> Trivyzas >> > > >> >> can you give an estimate when you expect this to be resolved? >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-23946 - >> > > >> Application >> > > >> >> mode fails fatally when being shut down -> A patch is being >> prepared. >> > > >> >> @David Morávek do you have an estimate when this patch will be >> there? >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-24596 - >> Bugs in >> > > >> >> sink.buffer-flush before upsert-kafka -> @fp...@apache.org has >> > > >> provided >> > > >> >> a PR is there, so I suspect it would take a couple of days >> before this >> > > >> is >> > > >> >> merged. >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-25022 - >> > > >> ClassLoader >> > > >> >> leak with ThreadLocals on the JM when submitting a job through >> the >> > > >> REST API >> > > >> >> -> @Chesnay Schepler has provided a PR, so I suspect it would >> also just >> > > >> >> take a couple of days before this is merged. >> > > >> >> > >>> >> > > >> >> > >>> Is there anyone who can help me with creating the actual >> release >> > > >> >> when these tickets are resolved >> > > >> >> > >>> >> > > >> >> > >>> Best regards, >> > > >> >> > >>> >> > > >> >> > >>> Martijn >> > > >> >> > >>> >> > > >> >> > >>> >> > > >> >> > >>> On Fri, 26 Nov 2021 at 12:08, Chesnay Schepler < >> > > >> ches...@apache.org> >> > > >> >> wrote: >> > > >> >> > >>>> >> > > >> >> > >>>> FLINK-25022: I will open a PR later today, and it should >> be >> > > >> easy to >> > > >> >> > >>>> backport. >> > > >> >> > >>>> FLINK-25027: Unlikely to make it for 1.14.1; I also >> wouldn't >> > > >> >> consider it >> > > >> >> > >>>> a blocker >> > > >> >> > >>>> >> > > >> >> > >>>> On 24/11/2021 19:40, Martijn Visser wrote: >> > > >> >> > >>>> > Hi all, >> > > >> >> > >>>> > >> > > >> >> > >>>> > I would like to start a discussion on releasing Flink >> 1.14.1. >> > > >> >> Flink 1.14 >> > > >> >> > >>>> > was released on the 29th of September [1] and so far >> 107 >> > > >> issues >> > > >> >> have been >> > > >> >> > >>>> > resolved, including multiple blockers and critical >> priorities >> > > >> >> [2]. >> > > >> >> > >>>> > >> > > >> >> > >>>> > There are currently 169 open tickets which contain a >> > > >> fixVersion >> > > >> >> for 1.14.1 >> > > >> >> > >>>> > [3]. I'm including the ones that are currently marked >> as >> > > >> >> critical or a >> > > >> >> > >>>> > blocker to verify if these should be included in Flink >> 1.14.1. >> > > >> >> It would be >> > > >> >> > >>>> > great if those that are assigned or working on one or >> more of >> > > >> >> these tickets >> > > >> >> > >>>> > can give an update on its status. >> > > >> >> > >>>> > >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-24543 - >> > > >> Zookeeper >> > > >> >> connection >> > > >> >> > >>>> > issue causes inconsistent state in Flink -> I think >> this >> > > >> depends >> > > >> >> on the >> > > >> >> > >>>> > outcome of dropping Zookeeper 3.4 as was proposed on >> the Dev >> > > >> >> mailing list >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-25027 - >> Allow >> > > >> GC >> > > >> >> of a >> > > >> >> > >>>> > finished job's JobMaster before the slot timeout is >> reached >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-25022 - >> > > >> >> ClassLoader leak with >> > > >> >> > >>>> > ThreadLocals on the JM when submitting a job through >> the REST >> > > >> API >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-24789 - >> > > >> >> IllegalStateException >> > > >> >> > >>>> > with CheckpointCleaner being closed already >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-24328 - >> Long >> > > >> term >> > > >> >> fix for >> > > >> >> > >>>> > receiving new buffer size before network reader >> configured -> >> > > >> >> I'm not sure >> > > >> >> > >>>> > if this would end up in Flink 1.14.1, I think it's >> more likely >> > > >> >> that it >> > > >> >> > >>>> > would be Flink 1.15. Anton/Dawid, could you confirm >> this? >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-23946 - >> > > >> >> Application mode >> > > >> >> > >>>> > fails fatally when being shut down -> This depends on >> > > >> >> > >>>> > https://issues.apache.org/jira/browse/FLINK-24038 and >> I don't >> > > >> >> see much >> > > >> >> > >>>> > happening there, so I also expect that this would move >> to >> > > >> Flink >> > > >> >> 1.15. >> > > >> >> > >>>> > David, could you confirm? >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-22113 - >> > > >> UniqueKey >> > > >> >> constraint >> > > >> >> > >>>> > is lost with multiple sources join in SQL >> > > >> >> > >>>> > * https://issues.apache.org/jira/browse/FLINK-21788 - >> Throw >> > > >> >> > >>>> > PartitionNotFoundException if the partition file has >> been lost >> > > >> >> for blocking >> > > >> >> > >>>> > shuffle -> I'm also expecting that this would move to >> Flink >> > > >> >> 1.15, can you >> > > >> >> > >>>> > confirm Yingjie ? >> > > >> >> > >>>> > >> > > >> >> > >>>> > There are quite some other tickets that I've excluded >> from >> > > >> this >> > > >> >> list, >> > > >> >> > >>>> > because they are either test instabilities or are not >> > > >> depending >> > > >> >> on a Flink >> > > >> >> > >>>> > release to be resolved. >> > > >> >> > >>>> > >> > > >> >> > >>>> > Note: there are quite a few test instabilities in the >> list and >> > > >> >> help on >> > > >> >> > >>>> > those is always appreciated. You can check all >> unassigned >> > > >> tickets >> > > >> >> > >>>> > instabilities in Jira [4]. >> > > >> >> > >>>> > >> > > >> >> > >>>> > Are there any other open tickets that we should wait >> for? Is >> > > >> >> there a PMC >> > > >> >> > >>>> > member who would like to manage the release? I'm more >> than >> > > >> happy >> > > >> >> to help >> > > >> >> > >>>> > with monitoring the status of the tickets. >> > > >> >> > >>>> > >> > > >> >> > >>>> > Best regards, >> > > >> >> > >>>> > >> > > >> >> > >>>> > Martijn >> > > >> >> > >>>> > >> > > >> >> > >>>> > [1] >> > > >> https://flink.apache.org/news/2021/09/29/release-1.14.0.html >> > > >> >> > >>>> > [2] >> > > >> >> > >>>> > >> > > >> >> >> > > >> >> https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC >> > > >> >> > >>>> > [3] >> > > >> >> > >>>> > >> > > >> >> >> > > >> >> https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC >> > > >> >> > >>>> > >> > > >> >> > >>>> > [4] >> > > >> >> > >>>> > >> > > >> >> >> > > >> >> https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20AND%20labels%20%3D%20test-stability%20AND%20assignee%20in%20(EMPTY)%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC >> > > >> >> > >>>> > >> > > >> >> > >>>> > Martijn Visser | Product Manager >> > > >> >> > >>>> > >> > > >> >> > >>>> > mart...@ververica.com >> > > >> >> > >>>> > >> > > >> >> > >>>> > <https://www.ververica.com/> >> > > >> >> > >>>> > >> > > >> >> > >>>> > >> > > >> >> > >>>> > Follow us @VervericaData >> > > >> >> > >>>> > >> > > >> >> > >>>> > -- >> > > >> >> > >>>> > >> > > >> >> > >>>> > Join Flink Forward <https://flink-forward.org/> - The >> Apache >> > > >> >> Flink >> > > >> >> > >>>> > Conference >> > > >> >> > >>>> > >> > > >> >> > >>>> > Stream Processing | Event Driven | Real Time >> > > >> >> > >>>> > >> > > >> >> > >>>> >> > > >> >> > >> >> > > >> >> > >> >> > > >> >> > >> -- >> > > >> >> > >> Marios >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> -- >> > > >> >> Best, Jingsong Lee >> > > >> >> >> > > >> > >> > > >> >> > > > >> >
