Hi Martijn, Thank you for reviving this thread. I have opened a PR to backport the Log4j Upgrade to 2.17.0 [1] to Flink 1.14 just now and am waiting for CI to pass.
Cheers, Konstantin [1] https://issues.apache.org/jira/browse/FLINK-25375 On Tue, Dec 21, 2021 at 5:26 PM David Morávek <d...@apache.org> wrote: > Hi Martijn, > > I'm also working on fixing the FLINK-25271 [1], but there's probably no > need to block a release on this as it only addresses test flakiness > > [1] https://issues.apache.org/jira/browse/FLINK-25271 > > Best, > D. > > On Tue, Dec 21, 2021 at 11:27 AM Martijn Visser <mart...@ververica.com> > wrote: > > > Hi everyone, > > > > I'm restarting this thread [1] with a new subject, given that Flink > 1.14.1 > > was a (cancelled) emergency release for the Log4j update and we've > released > > Flink 1.14.2 as an emergency release for Log4j updates [2]. > > > > To give an update, this is the current blocker for Flink 1.14.3: > > > > * https://issues.apache.org/jira/browse/FLINK-25132 - KafkaSource cannot > > work with object-reusing DeserializationSchema -> @renqs...@gmail.com > > <renqs...@gmail.com> can you provide an ETA for this ticket? > > > > There are two critical tickets open for Flink 1.14.3. That means that if > > the above ticket is resolved, these two will not block the release. If we > > can merge them in before the above ticket is completed, that's a bonus. > > > > * https://issues.apache.org/jira/browse/FLINK-25199 - fromValues does > not > > emit final MAX watermark -> @Marios Trivyzas <mat...@gmail.com> any > update > > or thoughts on this? > > * https://issues.apache.org/jira/browse/FLINK-25227 - Comparing the > > equality of the same (boxed) numeric values returns false -> @Caizhi Weng > > <tsreape...@gmail.com> any update or thoughts on this? > > > > Best regards, > > > > Martijn > > > > [1] https://lists.apache.org/thread/r0xhs9x01k8hnm0hyq2kk4ptrhkzgdw9 > > [2] https://flink.apache.org/news/2021/12/16/log4j-patch-releases.html > > > > On Thu, 9 Dec 2021 at 17:21, David Morávek <d...@apache.org> wrote: > > > > > Hi Martijn, I've just opened a backport PR [1] for FLINK-23946 [2]. > > > > > > [1] https://github.com/apache/flink/pull/18066 > > > [2] https://issues.apache.org/jira/browse/FLINK-23946 > > > > > > Best, > > > D. > > > > > > On Thu, Dec 9, 2021 at 4:59 PM Fabian Paul <fp...@apache.org> wrote: > > > > > >> Actually I meant https://issues.apache.org/jira/browse/FLINK-25126 > > >> sorry for the confusion. > > >> > > >> On Thu, Dec 9, 2021 at 4:55 PM Fabian Paul <fp...@apache.org> wrote: > > >> > > > >> > Hi Martijn, > > >> > > > >> > I just opened the backport for > > >> > https://issues.apache.org/jira/browse/FLINK-25132. The changes are > > >> > already approved I only wait for a green Azure build. > > >> > > > >> > Best, > > >> > Fabian > > >> > > > >> > On Thu, Dec 9, 2021 at 4:01 PM Martijn Visser < > mart...@ververica.com> > > >> wrote: > > >> > > > > >> > > Hi all, > > >> > > > > >> > > Thanks for the fixes Jingsong and Zhu! > > >> > > > > >> > > That means that we still have the following tickets open: > > >> > > > > >> > > * https://issues.apache.org/jira/browse/FLINK-23946 - Application > > >> mode > > >> > > fails fatally when being shut down -> A PR is there, just pending > a > > >> review. > > >> > > * https://issues.apache.org/jira/browse/FLINK-25126 - Kafka > > >> connector tries > > >> > > to commit aborted transaction in batch mode -> I believe this is > > >> pending a > > >> > > backport, correct @fp...@apache.org <fp...@apache.org> ? > > >> > > * https://issues.apache.org/jira/browse/FLINK-25132 - KafkaSource > > >> cannot > > >> > > work with object-reusing DeserializationSchema -> @ > > renqs...@gmail.com > > >> > > <renqs...@gmail.com> can you provide an ETA for this ticket? > > >> > > * https://issues.apache.org/jira/browse/FLINK-25199 - fromValues > > >> does not > > >> > > emit final MAX watermark -> @Marios Trivyzas <mat...@gmail.com> > can > > >> you > > >> > > provide an ETA for this ticket? > > >> > > * https://issues.apache.org/jira/browse/FLINK-25227 - Comparing > the > > >> > > equality of the same (boxed) numeric values returns false -> > @Caizhi > > >> Weng > > >> > > <tsreape...@gmail.com> mentioned that a fix is planned for > > >> today/tomorrow. > > >> > > I am wondering if this is indeed a blocker for 1.14.1, but given > > that > > >> there > > >> > > are still some blockers waiting to be merged we could probably > > >> include it. > > >> > > > > >> > > Best regards, > > >> > > > > >> > > Martijn > > >> > > > > >> > > On Thu, 9 Dec 2021 at 07:31, Caizhi Weng <tsreape...@gmail.com> > > >> wrote: > > >> > > > > >> > > > Hi devs! > > >> > > > > > >> > > > Sorry for the interruptions, but I just found an issue [1] > (which > > I > > >> think > > >> > > > is a blocking one) in every Flink version, including Flink > 1.14.1. > > >> > > > > > >> > > > For Flink < 1.15, this issue will cause incorrect result when > user > > >> cast > > >> > > > two strings to numerics and compare the numerics. > > >> > > > > > >> > > > I'm planning for a quick fix today or tomorrow. > > >> > > > > > >> > > > [1] https://issues.apache.org/jira/browse/FLINK-25227 > > >> > > > > > >> > > > Zhu Zhu <reed...@gmail.com> 于2021年12月9日周四 10:48写道： > > >> > > > > > >> > > >> update: backport of FLINK-19142 is done > > >> > > >> > > >> > > >> Thanks, > > >> > > >> Zhu > > >> > > >> > > >> > > >> Zhu Zhu <reed...@gmail.com> 于2021年12月8日周三 19:35写道： > > >> > > >> > > >> > > >> > Hi Martijn, > > >> > > >> > > > >> > > >> > I'd like to backport the fix of FLINK-19142 to 1.14.1. > > >> > > >> > The backport is in progress. > > >> > > >> > Will update it here when it is done. > > >> > > >> > > > >> > > >> > Thanks, > > >> > > >> > Zhu > > >> > > >> > > > >> > > >> > Jingsong Li <jingsongl...@gmail.com> 于2021年12月8日周三 10:33写道： > > >> > > >> > > > >> > > >> >> Hi Martijn, > > >> > > >> >> > > >> > > >> >> We just created a cherry-pick pull-request for > > >> > > >> >> https://issues.apache.org/jira/browse/FLINK-20370 > > >> > > >> >> We could finish it as soon as possible. > > >> > > >> >> > > >> > > >> >> Best, > > >> > > >> >> Jingsong > > >> > > >> >> > > >> > > >> >> On Fri, Dec 3, 2021 at 10:25 PM Fabian Paul < > fp...@apache.org > > > > > >> wrote: > > >> > > >> >> > > > >> > > >> >> > I just opened a PR for > > >> > > >> >> > https://issues.apache.org/jira/browse/FLINK-25126 I'll > > >> expect to > > >> > > >> merge > > >> > > >> >> > it sometime next week. > > >> > > >> >> > > > >> > > >> >> > Best, > > >> > > >> >> > Fabian > > >> > > >> >> > > > >> > > >> >> > On Fri, Dec 3, 2021 at 10:49 AM Martijn Visser < > > >> > > >> mart...@ververica.com> > > >> > > >> >> wrote: > > >> > > >> >> > > > > >> > > >> >> > > Hi all, > > >> > > >> >> > > > > >> > > >> >> > > Just a status update on the open blockers for 1.14.1: > > >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-22113 - > > >> UniqueKey > > >> > > >> >> constraint is lost with multiple sources join in SQL -> I > > >> believe most > > >> > > >> >> review comments have been fixed and it's just the final > review > > >> remarks > > >> > > >> >> before it's ready. > > >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-23946 - > > >> Application > > >> > > >> >> mode fails fatally when being shut down -> @David Morávek > can > > >> you > > >> > > >> provide > > >> > > >> >> an update? > > >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-25022 - > > >> ClassLoader > > >> > > >> >> leak with ThreadLocals on the JM when submitting a job > through > > >> the > > >> > > >> REST API > > >> > > >> >> -> I think this is just pending on a merge to master and > then > > >> creating > > >> > > >> a > > >> > > >> >> backport? > > >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-25126 - > > Kafka > > >> > > >> >> connector tries to commit aborted transaction in batch mode > -> > > >> This is > > >> > > >> a > > >> > > >> >> new blocker. @fp...@apache.org can you give an update? > > >> > > >> >> > > * https://issues.apache.org/jira/browse/FLINK-25132 - > > >> KafkaSource > > >> > > >> >> cannot work with object-reusing DeserializationSchema -> > > >> There's a PR > > >> > > >> >> that's being reviewed and then needs a backport. > > >> > > >> >> > > > > >> > > >> >> > > It would be great if we can finish all these blockers > next > > >> week to > > >> > > >> >> start a release. Do the assignees think that's realistic? > > >> > > >> >> > > > > >> > > >> >> > > Best regards, > > >> > > >> >> > > > > >> > > >> >> > > Martijn > > >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > > > >> > > >> >> > > On Thu, 2 Dec 2021 at 14:25, Marios Trivyzas < > > >> mat...@gmail.com> > > >> > > >> >> wrote: > > >> > > >> >> > >> > > >> > > >> >> > >> https://issues.apache.org/jira/browse/FLINK-22113 > will > > >> be merged > > >> > > >> >> today (most probably) > > >> > > >> >> > >> > > >> > > >> >> > >> On Mon, Nov 29, 2021 at 10:15 AM Martijn Visser < > > >> > > >> >> mart...@ververica.com> wrote: > > >> > > >> >> > >>> > > >> > > >> >> > >>> Thanks all for the updates! To summarize, these are > open > > >> tickets > > >> > > >> >> that are considered blockers for Flink 1.14.1: > > >> > > >> >> > >>> > > >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-22113 - > > >> UniqueKey > > >> > > >> >> constraint is lost with multiple sources join in SQL -> > > @Marios > > >> > > >> Trivyzas > > >> > > >> >> can you give an estimate when you expect this to be > resolved? > > >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-23946 - > > >> > > >> Application > > >> > > >> >> mode fails fatally when being shut down -> A patch is being > > >> prepared. > > >> > > >> >> @David Morávek do you have an estimate when this patch will > be > > >> there? > > >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-24596 - > > >> Bugs in > > >> > > >> >> sink.buffer-flush before upsert-kafka -> @fp...@apache.org > > has > > >> > > >> provided > > >> > > >> >> a PR is there, so I suspect it would take a couple of days > > >> before this > > >> > > >> is > > >> > > >> >> merged. > > >> > > >> >> > >>> * https://issues.apache.org/jira/browse/FLINK-25022 - > > >> > > >> ClassLoader > > >> > > >> >> leak with ThreadLocals on the JM when submitting a job > through > > >> the > > >> > > >> REST API > > >> > > >> >> -> @Chesnay Schepler has provided a PR, so I suspect it > would > > >> also just > > >> > > >> >> take a couple of days before this is merged. > > >> > > >> >> > >>> > > >> > > >> >> > >>> Is there anyone who can help me with creating the > actual > > >> release > > >> > > >> >> when these tickets are resolved > > >> > > >> >> > >>> > > >> > > >> >> > >>> Best regards, > > >> > > >> >> > >>> > > >> > > >> >> > >>> Martijn > > >> > > >> >> > >>> > > >> > > >> >> > >>> > > >> > > >> >> > >>> On Fri, 26 Nov 2021 at 12:08, Chesnay Schepler < > > >> > > >> ches...@apache.org> > > >> > > >> >> wrote: > > >> > > >> >> > >>>> > > >> > > >> >> > >>>> FLINK-25022: I will open a PR later today, and it > > should > > >> be > > >> > > >> easy to > > >> > > >> >> > >>>> backport. > > >> > > >> >> > >>>> FLINK-25027: Unlikely to make it for 1.14.1; I also > > >> wouldn't > > >> > > >> >> consider it > > >> > > >> >> > >>>> a blocker > > >> > > >> >> > >>>> > > >> > > >> >> > >>>> On 24/11/2021 19:40, Martijn Visser wrote: > > >> > > >> >> > >>>> > Hi all, > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > I would like to start a discussion on releasing > Flink > > >> 1.14.1. > > >> > > >> >> Flink 1.14 > > >> > > >> >> > >>>> > was released on the 29th of September [1] and so > far > > >> 107 > > >> > > >> issues > > >> > > >> >> have been > > >> > > >> >> > >>>> > resolved, including multiple blockers and critical > > >> priorities > > >> > > >> >> [2]. > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > There are currently 169 open tickets which contain > a > > >> > > >> fixVersion > > >> > > >> >> for 1.14.1 > > >> > > >> >> > >>>> > [3]. I'm including the ones that are currently > marked > > >> as > > >> > > >> >> critical or a > > >> > > >> >> > >>>> > blocker to verify if these should be included in > > Flink > > >> 1.14.1. > > >> > > >> >> It would be > > >> > > >> >> > >>>> > great if those that are assigned or working on one > or > > >> more of > > >> > > >> >> these tickets > > >> > > >> >> > >>>> > can give an update on its status. > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-24543 > > - > > >> > > >> Zookeeper > > >> > > >> >> connection > > >> > > >> >> > >>>> > issue causes inconsistent state in Flink -> I think > > >> this > > >> > > >> depends > > >> > > >> >> on the > > >> > > >> >> > >>>> > outcome of dropping Zookeeper 3.4 as was proposed > on > > >> the Dev > > >> > > >> >> mailing list > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-25027 > > - > > >> Allow > > >> > > >> GC > > >> > > >> >> of a > > >> > > >> >> > >>>> > finished job's JobMaster before the slot timeout is > > >> reached > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-25022 > > - > > >> > > >> >> ClassLoader leak with > > >> > > >> >> > >>>> > ThreadLocals on the JM when submitting a job > through > > >> the REST > > >> > > >> API > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-24789 > > - > > >> > > >> >> IllegalStateException > > >> > > >> >> > >>>> > with CheckpointCleaner being closed already > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-24328 > > - > > >> Long > > >> > > >> term > > >> > > >> >> fix for > > >> > > >> >> > >>>> > receiving new buffer size before network reader > > >> configured -> > > >> > > >> >> I'm not sure > > >> > > >> >> > >>>> > if this would end up in Flink 1.14.1, I think it's > > >> more likely > > >> > > >> >> that it > > >> > > >> >> > >>>> > would be Flink 1.15. Anton/Dawid, could you confirm > > >> this? > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-23946 > > - > > >> > > >> >> Application mode > > >> > > >> >> > >>>> > fails fatally when being shut down -> This depends > on > > >> > > >> >> > >>>> > https://issues.apache.org/jira/browse/FLINK-24038 > > and > > >> I don't > > >> > > >> >> see much > > >> > > >> >> > >>>> > happening there, so I also expect that this would > > move > > >> to > > >> > > >> Flink > > >> > > >> >> 1.15. > > >> > > >> >> > >>>> > David, could you confirm? > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-22113 > > - > > >> > > >> UniqueKey > > >> > > >> >> constraint > > >> > > >> >> > >>>> > is lost with multiple sources join in SQL > > >> > > >> >> > >>>> > * > https://issues.apache.org/jira/browse/FLINK-21788 > > - > > >> Throw > > >> > > >> >> > >>>> > PartitionNotFoundException if the partition file > has > > >> been lost > > >> > > >> >> for blocking > > >> > > >> >> > >>>> > shuffle -> I'm also expecting that this would move > to > > >> Flink > > >> > > >> >> 1.15, can you > > >> > > >> >> > >>>> > confirm Yingjie ? > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > There are quite some other tickets that I've > excluded > > >> from > > >> > > >> this > > >> > > >> >> list, > > >> > > >> >> > >>>> > because they are either test instabilities or are > not > > >> > > >> depending > > >> > > >> >> on a Flink > > >> > > >> >> > >>>> > release to be resolved. > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Note: there are quite a few test instabilities in > the > > >> list and > > >> > > >> >> help on > > >> > > >> >> > >>>> > those is always appreciated. You can check all > > >> unassigned > > >> > > >> tickets > > >> > > >> >> > >>>> > instabilities in Jira [4]. > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Are there any other open tickets that we should > wait > > >> for? Is > > >> > > >> >> there a PMC > > >> > > >> >> > >>>> > member who would like to manage the release? I'm > more > > >> than > > >> > > >> happy > > >> > > >> >> to help > > >> > > >> >> > >>>> > with monitoring the status of the tickets. > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Best regards, > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Martijn > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > [1] > > >> > > >> https://flink.apache.org/news/2021/09/29/release-1.14.0.html > > >> > > >> >> > >>>> > [2] > > >> > > >> >> > >>>> > > > >> > > >> >> > > >> > > >> > > >> > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC > > >> > > >> >> > >>>> > [3] > > >> > > >> >> > >>>> > > > >> > > >> >> > > >> > > >> > > >> > > > https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > [4] > > >> > > >> >> > >>>> > > > >> > > >> >> > > >> > > >> > > >> > > > https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20AND%20labels%20%3D%20test-stability%20AND%20assignee%20in%20(EMPTY)%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Martijn Visser | Product Manager > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > mart...@ververica.com > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > <https://www.ververica.com/> > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Follow us @VervericaData > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > -- > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Join Flink Forward <https://flink-forward.org/> - > > The > > >> Apache > > >> > > >> >> Flink > > >> > > >> >> > >>>> > Conference > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > Stream Processing | Event Driven | Real Time > > >> > > >> >> > >>>> > > > >> > > >> >> > >>>> > > >> > > >> >> > >> > > >> > > >> >> > >> > > >> > > >> >> > >> -- > > >> > > >> >> > >> Marios > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > >> >> -- > > >> > > >> >> Best, Jingsong Lee > > >> > > >> >> > > >> > > >> > > > >> > > >> > > >> > > > > > >> > > > > > > -- Konstantin Knauf https://twitter.com/snntrable https://github.com/knaufk
