Looking forward the first RC of Flink 1.10.1 . Good job Yu! Best, Jincheng
Jark Wu <imj...@gmail.com> 于2020年4月15日周三 下午6:28写道： > +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is > released. > > Best, > Jark > > On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <trohrm...@apache.org> wrote: > > > Great to see that will have the first RC for Flink 1.10.1 soon. Thanks a > > lot for driving this effort Yu! > > > > Cheers, > > Till > > > > On Sun, Apr 12, 2020 at 5:03 PM Yu Li <car...@gmail.com> wrote: > > > > > Thanks Weike and all others for the efforts! > > > > > > Here comes the latest status, we are in good shape and plan to produce > > RC1 > > > next week. > > > > > > * Blockers (1 left) > > > - [Closed] FLINK-16018 Improve error reporting when submitting batch > > job > > > (instead of AskTimeoutException) > > > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on > > repeated > > > job submission > > > - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException > > when > > > use flink-sql-connector-elasticsearch7 > > > - [Closed] FLINK-16262 Class loader problem with > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory > > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to > > > minimise its OutOfMemoryError > > > - [Closed] FLINK-16454 Update the copyright year in NOTICE files > > > - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before > > client > > > can retrieve JobResult > > > - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum > > > throws UnsupportedOperationException > > > - [Closed] FLINK-16626 Exception encountered when cancelling a job in > > > yarn per-job mode > > > - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when > the > > > input column is of composite type > > > - [PR reviewed] FLINK-16576 State inconsistency on restore with > memory > > > state backends > > > > > > * Critical (1 left) > > > - [Closed] FLINK-16047 Blink planner produces wrong aggregate results > > > with state clean up > > > - [Closed] FLINK-16070 Blink planner can not extract correct unique > key > > > for UpsertStreamTableSink > > > - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should > > be > > > handled as Fatal Error in TaskManager > > > - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" > error > > > - [May Postpone] FLINK-16408 Bind user code class loader to lifetime > > of a > > > slot > > > > > > Please let me know if any concerns/comments. Thanks. > > > > > > Best Regards, > > > Yu > > > > > > > > > On Fri, 3 Apr 2020 at 21:35, DONG, Weike <kyled...@connect.hku.hk> > > wrote: > > > > > > > Hi Yu, > > > > > > > > Thanks for your updates. I am still working on the fix for > FLINK-16626 > > > and > > > > it is expected to be completed by this Sunday after thorough testing. > > > > > > > > Sincerely, > > > > Weike > > > > > > > > On Fri, Apr 3, 2020 at 8:43 PM Yu Li <car...@gmail.com> wrote: > > > > > > > >> Updates for 1.10.1 watched issues (we are in good progress and > almost > > > >> there > > > >> to produce the first RC, thanks all for the efforts): > > > >> > > > >> * Blockers (3 left) > > > >> - [Closed] FLINK-16018 Improve error reporting when submitting > batch > > > job > > > >> (instead of AskTimeoutException) > > > >> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on > > > >> repeated > > > >> job submission > > > >> - [Closed] FLINK-16170 SearchTemplateRequest > ClassNotFoundException > > > when > > > >> use flink-sql-connector-elasticsearch7 > > > >> - [Closed] FLINK-16262 Class loader problem with > > > >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory > > > >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to > > > >> minimise its OutOfMemoryError > > > >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files > > > >> - [PR reviewed] FLINK-16576 State inconsistency on restore with > > memory > > > >> state backends > > > >> - [Under Discussion] FLINK-16626 Exception encountered when > > > cancelling a > > > >> job in yarn per-job mode > > > >> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before > > > >> client > > > >> can retrieve JobResult > > > >> - [PR reviewed] FLINK-16913 > > > ReadableConfigToConfigurationAdapter#getEnum > > > >> throws UnsupportedOperationException > > > >> > > > >> * Critical (1 left) > > > >> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" > > error > > > >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate > > results > > > >> with state clean up > > > >> - [Closed] FLINK-16070 Blink planner can not extract correct > unique > > > key > > > >> for UpsertStreamTableSink > > > >> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory > > should > > > be > > > >> handled as Fatal Error in TaskManager > > > >> - [May Postpone] FLINK-16408 Bind user code class loader to > lifetime > > > of > > > >> a > > > >> slot > > > >> > > > >> Please let me know if you find any missing ones, thanks. > > > >> > > > >> Best Regards, > > > >> Yu > > > >> > > > >> > > > >> On Fri, 27 Mar 2020 at 21:06, Yu Li <car...@gmail.com> wrote: > > > >> > > > >> > Here comes the latest status of issues in 1.10.1 watch list: > > > >> > > > > >> > * Blockers (4 left) > > > >> > - [Under Discussion] FLINK-16018 Improve error reporting when > > > >> submitting > > > >> > batch job (instead of AskTimeoutException) > > > >> > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on > > > >> > repeated job submission > > > >> > - [Closed] FLINK-16170 SearchTemplateRequest > > ClassNotFoundException > > > >> when > > > >> > use flink-sql-connector-elasticsearch7 > > > >> > - [PR Approved] FLINK-16262 Class loader problem with > > > >> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory > > > >> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace > to > > > >> > minimise its OutOfMemoryError > > > >> > - [Closed] FLINK-16454 Update the copyright year in NOTICE files > > > >> > - [In Progress] FLINK-16576 State inconsistency on restore with > > > memory > > > >> > state backends > > > >> > - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down > > > >> MiniCluster > > > >> > before client can retrieve JobResult > > > >> > > > > >> > * Critical (2 left) > > > >> > - [Closed] FLINK-16047 Blink planner produces wrong aggregate > > > results > > > >> > with state clean up > > > >> > - [PR Submitted] FLINK-16070 Blink planner can not extract > correct > > > >> > unique key for UpsertStreamTableSink > > > >> > - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory > should > > > be > > > >> > handled as Fatal Error in TaskManager > > > >> > - [Open] FLINK-16408 Bind user code class loader to lifetime of > a > > > slot > > > >> > > > > >> > There's a new blocker added. Please let me know if you find any > > other > > > >> > issues should be put into the watch list. Thanks. > > > >> > > > > >> > Best Regards, > > > >> > Yu > > > >> > > > > >> > > > > >> > On Sat, 21 Mar 2020 at 14:37, Yu Li <car...@gmail.com> wrote: > > > >> > > > > >> >> Hi All, > > > >> >> > > > >> >> Here is the status update of issues in 1.10.1 watch list: > > > >> >> > > > >> >> * Blockers (7) > > > >> >> > > > >> >> - [Under Discussion] FLINK-16018 Improve error reporting when > > > >> >> submitting batch job (instead of AskTimeoutException) > > > >> >> > > > >> >> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace > OOM > > > >> error > > > >> >> on repeated job submission > > > >> >> > > > >> >> - [PR Submitted] FLINK-16170 SearchTemplateRequest > > > >> >> ClassNotFoundException when use > flink-sql-connector-elasticsearch7 > > > >> >> > > > >> >> - [PR Submitted] FLINK-16262 Class loader problem with > > > >> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory > > > >> >> > > > >> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace > > to > > > >> >> minimise its OutOfMemoryError > > > >> >> > > > >> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE > files > > > >> >> > > > >> >> - [Open] FLINK-16576 State inconsistency on restore with memory > > > state > > > >> >> backends > > > >> >> > > > >> >> > > > >> >> * Critical (4) > > > >> >> > > > >> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate > > > results > > > >> >> with state clean up > > > >> >> > > > >> >> - [PR Submitted] FLINK-16070 Blink planner can not extract > > correct > > > >> >> unique key for UpsertStreamTableSink > > > >> >> > > > >> >> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should > > be > > > >> >> handled as Fatal Error in TaskManager > > > >> >> > > > >> >> - [Open] FLINK-16408 Bind user code class loader to lifetime > of a > > > >> slot > > > >> >> > > > >> >> Please let me know if you see any new blockers to add to the > list. > > > >> Thanks. > > > >> >> > > > >> >> Best Regards, > > > >> >> Yu > > > >> >> > > > >> >> > > > >> >> On Wed, 18 Mar 2020 at 00:11, Yu Li <car...@gmail.com> wrote: > > > >> >> > > > >> >>> Thanks for the updates Till! > > > >> >>> > > > >> >>> For FLINK-16018, maybe we could create two sub-tasks for easy > and > > > >> >>> complete fix separately, and only include the easy one in > 1.10.1? > > Or > > > >> please > > > >> >>> just feel free to postpone the whole task to 1.10.2 if "all or > > > >> nothing" > > > >> >>> policy is preferred (smile). Thanks. > > > >> >>> > > > >> >>> Best Regards, > > > >> >>> Yu > > > >> >>> > > > >> >>> > > > >> >>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann < > trohrm...@apache.org > > > > > > >> >>> wrote: > > > >> >>> > > > >> >>>> +1 for a soonish bug fix release. Thanks for volunteering as > our > > > >> release > > > >> >>>> manager Yu. > > > >> >>>> > > > >> >>>> I think we can soon merge the increase of metaspace size and > > > >> improving > > > >> >>>> the > > > >> >>>> error message. The assumption is that we currently don't have > too > > > >> many > > > >> >>>> small Flink 1.10 deployments with a process size <= 1GB. Of > > course, > > > >> the > > > >> >>>> sooner we release the bug fix release, the fewer deployments > will > > > be > > > >> >>>> affected by this change. > > > >> >>>> > > > >> >>>> For FLINK-16018, I think there would be an easy solution which > we > > > >> could > > > >> >>>> include in the bug fix release. The proper fix will most likely > > > take > > > >> a > > > >> >>>> bit > > > >> >>>> longer, though. > > > >> >>>> > > > >> >>>> Cheers, > > > >> >>>> Till > > > >> >>>> > > > >> >>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin < > > > >> azagre...@apache.org> > > > >> >>>> wrote: > > > >> >>>> > > > >> >>>> > > @Andrey and @Xintong - could we have a quick poll on the > user > > > >> >>>> mailing > > > >> >>>> > list > > > >> >>>> > > about increasing the metaspace size in Flink 1.10.1? > > > Specifically > > > >> >>>> asking > > > >> >>>> > > for who has very small TM setups? > > > >> >>>> > > > > >> >>>> > There has been a survey about this topic since 10 days: > > > >> >>>> > > > > >> >>>> > `[Survey] Default size for the new JVM Metaspace limit in > 1.10` > > > >> >>>> > I can ask there about the small TM setups. > > > >> >>>> > > > > >> >>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <car...@gmail.com> > > wrote: > > > >> >>>> > > > > >> >>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency > > on > > > >> >>>> restore > > > >> >>>> > with > > > >> >>>> > > memory state backends > > > >> >>>> > > > > > >> >>>> > > Let me recompile the watching list with recent feedbacks. > > > >> There're > > > >> >>>> > totally > > > >> >>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of > > > >> which 14 > > > >> >>>> > > already resolved and 31 left, and the below ones are > watched > > > >> >>>> (meaning > > > >> >>>> > that > > > >> >>>> > > once the below ones got fixed, we will kick of the > releasing > > > with > > > >> >>>> left > > > >> >>>> > ones > > > >> >>>> > > postponed to 1.10.2 unless objections): > > > >> >>>> > > > > > >> >>>> > > * Blockers (7) > > > >> >>>> > > - [Under Discussion] FLINK-16018 Improve error reporting > > when > > > >> >>>> > submitting > > > >> >>>> > > batch job (instead of AskTimeoutException) > > > >> >>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes > > Metaspace > > > >> OOM > > > >> >>>> error > > > >> >>>> > > on repeated job submission > > > >> >>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest > > > >> >>>> > ClassNotFoundException > > > >> >>>> > > when use flink-sql-connector-elasticsearch7 > > > >> >>>> > > - [PR Submitted] FLINK-16262 Class loader problem with > > > >> >>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib > directory > > > >> >>>> > > - [Closed] FLINK-16406 Increase default value for JVM > > > >> Metaspace to > > > >> >>>> > > minimise its OutOfMemoryError > > > >> >>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE > > > files > > > >> >>>> > > - [Open] FLINK-16576 State inconsistency on restore with > > > memory > > > >> >>>> state > > > >> >>>> > > backends > > > >> >>>> > > > > > >> >>>> > > * Critical (4) > > > >> >>>> > > - [Open] FLINK-16047 Blink planner produces wrong > aggregate > > > >> >>>> results > > > >> >>>> > with > > > >> >>>> > > state clean up > > > >> >>>> > > - [PR Submitted] FLINK-16070 Blink planner can not > extract > > > >> correct > > > >> >>>> > unique > > > >> >>>> > > key for UpsertStreamTableSink > > > >> >>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory > > > >> should be > > > >> >>>> > > handled as Fatal Error in TaskManager > > > >> >>>> > > - [Open] FLINK-16408 Bind user code class loader to > > lifetime > > > >> of a > > > >> >>>> slot > > > >> >>>> > > > > > >> >>>> > > Please raise your hand if you find any other issues should > be > > > put > > > >> >>>> into > > > >> >>>> > this > > > >> >>>> > > list, thanks. > > > >> >>>> > > > > > >> >>>> > > Please also note that 1.10.2 version is already created > > (thanks > > > >> for > > > >> >>>> the > > > >> >>>> > > help @jincheng), and feel free to adjust/assign fix version > > to > > > it > > > >> >>>> when > > > >> >>>> > > necessary. > > > >> >>>> > > > > > >> >>>> > > Best Regards, > > > >> >>>> > > Yu > > > >> >>>> > > > > > >> >>>> > > > > > >> >>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen < > se...@apache.org > > > > > > >> >>>> wrote: > > > >> >>>> > > > > > >> >>>> > > > @Andrey and @Xintong - could we have a quick poll on the > > user > > > >> >>>> mailing > > > >> >>>> > > list > > > >> >>>> > > > about increasing the metaspace size in Flink 1.10.1? > > > >> Specifically > > > >> >>>> > asking > > > >> >>>> > > > for who has very small TM setups? > > > >> >>>> > > > > > > >> >>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <car...@gmail.com> > > > >> wrote: > > > >> >>>> > > > > > > >> >>>> > > > > Thanks for the reminder Stephan and the > inputs/discussion > > > >> >>>> Andrey and > > > >> >>>> > > > > Xintong. I totally agree that we should try best not to > > > >> >>>> introduce new > > > >> >>>> > > > > issues when resolving the existing one. Will watch > > closely > > > >> >>>> about the > > > >> >>>> > > > > discussion and conclusion around metaspace > configuration. > > > >> >>>> > > > > > > > >> >>>> > > > > From the point as a developer instead of RM, I'm > > optimistic > > > >> that > > > >> >>>> > > > reserving > > > >> >>>> > > > > 160MB more memory to JVM metaspace by default won't > > cause a > > > >> big > > > >> >>>> > > problem, > > > >> >>>> > > > > based on the assumption that nowadays assigning 2GB/4GB > > > >> memory > > > >> >>>> per TM > > > >> >>>> > > is > > > >> >>>> > > > > the common case. > > > >> >>>> > > > > > > > >> >>>> > > > > Best Regards, > > > >> >>>> > > > > Yu > > > >> >>>> > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song < > > > >> >>>> tonysong...@gmail.com> > > > >> >>>> > > > wrote: > > > >> >>>> > > > > > > > >> >>>> > > > > > I think in FLINK-16406 we have not only increased the > > > >> default > > > >> >>>> value > > > >> >>>> > > for > > > >> >>>> > > > > > metaspace, but also increased the 'process.size' in > > > default > > > >> >>>> > > > > > 'flink-conf.yaml'. > > > >> >>>> > > > > > > > > >> >>>> > > > > > That means: > > > >> >>>> > > > > > > > > >> >>>> > > > > > - For people using the default 'flink-conf.yaml', > > it's > > > >> not > > > >> >>>> > really > > > >> >>>> > > > > taking > > > >> >>>> > > > > > memory from heap / managed for metaspace. > > > >> >>>> > > > > > - For people using custom configuration files: > > > >> >>>> > > > > > - If they are specifying 'flink.size', then > > > metaspace > > > >> >>>> does > > > >> >>>> > not > > > >> >>>> > > > > really > > > >> >>>> > > > > > affect the flink memory internal components > > (heap, > > > >> >>>> managed, > > > >> >>>> > > > etc.). > > > >> >>>> > > > > > - If they are specifying 'process.size' (most > > > likely > > > >> >>>> active > > > >> >>>> > > > > setups), > > > >> >>>> > > > > > then they will have less heap / managed memory, > > or > > > >> >>>> worse, > > > >> >>>> > their > > > >> >>>> > > > > > configuration may not work without adjustment. > > > >> >>>> > > > > > - For a TM with large process memory (e.g., > 4 > > GB > > > >> or > > > >> >>>> more), > > > >> >>>> > > it > > > >> >>>> > > > is > > > >> >>>> > > > > > probably ok that the metaspace takes 160 MB > > more > > > >> >>>> memory. > > > >> >>>> > > > > > - For a TM with smaller total memory (1 or 2 > > GB) > > > >> the > > > >> >>>> > > metaspace > > > >> >>>> > > > > > increase is more likely to cause problem. > > > >> >>>> > > > > > > > > >> >>>> > > > > > So basically only people have small 'process.size' in > > > >> custom > > > >> >>>> config > > > >> >>>> > > > file > > > >> >>>> > > > > > are really affected. I'm not sure what is the > > proportion > > > of > > > >> >>>> such > > > >> >>>> > use > > > >> >>>> > > > > cases > > > >> >>>> > > > > > though. (From questions asked on the user ML, > probably > > > not > > > >> >>>> much). > > > >> >>>> > > > > > > > > >> >>>> > > > > > Thank you~ > > > >> >>>> > > > > > > > > >> >>>> > > > > > Xintong Song > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen < > > > >> >>>> se...@apache.org> > > > >> >>>> > > > wrote: > > > >> >>>> > > > > > > > > >> >>>> > > > > > > No need to revert it now - I am not saying it > should > > > not > > > >> go > > > >> >>>> into > > > >> >>>> > > > > 1.10.1, > > > >> >>>> > > > > > I > > > >> >>>> > > > > > > am just saying this is not clear to me yet. > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > We have to trade off the fact that we may break > some > > > >> >>>> deployments > > > >> >>>> > > with > > > >> >>>> > > > > the > > > >> >>>> > > > > > > fact that we will "safe" a lot of new deployments. > > > >> >>>> > > > > > > I simply lack the data points / insight at the > moment > > > to > > > >> >>>> > understand > > > >> >>>> > > > how > > > >> >>>> > > > > > > significant both cases are, meaning how many users > > > would > > > >> be > > > >> >>>> > > affected > > > >> >>>> > > > > and > > > >> >>>> > > > > > > how badly. > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > Independent of that, improving the error message is > > > >> always > > > >> >>>> > helpful. > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin < > > > >> >>>> > > > > > > azagrebin.apa...@gmail.com> > > > >> >>>> > > > > > > wrote: > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, > because > > > >> users > > > >> >>>> expect > > > >> >>>> > > to > > > >> >>>> > > > > > > upgrade > > > >> >>>> > > > > > > > > that without config adjustments. That might not > > be > > > >> >>>> possible > > > >> >>>> > > with > > > >> >>>> > > > > that > > > >> >>>> > > > > > > > > change. > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and > only > > > >> try to > > > >> >>>> > > improve > > > >> >>>> > > > > > error > > > >> >>>> > > > > > > > message and docs. > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen < > > > >> >>>> se...@apache.org> > > > >> >>>> > > wrote: > > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > @Andrey about the increase in metaspace size > > > >> >>>> > > > > > > > > - I have no concerns for 1.11.0. > > > >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, > because > > > >> users > > > >> >>>> expect > > > >> >>>> > > to > > > >> >>>> > > > > > > upgrade > > > >> >>>> > > > > > > > > that without config adjustments. That might not > > be > > > >> >>>> possible > > > >> >>>> > > with > > > >> >>>> > > > > that > > > >> >>>> > > > > > > > > change. > > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey > Zagrebin > > < > > > >> >>>> > > > > > > > azagrebin.apa...@gmail.com> > > > >> >>>> > > > > > > > > wrote: > > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes > Metaspace > > > OOM > > > >> >>>> error > > > >> >>>> > on > > > >> >>>> > > > > > repeated > > > >> >>>> > > > > > > > >> job” > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> My understanding that the issue is basically > > > covered > > > >> >>>> by: > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should > > be > > > >> >>>> handled as > > > >> >>>> > > > Fatal > > > >> >>>> > > > > > > Error > > > >> >>>> > > > > > > > >> in TaskManager > > > >> >>>> > > > > > > > >> no full consensus there but improving error > > > >> message > > > >> >>>> for > > > >> >>>> > > > existing > > > >> >>>> > > > > > > task > > > >> >>>> > > > > > > > >> thread fatal handling could be done at least > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM > > > >> >>>> Metaspace to > > > >> >>>> > > > > minimise > > > >> >>>> > > > > > > its > > > >> >>>> > > > > > > > >> OutOfMemoryError > > > >> >>>> > > > > > > > >> see further > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> - [FLINK-16246] Exclude > > "SdkMBeanRegistrySupport" > > > >> from > > > >> >>>> > > > dynamically > > > >> >>>> > > > > > > > loaded > > > >> >>>> > > > > > > > >> AWS connectors > > > >> >>>> > > > > > > > >> not sure whether this is a blocker but looks > > > close > > > >> to > > > >> >>>> be > > > >> >>>> > > > resolved > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for > > JVM > > > >> >>>> > Metaspace" > > > >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a > > > bugfix > > > >> >>>> release? > > > >> >>>> > > It > > > >> >>>> > > > > > > changes > > > >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed > > > >> memory > > > >> >>>> on > > > >> >>>> > > > existing > > > >> >>>> > > > > > > setups > > > >> >>>> > > > > > > > >>> that keep their flink-conf.yaml. > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> My understanding was that increasing to 256m > > > >> resolved > > > >> >>>> the > > > >> >>>> > > > reported > > > >> >>>> > > > > > > > problems > > > >> >>>> > > > > > > > >> and we decided to make the change so I have > > merged > > > >> it > > > >> >>>> today > > > >> >>>> > as > > > >> >>>> > > > > there > > > >> >>>> > > > > > > > were > > > >> >>>> > > > > > > > >> no more concerns. > > > >> >>>> > > > > > > > >> If there are concerns I can revert it. > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> On the other hand, I think improving the > message > > > >> error > > > >> >>>> with > > > >> >>>> > > > > > reference > > > >> >>>> > > > > > > to > > > >> >>>> > > > > > > > >> the metaspace option should help the most > > > >> >>>> > > > > > > > >> because user would not have to read all docs > to > > > fix > > > >> it > > > >> >>>> > > > > > > > >> then maybe this change is not even needed. > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> Best, > > > >> >>>> > > > > > > > >> Andrey > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen < > > > >> >>>> se...@apache.org> > > > >> >>>> > > > wrote: > > > >> >>>> > > > > > > > >>> > > > >> >>>> > > > > > > > >>> Good idea to go ahead with 1.10.1 > > > >> >>>> > > > > > > > >>> > > > >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes > Metaspace > > > OOM > > > >> >>>> error > > > >> >>>> > on > > > >> >>>> > > > > > repeated > > > >> >>>> > > > > > > > >> job" > > > >> >>>> > > > > > > > >>> - I don't think we have consensus on the > exact > > > >> >>>> solution, > > > >> >>>> > yet, > > > >> >>>> > > > and > > > >> >>>> > > > > > > some > > > >> >>>> > > > > > > > >> of > > > >> >>>> > > > > > > > >>> the changes might also have side effects that > > are > > > >> >>>> hard to > > > >> >>>> > > > > predict, > > > >> >>>> > > > > > > so I > > > >> >>>> > > > > > > > >> am > > > >> >>>> > > > > > > > >>> not sure we should rush this in. > > > >> >>>> > > > > > > > >>> > > > >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for > > JVM > > > >> >>>> > Metaspace" > > > >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a > > > bugfix > > > >> >>>> release? > > > >> >>>> > > It > > > >> >>>> > > > > > > changes > > > >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed > > > >> memory > > > >> >>>> on > > > >> >>>> > > > existing > > > >> >>>> > > > > > > setups > > > >> >>>> > > > > > > > >>> that keep their flink-conf.yaml. > > > >> >>>> > > > > > > > >>> > > > >> >>>> > > > > > > > >>> We may need to unblock the release form these > > two > > > >> >>>> issues > > > >> >>>> > and > > > >> >>>> > > > > think > > > >> >>>> > > > > > > > about > > > >> >>>> > > > > > > > >>> having 1.10.2 in the near future. > > > >> >>>> > > > > > > > >>> > > > >> >>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li < > > > >> >>>> car...@gmail.com> > > > >> >>>> > > > wrote: > > > >> >>>> > > > > > > > >>> > > > >> >>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an > eye > > > on > > > >> >>>> these > > > >> >>>> > two. > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >>>> Best Regards, > > > >> >>>> > > > > > > > >>>> Yu > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu < > > > >> >>>> imj...@gmail.com> > > > >> >>>> > > > wrote: > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >>>>> Thanks for driving this release, Yu! > > > >> >>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle. > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>>> From the Table SQL module, I think we > should > > > also > > > >> >>>> try to > > > >> >>>> > > get > > > >> >>>> > > > in > > > >> >>>> > > > > > the > > > >> >>>> > > > > > > > >>>>> following issues: > > > >> >>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override > > > flink-conf > > > >> >>>> > > parameters > > > >> >>>> > > > > from > > > >> >>>> > > > > > > SQL > > > >> >>>> > > > > > > > >> CLI > > > >> >>>> > > > > > > > >>>>> environment > > > >> >>>> > > > > > > > >>>>> this allows users to set e.g. statebackend, > > > >> >>>> watermark > > > >> >>>> > > > interval, > > > >> >>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI > > > >> >>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any > > > >> uncatched > > > >> >>>> > > > exception > > > >> >>>> > > > > is > > > >> >>>> > > > > > > > >> thrown > > > >> >>>> > > > > > > > >>>>> this will improve much experience when > using > > > SQL > > > >> CLI > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>>> Best, > > > >> >>>> > > > > > > > >>>>> Jark > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li < > > > >> >>>> car...@gmail.com> > > > >> >>>> > > > wrote: > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've > added > > > >> >>>> 1.10.1 into > > > >> >>>> > > > > > > FLINK-16225 > > > >> >>>> > > > > > > > >>>> fix > > > >> >>>> > > > > > > > >>>>>> versions and promoted its priority to > > > Critical. > > > >> >>>> Will > > > >> >>>> > also > > > >> >>>> > > > > watch > > > >> >>>> > > > > > > the > > > >> >>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408. > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>>> Best Regards, > > > >> >>>> > > > > > > > >>>>>> Yu > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey > > Zagrebin > > > < > > > >> >>>> > > > > > > azagre...@apache.org > > > >> >>>> > > > > > > > > > > > >> >>>> > > > > > > > >>>>>> wrote: > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>>>> Hi Yu, > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release > > > >> >>>> discussion! > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> Apart from > > > >> >>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for > > JVM > > > >> >>>> Metaspace > > > >> >>>> > to > > > >> >>>> > > > > > > minimise > > > >> >>>> > > > > > > > >>>> its > > > >> >>>> > > > > > > > >>>>>>> OutOfMemoryError > > > >> >>>> > > > > > > > >>>>>>> which should be merged soon > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> I think we should also try to get in the > > > >> following > > > >> >>>> > > issues: > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory > > > should > > > >> be > > > >> >>>> > handled > > > >> >>>> > > > as > > > >> >>>> > > > > > > Fatal > > > >> >>>> > > > > > > > >>>>> Error > > > >> >>>> > > > > > > > >>>>>> in > > > >> >>>> > > > > > > > >>>>>>> TaskManager > > > >> >>>> > > > > > > > >>>>>>> This should solve the Metaspace problem > > even > > > >> in a > > > >> >>>> > better > > > >> >>>> > > > way > > > >> >>>> > > > > > > > because > > > >> >>>> > > > > > > > >>>>> OOM > > > >> >>>> > > > > > > > >>>>>>> failure should point users to the docs > > > >> immediately > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class > loader > > > to > > > >> >>>> lifetime > > > >> >>>> > > of > > > >> >>>> > > > a > > > >> >>>> > > > > > slot > > > >> >>>> > > > > > > > >>>>>>> This should give a better protection > > against > > > >> class > > > >> >>>> > > loading > > > >> >>>> > > > > > leaks > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting > > when > > > >> >>>> submitting > > > >> >>>> > > > batch > > > >> >>>> > > > > > job > > > >> >>>> > > > > > > > >>>>>> (instead > > > >> >>>> > > > > > > > >>>>>>> of AskTimeoutException) > > > >> >>>> > > > > > > > >>>>>>> This problem has recently happened for > > > multiple > > > >> >>>> users > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> Best, > > > >> >>>> > > > > > > > >>>>>>> Andrey > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong > > Li < > > > >> >>>> > > > > > > > jingsongl...@gmail.com> > > > >> >>>> > > > > > > > >>>>>>> wrote: > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting > > the > > > >> >>>> 1.10.1 > > > >> >>>> > > > release. > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> Some issues are very important, Users > are > > > >> looking > > > >> >>>> > > forward > > > >> >>>> > > > to > > > >> >>>> > > > > > > them. > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> Best, > > > >> >>>> > > > > > > > >>>>>>>> Jingsong Lee > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze > > Guo < > > > >> >>>> > > > > > karma...@gmail.com> > > > >> >>>> > > > > > > > >>>>> wrote: > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu! > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release > cycle. > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> Best, > > > >> >>>> > > > > > > > >>>>>>>>> Yangze Guo > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong > > > Song > > > >> < > > > >> >>>> > > > > > > > >>>>> tonysong...@gmail.com> > > > >> >>>> > > > > > > > >>>>>>>>> wrote: > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>> Yu, > > > >> >>>> > > > > > > > >>>>>>>>>> Thanks for the explanation. > > > >> >>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to > > get > > > >> some > > > >> >>>> > inputs > > > >> >>>> > > > for > > > >> >>>> > > > > > > > >>>>>>> prioritizing > > > >> >>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds > > good > > > to > > > >> >>>> me. > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>> Thank you~ > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>> Xintong Song > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu > Li < > > > >> >>>> > > > car...@gmail.com> > > > >> >>>> > > > > > > > >>>> wrote: > > > >> >>>> > > > > > > > >>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1? > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> According to the history, the first > > patch > > > >> >>>> release > > > >> >>>> > of > > > >> >>>> > > a > > > >> >>>> > > > > > major > > > >> >>>> > > > > > > > >>>>>>> version > > > >> >>>> > > > > > > > >>>>>>>>> will > > > >> >>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started, > > > >> >>>> depending on > > > >> >>>> > > the > > > >> >>>> > > > > > speed > > > >> >>>> > > > > > > > >>>> of > > > >> >>>> > > > > > > > >>>>>>>> blocker > > > >> >>>> > > > > > > > >>>>>>>>>>> issue resolving: > > > >> >>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May > 28th > > > >> [1], > > > >> >>>> > released > > > >> >>>> > > > on > > > >> >>>> > > > > > Jul > > > >> >>>> > > > > > > > >>>>> 3rd > > > >> >>>> > > > > > > > >>>>>>> [2] > > > >> >>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep > 23rd > > > >> [3], > > > >> >>>> > released > > > >> >>>> > > > on > > > >> >>>> > > > > > Oct > > > >> >>>> > > > > > > > >>>>> 19th > > > >> >>>> > > > > > > > >>>>>>> [4] > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of > > > >> course, > > > >> >>>> but > > > >> >>>> > > could > > > >> >>>> > > > > use > > > >> >>>> > > > > > > > >>>> it > > > >> >>>> > > > > > > > >>>>>> as a > > > >> >>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to > let > > me > > > >> >>>> know if > > > >> >>>> > any > > > >> >>>> > > > > > > > >>>> concerns > > > >> >>>> > > > > > > > >>>>>>>> Xintong. > > > >> >>>> > > > > > > > >>>>>>>>>>> Thanks. > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> Best Regards, > > > >> >>>> > > > > > > > >>>>>>>>>>> Yu > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> [1] > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html > > > >> >>>> > > > > > > > >>>>>>>>>>> [2] > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html > > > >> >>>> > > > > > > > >>>>>>>>>>> [3] > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html > > > >> >>>> > > > > > > > >>>>>>>>>>> [4] > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong > > > Song > > > >> < > > > >> >>>> > > > > > > > >>>>>> tonysong...@gmail.com> > > > >> >>>> > > > > > > > >>>>>>>>> wrote: > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and > > > >> volunteering > > > >> >>>> to be > > > >> >>>> > > the > > > >> >>>> > > > > > > > >>>>> release > > > >> >>>> > > > > > > > >>>>>>>>> manager. > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal. > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time > > > plan > > > >> for > > > >> >>>> > > 1.10.1? > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> Thank you~ > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> Xintong Song > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM > > > Zhijiang > > > >> >>>> > > > > > > > >>>>>>>>>>>> <wangzhijiang...@aliyun.com > .invalid> > > > >> wrote: > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, > Yu! > > > >> >>>> > > > > > > > >>>>>>>>>>>>> +1 on my side > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Best, > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Zhijiang > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > > > >> >>>> > > > > ------------------------------------------------------------------ > > > >> >>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <car...@gmail.com> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25 > > > >> >>>> > > > > > > > >>>>>>>>>>>>> To:dev <dev@flink.apache.org> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing > Flink > > > >> 1.10.1 > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. > Yes > > > will > > > >> >>>> also > > > >> >>>> > keep > > > >> >>>> > > > an > > > >> >>>> > > > > > > > >>>> eye > > > >> >>>> > > > > > > > >>>>> on > > > >> >>>> > > > > > > > >>>>>>>> these > > > >> >>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues. > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Best Regards, > > > >> >>>> > > > > > > > >>>>>>>>>>>>> Yu > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn > > > >> Cheng < > > > >> >>>> > > > > > > > >>>>> he...@apache.org> > > > >> >>>> > > > > > > > >>>>>>>>> wrote: > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the > > > discussion > > > >> and > > > >> >>>> > > > volunteer > > > >> >>>> > > > > > > > >>>> as > > > >> >>>> > > > > > > > >>>>>> the > > > >> >>>> > > > > > > > >>>>>>>>> release > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> manager! > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other > > issues[1] > > > >> >>>> which are > > > >> >>>> > > > > marked > > > >> >>>> > > > > > > > >>>>> as > > > >> >>>> > > > > > > > >>>>>> a > > > >> >>>> > > > > > > > >>>>>>>>> blocker: > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright > > > year > > > >> in > > > >> >>>> > NOTICE > > > >> >>>> > > > > files > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem > > > with > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE > > > >> >>>> and > > > >> >>>> > > usrlib > > > >> >>>> > > > > > > > >>>>>> directory > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 > SearchTemplateRequest > > > >> >>>> > > > > > > > >>>>> ClassNotFoundException > > > >> >>>> > > > > > > > >>>>>>> when > > > >> >>>> > > > > > > > >>>>>>>>> use > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7 > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error > > reporting > > > >> when > > > >> >>>> > > > submitting > > > >> >>>> > > > > > > > >>>>> batch > > > >> >>>> > > > > > > > >>>>>>> job > > > >> >>>> > > > > > > > >>>>>>>>>>>> (instead > > > >> >>>> > > > > > > > >>>>>>>>>>>>> of > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException) > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved > > in > > > >> >>>> 1.10.1. > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Best, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Hequn > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> [1] > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > > > >> >>>> > https://issues.apache.org/jira/projects/FLINK/versions/12346891 > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu > > Li > > > < > > > >> >>>> > > > > > > > >>>> car...@gmail.com> > > > >> >>>> > > > > > > > >>>>>>>> wrote: > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very > > helpful. > > > >> >>>> Thanks a > > > >> >>>> > > lot! > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yu > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, > > > jincheng > > > >> >>>> sun < > > > >> >>>> > > > > > > > >>>>>>>>>>> sunjincheng...@gmail.com > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> wrote: > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the > discussion > > > >> Yu. I > > > >> >>>> would > > > >> >>>> > > > like > > > >> >>>> > > > > > > > >>>>> to > > > >> >>>> > > > > > > > >>>>>>> give > > > >> >>>> > > > > > > > >>>>>>>>> you a > > > >> >>>> > > > > > > > >>>>>>>>>>>>> hand > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> at > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is > > > >> >>>> finished.(If you > > > >> >>>> > > > > > > > >>>> need) > > > >> >>>> > > > > > > > >>>>>> :) > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Best, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <car...@gmail.com> > > > >> 于2020年3月10日周二 > > > >> >>>> > > 下午5:49写道： > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month > > since > > > we > > > >> >>>> > released > > > >> >>>> > > > > > > > >>>>> Flink > > > >> >>>> > > > > > > > >>>>>>>>> 1.10.0. We > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> already > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved > > > >> >>>> improvements/bugs > > > >> >>>> > in > > > >> >>>> > > > > > > > >>>> the > > > >> >>>> > > > > > > > >>>>>>>>>>> release-1.10 > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> branch, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the > 1.10.1 > > > >> >>>> release > > > >> >>>> > > cycle. > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are: > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove > the > > > >> >>>> license and > > > >> >>>> > > > > > > > >>>> notice > > > >> >>>> > > > > > > > >>>>>>> file > > > >> >>>> > > > > > > > >>>>>>>> in > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> module > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB > > resource > > > >> leak > > > >> >>>> in > > > >> >>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero > > > should > > > >> be > > > >> >>>> known > > > >> >>>> > > in > > > >> >>>> > > > > > > > >>>>>>>>> HiveCatalog > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336 > > > >> >>>> ArrayIndexOufOBoundsException in > > > >> >>>> > > > > > > > >>>>>>>> TypeExtractor > > > >> >>>> > > > > > > > >>>>>>>>>>> when > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> mapping > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample > is > > > >> >>>> failed if > > > >> >>>> > > > > > > > >>>> running > > > >> >>>> > > > > > > > >>>>>> in > > > >> >>>> > > > > > > > >>>>>>>>> blink > > > >> >>>> > > > > > > > >>>>>>>>>>>>> planner > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location > > > constraints > > > >> >>>> are not > > > >> >>>> > > > > > > > >>>> reset > > > >> >>>> > > > > > > > >>>>>> on > > > >> >>>> > > > > > > > >>>>>>>> task > > > >> >>>> > > > > > > > >>>>>>>>>>>>> recovery > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> in > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf > > > >> function > > > >> >>>> using > > > >> >>>> > > sql > > > >> >>>> > > > > > > > >>>>>>> casuing > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL > > validation > > > >> >>>> failed > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the > > following > > > >> >>>> issues > > > >> >>>> > > should > > > >> >>>> > > > > > > > >>>> be > > > >> >>>> > > > > > > > >>>>>>>> merged > > > >> >>>> > > > > > > > >>>>>>>>>>> before > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1 > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the > Metaspace > > > OOM > > > >> >>>> issue): > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak > causes > > > >> >>>> Metaspace > > > >> >>>> > OOM > > > >> >>>> > > > > > > > >>>>> error > > > >> >>>> > > > > > > > >>>>>> on > > > >> >>>> > > > > > > > >>>>>>>>>>> repeated > > > >> >>>> > > > > > > > >>>>>>>>>>>>> job > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> submission > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default > > > value > > > >> >>>> for JVM > > > >> >>>> > > > > > > > >>>>>> Metaspace > > > >> >>>> > > > > > > > >>>>>>> to > > > >> >>>> > > > > > > > >>>>>>>>>>>> minimise > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> its > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner > > > produces > > > >> >>>> wrong > > > >> >>>> > > > > > > > >>>>> aggregate > > > >> >>>> > > > > > > > >>>>>>>>> results > > > >> >>>> > > > > > > > >>>>>>>>>>> with > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> state > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can > > not > > > >> >>>> extract > > > >> >>>> > > > > > > > >>>> correct > > > >> >>>> > > > > > > > >>>>>>>> unique > > > >> >>>> > > > > > > > >>>>>>>>> key > > > >> >>>> > > > > > > > >>>>>>>>>>>> for > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the > release > > > >> >>>> manager and > > > >> >>>> > > kick > > > >> >>>> > > > > > > > >>>>> off > > > >> >>>> > > > > > > > >>>>>>> the > > > >> >>>> > > > > > > > >>>>>>>>>>> release > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> process > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. > > > What > > > >> do > > > >> >>>> you > > > >> >>>> > > > > > > > >>>> think? > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or > > > missing > > > >> >>>> blocker > > > >> >>>> > > > > > > > >>>> issues > > > >> >>>> > > > > > > > >>>>>>> need > > > >> >>>> > > > > > > > >>>>>>>>> to be > > > >> >>>> > > > > > > > >>>>>>>>>>>>> fixed > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> in > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. > > Thanks. > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards, > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>>> -- > > > >> >>>> > > > > > > > >>>>>>>> Best, Jingsong Lee > > > >> >>>> > > > > > > > >>>>>>>> > > > >> >>>> > > > > > > > >>>>>>> > > > >> >>>> > > > > > > > >>>>>> > > > >> >>>> > > > > > > > >>>>> > > > >> >>>> > > > > > > > >>>> > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > >> > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > > >> >>>> > > > > > > > > > >> >>>> > > > > > > > > >> >>>> > > > > > > > >> >>>> > > > > > > >> >>>> > > > > > >> >>>> > > > > >> >>>> > > > >> >>> > > > >> > > > > > > > > > >
