Re: [DISCUSS] FLIP-84 Feedback Summary

2020-05-16 Thread godfrey he
hi everyone, I would like to bring up another topic about the return value of TableResult#collect() method. Currently, the return type is `Iterator`, we meet some problems when implementing FLINK-14807[1]. In current design, the sink operator has a buffer pool which buffers the data from

[jira] [Created] (FLINK-17768) UnalignedCheckpointITCase.shouldPerformUnalignedCheckpointOnLocalAndRemoteChannel is instable

2020-05-16 Thread Dian Fu (Jira)
Dian Fu created FLINK-17768: --- Summary: UnalignedCheckpointITCase.shouldPerformUnalignedCheckpointOnLocalAndRemoteChannel is instable Key: FLINK-17768 URL: https://issues.apache.org/jira/browse/FLINK-17768

[jira] [Created] (FLINK-17767) Tumble and Hop window support offset

2020-05-16 Thread hailong wang (Jira)
hailong wang created FLINK-17767: Summary: Tumble and Hop window support offset Key: FLINK-17767 URL: https://issues.apache.org/jira/browse/FLINK-17767 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-17766) Use checkpoint lock instead of fine-grained locking in Kafka AbstractFetcher

2020-05-16 Thread Aljoscha Krettek (Jira)
Aljoscha Krettek created FLINK-17766: Summary: Use checkpoint lock instead of fine-grained locking in Kafka AbstractFetcher Key: FLINK-17766 URL: https://issues.apache.org/jira/browse/FLINK-17766

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Zhu Zhu
+1 Thanks, Zhu Zhu Yun Tang 于2020年5月16日周六 下午11:18写道: > > +1 (non-binding) > > Yun Tang > > From: Benchao Li > Sent: Saturday, May 16, 2020 22:00 > To: dev > Subject: Re: [VOTE] Guarantee that @PublicEvolving classes are API and > binary compatible across bug

[jira] [Created] (FLINK-17765) Verbose client error messages

2020-05-16 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-17765: - Summary: Verbose client error messages Key: FLINK-17765 URL: https://issues.apache.org/jira/browse/FLINK-17765 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-17763) No log files when starting scala-shell

2020-05-16 Thread Jeff Zhang (Jira)
Jeff Zhang created FLINK-17763: -- Summary: No log files when starting scala-shell Key: FLINK-17763 URL: https://issues.apache.org/jira/browse/FLINK-17763 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-17764) Update tips about the default planner when the parameter planner is not recognized

2020-05-16 Thread xushiwei (Jira)
xushiwei created FLINK-17764: Summary: Update tips about the default planner when the parameter planner is not recognized Key: FLINK-17764 URL: https://issues.apache.org/jira/browse/FLINK-17764 Project:

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Yun Tang
+1 (non-binding) Yun Tang From: Benchao Li Sent: Saturday, May 16, 2020 22:00 To: dev Subject: Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v) +1 (non-binding) Piotr Nowojski

[jira] [Created] (FLINK-17762) Postgres Catalog should pass table's primary key to catalogTable

2020-05-16 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-17762: -- Summary: Postgres Catalog should pass table's primary key to catalogTable Key: FLINK-17762 URL: https://issues.apache.org/jira/browse/FLINK-17762 Project: Flink

Re: [DISCUSS] increase "state.backend.fs.memory-threshold" from 1K to 100K

2020-05-16 Thread Yun Tang
If we cannot get rid of union state, I think we should introduce memory control on the serialized TDDs when deploying tasks instead of how union state is implemented when assign state in StateAssignmentOperation. The duplicated TaskStateSnapshot would not really increase much memory as the

[jira] [Created] (FLINK-17761) FutureCompletingBlockingQueue should have a capacity limit.

2020-05-16 Thread Jiangjie Qin (Jira)
Jiangjie Qin created FLINK-17761: Summary: FutureCompletingBlockingQueue should have a capacity limit. Key: FLINK-17761 URL: https://issues.apache.org/jira/browse/FLINK-17761 Project: Flink

[jira] [Created] (FLINK-17760) Rework tests to not rely on legacy scheduling logics in ExecutionGraph anymore

2020-05-16 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-17760: --- Summary: Rework tests to not rely on legacy scheduling logics in ExecutionGraph anymore Key: FLINK-17760 URL: https://issues.apache.org/jira/browse/FLINK-17760 Project: Flink

[jira] [Created] (FLINK-17759) Remove RestartIndividualStrategy

2020-05-16 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-17759: --- Summary: Remove RestartIndividualStrategy Key: FLINK-17759 URL: https://issues.apache.org/jira/browse/FLINK-17759 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-17758) Remove AdaptedRestartPipelinedRegionStrategyNG

2020-05-16 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-17758: --- Summary: Remove AdaptedRestartPipelinedRegionStrategyNG Key: FLINK-17758 URL: https://issues.apache.org/jira/browse/FLINK-17758 Project: Flink Issue Type: Sub-task

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Benchao Li
+1 (non-binding) Piotr Nowojski 于2020年5月16日周六 下午8:10写道: > +1 > > Piotrek > > > On 16 May 2020, at 13:21, Yu Li wrote: > > > > +1 (not sure whether my vote is binding, I guess yes since this is a > > development-oriented vote?) > > > > Minor: > > bq. This means that a version x.y.u is API and

[jira] [Created] (FLINK-17757) Implement format factory for Avro serialization and deseriazation schema of RowData type

2020-05-16 Thread Danny Chen (Jira)
Danny Chen created FLINK-17757: -- Summary: Implement format factory for Avro serialization and deseriazation schema of RowData type Key: FLINK-17757 URL: https://issues.apache.org/jira/browse/FLINK-17757

Re: Google Season of Docs

2020-05-16 Thread Marta Paes Moreira
Hi, Amr! You can certainly apply for Google Season of Docs with the Flink project. All you have to do is have a look at our project ideas [1], think about what you'd be interested in working on and submit your application once the application phase opens (June 9th) [2]. Let me know if there is

Re: regarding Google Season of Docs

2020-05-16 Thread Marta Paes Moreira
Hi, Yuvraj! Thanks for your interest in contributing to Flink during Google Season of Docs! You can start by reading FLIP-60 [1] and thinking about what areas of the documentation you'd like to focus on. There is a lot of work to be done, so there is some flexibility for you to choose topics

[jira] [Created] (FLINK-17756) Drop table/view shouldn't take affect on each other

2020-05-16 Thread Kurt Young (Jira)
Kurt Young created FLINK-17756: -- Summary: Drop table/view shouldn't take affect on each other Key: FLINK-17756 URL: https://issues.apache.org/jira/browse/FLINK-17756 Project: Flink Issue Type:

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Piotr Nowojski
+1 Piotrek > On 16 May 2020, at 13:21, Yu Li wrote: > > +1 (not sure whether my vote is binding, I guess yes since this is a > development-oriented vote?) > > Minor: > bq. This means that a version x.y.u is API and binary compatible to x.y.v > with u <= v wrt all @PublicEvolving classes. > I

Re: Interested In Google Season of Docs!

2020-05-16 Thread Marta Paes Moreira
Hi, Roopal. Thanks for reaching out, we're glad to see that you're interested in giving the Flink docs a try! To participate in Google Season of Docs (GSoD), you'll have to submit an application once the application period opens (June 9th). Because FLIP-60 is very broad, the first step would be

Re: [PROPOSAL] Google Season of Docs 2020.

2020-05-16 Thread Marta Paes Moreira
Thanks, Robert! I made the wrong assumption that most people would be familiar with how mailing lists work and should have been clearer in the announcement. On Sat, May 16, 2020 at 8:35 AM Robert Metzger wrote: > FYI: I'm a moderator of the dev@ list, and we've received about 5 emails > from

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Yu Li
+1 (not sure whether my vote is binding, I guess yes since this is a development-oriented vote?) Minor: bq. This means that a version x.y.u is API and binary compatible to x.y.v with u <= v wrt all @PublicEvolving classes. I guess you mean "with u *>* v" to keep backward compatibility instead of

Running individual test files taking too long

2020-05-16 Thread Manish G
Hi, I am trying to run individual unit test files, and every time it spends lots of time(almost 4-5 minutes) on some build process, and then I get error, like this: Error running 'FlinkKafkaConsumerBaseTest': Command line is too long I recently switched from IntelliJ Ulimate to CE. Can that be

Re: [DISCUSS] increase "state.backend.fs.memory-threshold" from 1K to 100K

2020-05-16 Thread Stephan Ewen
Okay, thank you for all the feedback. So we should definitely work on getting rid of the Union State, or at least change the way it is implemented (avoid duplicate serializer snapshots). Can you estimate from which size of the cluster on the JM heap usage becomes critical (if we increased the

[jira] [Created] (FLINK-17755) Expiring states should be given as side output

2020-05-16 Thread Roey Shem Tov (Jira)
Roey Shem Tov created FLINK-17755: - Summary: Expiring states should be given as side output Key: FLINK-17755 URL: https://issues.apache.org/jira/browse/FLINK-17755 Project: Flink Issue Type:

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Jeff Zhang
Definitely +1 Dian Fu 于2020年5月16日周六 下午5:48写道: > +1 (non-binding) > > Regards, > Dian > > > 在 2020年5月16日,下午2:33,Congxian Qiu 写道: > > > > +1 (non-binding) > > Best, > > Congxian > > > > > > Yangze Guo 于2020年5月16日周六 上午12:51写道: > > > >> +1 > >> > >> Best, > >> Yangze Guo > >> > >> On Sat, May 16,

[jira] [Created] (FLINK-17754) Walkthrough Table Java nightly end-to-end test failed to compile

2020-05-16 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-17754: -- Summary: Walkthrough Table Java nightly end-to-end test failed to compile Key: FLINK-17754 URL: https://issues.apache.org/jira/browse/FLINK-17754 Project: Flink

Re: [DISCUSS] Exact feature freeze date

2020-05-16 Thread Dian Fu
+1 for Monday morning in Europe. Regards, Dian > 在 2020年5月16日,下午1:51,Congxian Qiu 写道: > > +1 for Monday morning in Europe. > Best, > Congxian > > > Danny Chan 于2020年5月16日周六 上午10:40写道: > >> +1 for Monday morning in Europe. >> >> Best, >> Danny Chan >> 在 2020年5月15日 +0800 PM10:51,Zhijiang >

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Dian Fu
+1 (non-binding) Regards, Dian > 在 2020年5月16日,下午2:33,Congxian Qiu 写道: > > +1 (non-binding) > Best, > Congxian > > > Yangze Guo 于2020年5月16日周六 上午12:51写道: > >> +1 >> >> Best, >> Yangze Guo >> >> On Sat, May 16, 2020 at 12:26 AM Yuan Mei wrote: >>> >>> +1 >>> >>> On Sat, May 16, 2020 at

[jira] [Created] (FLINK-17753) watermark defined in ddl does not work in Table api

2020-05-16 Thread godfrey he (Jira)
godfrey he created FLINK-17753: -- Summary: watermark defined in ddl does not work in Table api Key: FLINK-17753 URL: https://issues.apache.org/jira/browse/FLINK-17753 Project: Flink Issue Type:

[jira] [Created] (FLINK-17752) Align the timestamp format with Flink SQL types in JSON format

2020-05-16 Thread Jark Wu (Jira)
Jark Wu created FLINK-17752: --- Summary: Align the timestamp format with Flink SQL types in JSON format Key: FLINK-17752 URL: https://issues.apache.org/jira/browse/FLINK-17752 Project: Flink Issue

[jira] [Created] (FLINK-17751) proctime defined in ddl can't work with over window in Table api

2020-05-16 Thread godfrey he (Jira)
godfrey he created FLINK-17751: -- Summary: proctime defined in ddl can't work with over window in Table api Key: FLINK-17751 URL: https://issues.apache.org/jira/browse/FLINK-17751 Project: Flink

[jira] [Created] (FLINK-17750) YARNHighAvailabilityITCase.testKillYarnSessionClusterEntrypoint failed on azure

2020-05-16 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-17750: - Summary: YARNHighAvailabilityITCase.testKillYarnSessionClusterEntrypoint failed on azure Key: FLINK-17750 URL: https://issues.apache.org/jira/browse/FLINK-17750

Re: [PROPOSAL] Google Season of Docs 2020.

2020-05-16 Thread Robert Metzger
FYI: I'm a moderator of the dev@ list, and we've received about 5 emails from applicants that were not subscribed to the list. Initially, I rejected their messages, asking them to subscribe and send the email again. This has not happened from any of them. That's why I accepted new applicant

Re: [VOTE] Guarantee that @PublicEvolving classes are API and binary compatible across bug fix releases (x.y.u and x.y.v)

2020-05-16 Thread Congxian Qiu
+1 (non-binding) Best, Congxian Yangze Guo 于2020年5月16日周六 上午12:51写道: > +1 > > Best, > Yangze Guo > > On Sat, May 16, 2020 at 12:26 AM Yuan Mei wrote: > > > > +1 > > > > On Sat, May 16, 2020 at 12:21 AM Ufuk Celebi wrote: > > > > > +1 > > > > > > – Ufuk > > > > > > > > > On Fri, May 15, 2020

Re: [DISCUSS] Stability guarantees for @PublicEvolving classes

2020-05-16 Thread Congxian Qiu
Sorry for the late jump in. +1 to keep the compatibility of @PublicEvolving between minor releases(x.y.a -> x.y.b), as a user I always think this as a bug-fix release, break the compatibility between minor releases may give users a surprise. As the previous emails said, how and when will a

Interested In Google Season of Docs!

2020-05-16 Thread Roopal Jain
Hello Flink Dev Community! I am interested in participating in Google Season of Docs for Apache Flink. I went through the FLIP-60 detailed proposal and thought this is something I could do well. I am currently working as a software engineer and have a B.E in Computer Engineering from one of

Google Season of Docs

2020-05-16 Thread Amr Maghraby
Dear Apace Flink, My name is Amr Maghraby, I am a new graduate from AAST college got the first rank on my class with CGPA 3.92 and joined the international competition in the US called ROV got the second worldwide and last summer I have involved in Google Summer of code 2019 and did good work

regarding Google Season of Docs

2020-05-16 Thread Yuvraj Manral
Respected sir/mam, I came around the projects proposed by Apache Flink for Season of Docs 2020. I am a newbie to organisation but really liked the ideas of projects and would love to start contributing and prepare my proposal for Season of Docs. Please guide me through. Where should I start and

Re: [DISCUSS] increase "state.backend.fs.memory-threshold" from 1K to 100K

2020-05-16 Thread Congxian Qiu
Hi, Overall, I agree with increasing this value. but the default value set to 100K maybe something too large from my side. I want to share some more information from my side. The small files problem is indeed a problem many users may encounter in production env. The states(Keyed state and