[jira] [Created] (FLINK-15065) RocksDB configurable options doc description error

YufeiLiu created FLINK-15065: Summary: RocksDB configurable options doc description error Key: FLINK-15065 URL: https://issues.apache.org/jira/browse/FLINK-15065 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-15064) Remove XmlOutput util class in blink planner

Jark Wu created FLINK-15064: --- Summary: Remove XmlOutput util class in blink planner Key: FLINK-15064 URL: https://issues.apache.org/jira/browse/FLINK-15064 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-15063) input group and output group of the task metric are reversed

lining created FLINK-15063: -- Summary: input group and output group of the task metric are reversed Key: FLINK-15063 URL: https://issues.apache.org/jira/browse/FLINK-15063 Project: Flink Issue

[jira] [Created] (FLINK-15062) Orc reader should use java.sql.Timestamp to read for respecting time zone

Jingsong Lee created FLINK-15062: Summary: Orc reader should use java.sql.Timestamp to read for respecting time zone Key: FLINK-15062 URL: https://issues.apache.org/jira/browse/FLINK-15062 Project:

Re: [VOTE] Improve the Pyflink command line options (Adjustment to FLIP-78)

+1 (non-binding) Regards, Dian > 在 2019年12月5日，上午11:11，Jark Wu 写道： > > +1 (binding) > > Best, > Jark > > On Thu, 5 Dec 2019 at 10:45, Wei Zhong wrote: > >> Hi all, >> >> According to our previous discussion in [1], I'd like to bring up a vote >> to apply the adjustment [2] to the

Re: [VOTE] Improve the Pyflink command line options (Adjustment to FLIP-78)

+1 (binding) Best, Jark On Thu, 5 Dec 2019 at 10:45, Wei Zhong wrote: > Hi all, > > According to our previous discussion in [1], I'd like to bring up a vote > to apply the adjustment [2] to the command-line option design of FLIP-78 > [3]. > > The vote will be open for at least 72 hours unless

Re: [DISCUSS] Drop Kafka 0.8/0.9

+1 for dropping them. Just FYI: there was a similar discussion few months ago [1]. [1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/DISCUSS-Drop-older-versions-of-Kafka-Connectors-0-9-0-10-for-Flink-1-10-td29916.html#a29997

[jira] [Created] (FLINK-15061) create/alter table/databases properties should be case sensitive stored in catalog

Terry Wang created FLINK-15061: -- Summary: create/alter table/databases properties should be case sensitive stored in catalog Key: FLINK-15061 URL: https://issues.apache.org/jira/browse/FLINK-15061

[VOTE] Improve the Pyflink command line options (Adjustment to FLIP-78)

Hi all, According to our previous discussion in [1], I'd like to bring up a vote to apply the adjustment [2] to the command-line option design of FLIP-78 [3]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. Best, Wei [1]

Re: [DISCUSS] Improve the Pyflink command line options (Adjustment to FLIP-78)

Hi all, Thanks for all of your quick response! I will bring up the VOTE. Best, Wei > 在 2019年12月5日，10:39，Jark Wu 写道： > > Hi Wei, > > Thanks for bringing this discussion up, the changes look good to me. > Looking forward to the vote. And you can prepare the pull request at the > same time (in

Re: [DISCUSS] Improve the Pyflink command line options (Adjustment to FLIP-78)

Hi Wei, Thanks for bringing this discussion up, the changes look good to me. Looking forward to the vote. And you can prepare the pull request at the same time (in order to checkin in time). Best, Jark On Thu, 5 Dec 2019 at 10:27, Hequn Cheng wrote: > Hi all, > > Thanks a lot for the

Re: [VOTE] Multi-topics consuming from KafkaTableSource

Hi Leo, IMO, if you want to vote a proposal. It would be better to start a new VOTE ML thread. Best, Vino leo 于2019年12月5日周四 上午9:07写道： > Hi everyone, > > > Please vote for this FLINK-14729. The JIRA can be found here, > > https://issues.apache.org/jira/browse/FLINK-14729 > > The discussion can

Re: [VOTE] FLIP-27 - Refactor Source Interface

+1 (binding), and looking forward to seeing the new interface in the master. Best, Jincheng Becket Qin 于2019年12月5日周四 上午8:05写道： > Hi all, > > I would like to start the vote for FLIP-27 which proposes to introduce a > new Source connector interface to address a few problems in the existing >

Re: [DISCUSS] Drop Kafka 0.8/0.9

+1 jincheng sun 于2019年12月5日周四 上午10:26写道： > +1 for drop it, and Thanks for bring up this discussion Chesnay! > > Best, > Jincheng > > Jark Wu 于2019年12月5日周四 上午10:19写道： > >> +1 for dropping, also cc'ed user mailing list. >> >> >> Best, >> Jark >> >> On Thu, 5 Dec 2019 at 03:39, Konstantin Knauf

Re: [DISCUSS] Improve the Pyflink command line options (Adjustment to FLIP-78)

Hi all, Thanks a lot for the discussion! Using "#" also makes sense to me. And +1 to have these improvements in 1.10 as we don't want to introduce compatibility problems later. Looking forward to the vote! Best, Hequn On Thu, Dec 5, 2019 at 10:02 AM jincheng sun wrote: > Hi all, > > Thanks

Re: [DISCUSS] Drop Kafka 0.8/0.9

+1 for drop it, and Thanks for bring up this discussion Chesnay! Best, Jincheng Jark Wu 于2019年12月5日周四 上午10:19写道： > +1 for dropping, also cc'ed user mailing list. > > > Best, > Jark > > On Thu, 5 Dec 2019 at 03:39, Konstantin Knauf > wrote: > > > Hi Chesnay, > > > > +1 for dropping. I have

Re: [DISCUSS] Improve the Pyflink command line options (Adjustment to FLIP-78)

Thanks for bringing up this discussion Wei. +1 for this proposal! As these options are proposed in 1.10, it will be great if we can improve them in 1.10. Then it will not cause compatible issues. Thanks, Dian > 在 2019年12月5日，上午10:01，jincheng sun 写道： > > Hi all, > > Thanks for the quick

Re: [DISCUSS] Drop Kafka 0.8/0.9

+1 for dropping, also cc'ed user mailing list. Best, Jark On Thu, 5 Dec 2019 at 03:39, Konstantin Knauf wrote: > Hi Chesnay, > > +1 for dropping. I have not heard from any user using 0.8 or 0.9 for a long > while. > > Cheers, > > Konstantin > > On Wed, Dec 4, 2019 at 1:57 PM Chesnay Schepler

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

+1 for Azure pipeline because it promises better performance. However, I have 2 concerns: 1) Travis provides personal free service for testing personal branches. Usually, contributors use this feature to test PoC or run CRON jobs for pull requests. Using local machine will cost a lot of

Re: [DISCUSS] Improve the Pyflink command line options (Adjustment to FLIP-78)

Hi all, Thanks for the quick response Aljoscha & Wei ! It seems unify the options is necessary, and 1.10 will be code frozen. I would be like to bring up the VOTE thread for this change ASAP, and more detail can continue discuss in the PR. What do you think? Best, Jincheng Aljoscha Krettek

[jira] [Created] (FLINK-15060) Add aarch64 support for tpcds e2e test

wangxiyuan created FLINK-15060: -- Summary: Add aarch64 support for tpcds e2e test Key: FLINK-15060 URL: https://issues.apache.org/jira/browse/FLINK-15060 Project: Flink Issue Type: Sub-task

Re: [VOTE] Release 1.8.3, release candidate #2

Thanks, Till. I will kick off the next RC soon. Best, Hequn On Thu, Dec 5, 2019 at 5:20 AM Till Rohrmann wrote: > Hi everyone, > > the fix for FLINK-15036 has been merged into the release-1.8 branch. Hence > we can kick off the next RC. > > Cheers, > Till > > On Wed, Dec 4, 2019 at 11:54 AM

[VOTE] Multi-topics consuming from KafkaTableSource

Hi everyone, Please vote for this FLINK-14729. The JIRA can be found here, https://issues.apache.org/jira/browse/FLINK-14729 The discussion can be found here,

Re: [DISCUSS] Multi-topics consuming from KafkaTableSource

Hi all, Thanks for your involvement. The benefits can be summarized here for a multi-topics consuming via KafkaTableSource, First, It boosts developing and managing a source with so many topics, especially when the schema of topics is the same. Second, It is aligned with existed

[VOTE] FLIP-27 - Refactor Source Interface

Hi all, I would like to start the vote for FLIP-27 which proposes to introduce a new Source connector interface to address a few problems in the existing source connector. The main goals of the the FLIP are following: 1. Unify the Source interface in Flink for batch and stream. 2. Significantly

[jira] [Created] (FLINK-15059) DataTypeExtractorTest fails on travis

Dawid Wysakowicz created FLINK-15059: Summary: DataTypeExtractorTest fails on travis Key: FLINK-15059 URL: https://issues.apache.org/jira/browse/FLINK-15059 Project: Flink Issue Type:

Re: [VOTE] Release 1.8.3, release candidate #2

Hi everyone, the fix for FLINK-15036 has been merged into the release-1.8 branch. Hence we can kick off the next RC. Cheers, Till On Wed, Dec 4, 2019 at 11:54 AM Hequn Cheng wrote: > Hi everyone, > > Unfortunately, we need to cancel this vote due to a new critical issue( > FLINK-15036

Re: Checkpointing under backpressure

Hi Arvid, Thanks for putting together the proposal [1] I'm planning to take a closer look in the next few days. Has any of the work been translated to JIRAs yet and what would be the approximate target release? Thanks, Thomas [1]

Re: [DISCUSS] Drop Kafka 0.8/0.9

Hi Chesnay, +1 for dropping. I have not heard from any user using 0.8 or 0.9 for a long while. Cheers, Konstantin On Wed, Dec 4, 2019 at 1:57 PM Chesnay Schepler wrote: > Hello, > > What's everyone's take on dropping the Kafka 0.8/0.9 connectors from the > Flink codebase? > > We haven't

[jira] [Created] (FLINK-15058) Log required config keys if TaskManager memory configuration is invalid

Gary Yao created FLINK-15058: Summary: Log required config keys if TaskManager memory configuration is invalid Key: FLINK-15058 URL: https://issues.apache.org/jira/browse/FLINK-15058 Project: Flink

[jira] [Created] (FLINK-15057) Set taskmanager.memory.total-process.size in jepsen tests

Gary Yao created FLINK-15057: Summary: Set taskmanager.memory.total-process.size in jepsen tests Key: FLINK-15057 URL: https://issues.apache.org/jira/browse/FLINK-15057 Project: Flink Issue

[jira] [Created] (FLINK-15056) Python test: ExecutionEnvironmentCompletenessTests.test_completeness failed

Robert Metzger created FLINK-15056: -- Summary: Python test: ExecutionEnvironmentCompletenessTests.test_completeness failed Key: FLINK-15056 URL: https://issues.apache.org/jira/browse/FLINK-15056

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

+1 Till Rohrmann 于2019年12月4日周三 下午10:43写道： > +1 for moving to Azure pipelines as it promises better scalability and > tooling. Looking forward to having faster builds and hence shorter feedback > cycles :-) > > Cheers, > Till > > On Wed, Dec 4, 2019 at 1:24 PM Chesnay Schepler > wrote: > > >

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

+1 for moving to Azure pipelines as it promises better scalability and tooling. Looking forward to having faster builds and hence shorter feedback cycles :-) Cheers, Till On Wed, Dec 4, 2019 at 1:24 PM Chesnay Schepler wrote: > @robert Can you expand how the azure setup interacts with CiBot?

[jira] [Created] (FLINK-15055) YarnDistributedCacheITCase does not generate logs.

Xintong Song created FLINK-15055: Summary: YarnDistributedCacheITCase does not generate logs. Key: FLINK-15055 URL: https://issues.apache.org/jira/browse/FLINK-15055 Project: Flink Issue

Re: [DISCUSS] Add N-Ary Stream Operator

Hi Jingsong, Thanks for the feedback :) Could you clarify a little bit what do you mean by your wished use cases? > There are a large number jobs (in production environment) that their > TwoInputOperators that can be chained. We used to only watch the last > ten tasks transmit data through

[jira] [Created] (FLINK-15054) Add FAQ section to IntelliJ setup

Chesnay Schepler created FLINK-15054: Summary: Add FAQ section to IntelliJ setup Key: FLINK-15054 URL: https://issues.apache.org/jira/browse/FLINK-15054 Project: Flink Issue Type:

[jira] [Created] (FLINK-15053) Configurations with values contains space may cause TM failures on Yarn

Xintong Song created FLINK-15053: Summary: Configurations with values contains space may cause TM failures on Yarn Key: FLINK-15053 URL: https://issues.apache.org/jira/browse/FLINK-15053 Project:

Re: [DISCUSS] Expose or setup a secur...@flink.apache.org mailing list for security report and discussion

Turns out we already have a link to the Apache security page; in the Apache section at the very bottom of the sidebar. If I open the page it is unfortunately not visible...there are too many things in the sidebar. Nevertheless an additional entry as done in the PR cannot hurt. I'm taking a

Re: [DISCUSS] Add N-Ary Stream Operator

Hi Piotr, Huge +1 for N-Ary Stream Operator. And I love this Golden Shovel award very much! There are a large number jobs (in production environment) that their TwoInputOperators that can be chained. We used to only watch the last ten tasks transmit data through disk and network, which could

[DISCUSS] Drop Kafka 0.8/0.9

Hello, What's everyone's take on dropping the Kafka 0.8/0.9 connectors from the Flink codebase? We haven't touched either of them for the 1.10 release, and it seems quite unlikely that we will do so in the future. We could finally close a number of test stability tickets that have been

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

@robert Can you expand how the azure setup interacts with CiBot? Do we have to continue mirroring builds into flink-ci? How will the cronjob configuration work? We should have a general idea on how to implement this before proceeding. Additionally, moving /all /jobs into flink-ci requires

[jira] [Created] (FLINK-15052) sql client doesn't clear previous job graph

Kurt Young created FLINK-15052: -- Summary: sql client doesn't clear previous job graph Key: FLINK-15052 URL: https://issues.apache.org/jira/browse/FLINK-15052 Project: Flink Issue Type: Bug

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

From what I've seen so far Azure will provide us a better experience, so I'd say +1 for the transition as a whole. I'd delay merge at least until the feature branch is cut. Given the parental leave it may even make sense to only start merging in January afterwards, to reduce the total time

Re: [DISCUSS] Voting from apache.org addresses

Hi Dawid, Thanks for the reply. Counting all the votes from non apache addresses as non-binding makes sense. Just as Jark mentioned, we can always remind the committer/PMC to vote again to use the apache address if necessary (i.e. when the number of binding votes is not enough). Thanks, Dian

[jira] [Created] (FLINK-15051) Typo in RocksDBStateBackend getNumberOfTransferingThread

Stephan Ewen created FLINK-15051: Summary: Typo in RocksDBStateBackend getNumberOfTransferingThread Key: FLINK-15051 URL: https://issues.apache.org/jira/browse/FLINK-15051 Project: Flink

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

Thanks Robert for driving this. There is another big pain point of current travis, which is its cache mechanism will fail from time to time. Almost around 50% of the build fails are caused by cache problem. I opened this issue to travis but got no response yet. So big +1 from my side. Just one

Re: [DISCUSS] Voting from apache.org addresses

+1 (from my apache email ;-)) Best, Kurt On Wed, Dec 4, 2019 at 7:22 PM Jark Wu wrote: > I'm +1 on this proposal. > > Regarding to the case that Dian mentioned, we can reminder the > committer/PMC to vote again use the apache email, > and of course the non-apache vote is counted as

Re: [DISCUSS] Voting from apache.org addresses

I'm +1 on this proposal. Regarding to the case that Dian mentioned, we can reminder the committer/PMC to vote again use the apache email, and of course the non-apache vote is counted as non-binding. Best, Jark On Wed, 4 Dec 2019 at 17:33, Dawid Wysakowicz wrote: > Hi Dian, > > I don't want to

Re: [DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

Thanks Robert for the updates! And thanks a lot for all the efforts to investigate, experiment and tune Azure Pipelines for Flink building. Big +1 for it. It would be great that the community building can be extended with custom machines so that the tests would not be queued for long with daily

Re: [DISCUSS] FLIP-27: Refactor Source Interface

Thanks Stephan, I have to mention that most of the design work and FLIP wiki had actually been done by Aljoscha, Biao and you, before I picked up this FLIP. Given that this FLIP has gone through an extended discussion and release 1.10 code freeze is approaching, I'd like to start a vote thread

Re: [DISCUSS] FLIP-27: Refactor Source Interface

Hi Jiayi, For now there is no communication between the coordinators. And I do see some use cases if we can open up that channel. But it won't be in this FLIP. Thanks, Jiangjie (Becket) Qin On Wed, Dec 4, 2019 at 6:53 PM bupt_ljy wrote: > Hi Becket, > > > Thanks for updating the progress! >

Re: Building with Hadoop 3

Hi Marton, Thanks for your explanation. Personally, I look forward to your contribution! Best, Vino Márton Balassi 于2019年12月4日周三 下午5:15写道： > Wearing my Cloudera hat I can tell you that we have done this exercise for > our distros of the 3.0 and 3.1 Hadoop versions. We have not contributed >

Re: [VOTE] Release 1.8.3, release candidate #2

Hi everyone, Unfortunately, we need to cancel this vote due to a new critical issue( FLINK-15036 ) we have spotted recently. Since 1.8.3 will most likely be our last bugfix release for Flink 1.8, we think it is good to have the fix in 1.8.3. I

Re: [DISCUSS] FLIP-27: Refactor Source Interface

Hi Becket, Thanks for updating the progress! I have a question about the #OperatorCoordinator. Will there be any communication between different #OperatorCoordinators (or in the future plan)? Because in that way it may be able to cover some cases in FLIP-27[1] like initializing static data

[DISCUSS] Migrate build infrastructure from Travis CI to Azure Pipelines

Hi all, as a follow up from our discussion on reducing the build time [1], I would like to propose migrating our build infrastructure to Azure Pipelines (away from Travis). I believe that we have reached the limits of what Travis can provide the Flink community, and I don't want the build system

Re: [DISCUSS] FLIP-27: Refactor Source Interface

Thanks, Becket, for updating this. I agree with moving the aspects you mentioned into separate FLIPs - this one way becoming unwieldy in size. +1 to the FLIP in its current state. Its a very detailed write-up, nicely done! On Wed, Dec 4, 2019 at 7:38 AM Becket Qin wrote: > Hi all, > > Sorry

Re: [DISCUSS] Voting from apache.org addresses

Hi Dian, I don't want to be very strict, but I think it should be counted as non-binding, if it comes from non apache address, yes. Anybody should be able to verify a vote. Moreover I think this the only way to "encourage" all committers to use their apache addresses ;) Best, Dawid On

Re: [DISCUSS] Voting from apache.org addresses

Thanks for your explanation Dawid! It makes sense to me now. +1. Just one minor question: Does this mean that if a committer/PMC accidentally votes using the non apache email, even if the person who summarizes the votes clearly KNOWS who he/she is, that vote will still be counted as

Re: [DISCUSS] Add N-Ary Stream Operator

Hi, First and foremost I would like to nominate myself to the Golden Shovel award for digging out this topic: Secondly, I would like to discuss coming back to this particular idea of implementing N-Ary Stream Operator. This time motivation doesn’t come from the Side Inputs, but to

Re: Building with Hadoop 3

Wearing my Cloudera hat I can tell you that we have done this exercise for our distros of the 3.0 and 3.1 Hadoop versions. We have not contributed these back just yet, but we are open to do so. If the community is interested we can contribute those changes back to flink-shaded and suggest the

Re: [DISCUSS] Voting from apache.org addresses

Very sensible! +1 > On 4. Dec 2019, at 10:02, Chesnay Schepler wrote: > > I believe this to be a sensible approach by Dawid; +1. > > On 04/12/2019 09:04, Dawid Wysakowicz wrote: >> >> Hi all, >> >> Sorry I think I was not clear enough on my initial e-mail. Let me first >> clarify two things

Re: [DISCUSS] Improve the Pyflink command line options (Adjustment to FLIP-78)

Perfect, thanks for the background info! I also found this section now, which mentions that it comes from Hadoop: https://spark.apache.org/docs/latest/running-on-yarn.html#important-notes. I think the proposed changes are good! Best, Aljoscha > On 4. Dec 2019, at 04:34, Wei Zhong wrote: > >

Re: [DISCUSS] Voting from apache.org addresses

I believe this to be a sensible approach by Dawid; +1. On 04/12/2019 09:04, Dawid Wysakowicz wrote: Hi all, Sorry I think I was not clear enough on my initial e-mail. Let me first clarify two things and later on try to rephrase my initial suggestion. 1. I do not want to count all votes

Re: [DISCUSS] Voting from apache.org addresses

Hi Dawid, Thanks for you explanation. You mean that "people who vote for binding +1 need to use apache email addresses". Although I can't vote binding +1, I am +1 for this suggestion. Best, Jingsong Lee On Wed, Dec 4, 2019 at 4:04 PM Dawid Wysakowicz wrote: > Hi all, > > Sorry I think I was

Re: Building with Hadoop 3

There's no JIRA and no one actively working on it. I'm not aware of any investigations on the matter; hence the first step would be to just try it out. A flink-shaded artifact isn't a hard requirement; Flink will work with any 2.X hadoop distribution (provided that there aren't any dependency

Re: [DISCUSS] Voting from apache.org addresses

Hi all, Sorry I think I was not clear enough on my initial e-mail. Let me first clarify two things and later on try to rephrase my initial suggestion. 1. I do not want to count all votes from @apache.org addresses as binding 2. I do not want to discourage people that do not have @apache.org