Re: [VOTE] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-23 Thread Yang Wang
+1 (non-binding) Best, Yang 刘建刚 于2021年6月24日周四 下午12:17写道: > +1 (binding) > > Thanks > liujiangang > > Zhu Zhu 于2021年6月24日周四 上午11:38写道: > > > +1 (binding) > > > > Thanks, > > Zhu > > > > Yangze Guo 于2021年6月21日周一 下午3:42写道: > > > > > According to the latest comment of Zhu Zhu[1], I append the

Re: [VOTE] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-23 Thread 刘建刚
+1 (binding) Thanks liujiangang Zhu Zhu 于2021年6月24日周四 上午11:38写道: > +1 (binding) > > Thanks, > Zhu > > Yangze Guo 于2021年6月21日周一 下午3:42写道: > > > According to the latest comment of Zhu Zhu[1], I append the potential > > resource deadlock in batch jobs as a known limitation to this FLIP. > >

Re: [VOTE] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-23 Thread Zhu Zhu
+1 (binding) Thanks, Zhu Yangze Guo 于2021年6月21日周一 下午3:42写道: > According to the latest comment of Zhu Zhu[1], I append the potential > resource deadlock in batch jobs as a known limitation to this FLIP. > Thus, I'd extend the voting period for another 72h. > > [1] >

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-23 Thread Jark Wu
Thanks to Xintong for bringing up this topic, I'm +1 in general. However, I think it's still not very clear how we address the unstable tests. I think this is a very important part of this new guideline. According to the discussion above, if some tests are unstable, we can manually disable it.

[jira] [Created] (FLINK-23132) flink upgrade issue(1.11.3->1.13.0)

2021-06-23 Thread Jeff Hu (Jira)
Jeff Hu created FLINK-23132: --- Summary: flink upgrade issue(1.11.3->1.13.0) Key: FLINK-23132 URL: https://issues.apache.org/jira/browse/FLINK-23132 Project: Flink Issue Type: Bug

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-23 Thread Xintong Song
Thanks all for the feedback. @Till @Yangze I'm also not convinced by the idea of having an exception for local builds. We need to execute the entire build (or at least the failing stage) locally, to make sure subsequent test cases prevented by the failure one are all executed. In that case, it's

[jira] [Created] (FLINK-23131) Remove scala from plugin parent-first patterns

2021-06-23 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23131: Summary: Remove scala from plugin parent-first patterns Key: FLINK-23131 URL: https://issues.apache.org/jira/browse/FLINK-23131 Project: Flink Issue

[jira] [Created] (FLINK-23130) RestServerEndpoint references on netty 3

2021-06-23 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23130: Summary: RestServerEndpoint references on netty 3 Key: FLINK-23130 URL: https://issues.apache.org/jira/browse/FLINK-23130 Project: Flink Issue Type:

[jira] [Created] (FLINK-23129) When cancelling any running job of multiple jobs in an application cluster, JobManager shuts down

2021-06-23 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-23129: -- Summary: When cancelling any running job of multiple jobs in an application cluster, JobManager shuts down Key: FLINK-23129 URL:

Re: [DISCUSS] Dashboard/HistoryServer authentication

2021-06-23 Thread Austin Cawley-Edwards
Hi all, Thanks, Konstantin and Till, for guiding the discussion. I was not aware of the results of the call with Konstantin and was attempting to resolve the unanswered questions before more, potentially fruitless, work was done. I am also looking forward to the coming proposal, as well as

[jira] [Created] (FLINK-23128) Translate update to operations playground docs to Chinese

2021-06-23 Thread David Anderson (Jira)
David Anderson created FLINK-23128: -- Summary: Translate update to operations playground docs to Chinese Key: FLINK-23128 URL: https://issues.apache.org/jira/browse/FLINK-23128 Project: Flink

[jira] [Created] (FLINK-23127) Can't use plugins for GCS filesystem

2021-06-23 Thread Yaroslav Tkachenko (Jira)
Yaroslav Tkachenko created FLINK-23127: -- Summary: Can't use plugins for GCS filesystem Key: FLINK-23127 URL: https://issues.apache.org/jira/browse/FLINK-23127 Project: Flink Issue Type:

[jira] [Created] (FLINK-23126) Refactor smoke-e2e into smoke-e2e-common and smoke-e2e-embedded

2021-06-23 Thread Evans Ye (Jira)
Evans Ye created FLINK-23126: Summary: Refactor smoke-e2e into smoke-e2e-common and smoke-e2e-embedded Key: FLINK-23126 URL: https://issues.apache.org/jira/browse/FLINK-23126 Project: Flink

[jira] [Created] (FLINK-23125) Run StateFun smoke E2E tests for multiple language SDKs

2021-06-23 Thread Evans Ye (Jira)
Evans Ye created FLINK-23125: Summary: Run StateFun smoke E2E tests for multiple language SDKs Key: FLINK-23125 URL: https://issues.apache.org/jira/browse/FLINK-23125 Project: Flink Issue Type:

Re: [DISCUSS] [FLINK-23122] Provide the Dynamic register converter

2021-06-23 Thread Jark Wu
Hi, `TIMESTAMP_WITH_TIME_ZONE` is not supported in the Flink SQL engine, even though it is listed in the type API. I think what you are looking for is the RawValueType which can be used as user-defined type. You can use `DataTypes.RAW(TypeInformation)` to define a Raw type with the given

[jira] [Created] (FLINK-23124) Implement exactly-once Kafka Sink

2021-06-23 Thread Fabian Paul (Jira)
Fabian Paul created FLINK-23124: --- Summary: Implement exactly-once Kafka Sink Key: FLINK-23124 URL: https://issues.apache.org/jira/browse/FLINK-23124 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23123) Implement at-least-once Kafka Sink

2021-06-23 Thread Fabian Paul (Jira)
Fabian Paul created FLINK-23123: --- Summary: Implement at-least-once Kafka Sink Key: FLINK-23123 URL: https://issues.apache.org/jira/browse/FLINK-23123 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-23 Thread JING ZHANG
Hi Xintong, +1 to the proposal. In order to better comply with the rule, it is necessary to describe what's best practice if encountering test failure which seems unrelated with the current commits. How to avoid merging PR with test failures and not blocking code merging for a long time? I tried

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Chesnay Schepler
If we want to publicize this plan more shouldn't we have a rough timeline for when 2.0 is on the table? On 6/23/2021 2:44 PM, Stephan Ewen wrote: Thanks for writing this up, this also reflects my understanding. I think a blog post would be nice, ideally with an explicit call for feedback so

[DISCUSS] [FLINK-23122] Provide the Dynamic register converter

2021-06-23 Thread 云华
Hi everyone, I want to rework type conversion system in connector and flink table module to be resuable and scalability. I Postgres system, the type '_citext' will not supported in org.apache.flink.connector.jdbc.catalog.PostgresCatalog#fromJDBCType. what's more,

[jira] [Created] (FLINK-23122) Provide the Dynamic register converter

2021-06-23 Thread lqjacklee (Jira)
lqjacklee created FLINK-23122: - Summary: Provide the Dynamic register converter Key: FLINK-23122 URL: https://issues.apache.org/jira/browse/FLINK-23122 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Stephan Ewen
Thanks for writing this up, this also reflects my understanding. I think a blog post would be nice, ideally with an explicit call for feedback so we learn about user concerns. A blog post has a lot more reach than an ML thread. Best, Stephan On Wed, Jun 23, 2021 at 12:23 PM Timo Walther

Re: [DISCUSS] Feedback Collection Jira Bot

2021-06-23 Thread JING ZHANG
Hi Konstantin, Chesnay, > I would like it to not unassign people if a PR is open. These are > usually blocked by the reviewer, not the assignee, and having the > assignees now additionally having to update JIRA periodically is a bit > like rubbing salt into the wound. I agree with Chesnay about

Re: [DISCUSS] Drop Mesos in 1.14

2021-06-23 Thread Stephan Ewen
I would prefer to remove Mesos from the Flink core as well. I also had a similar thought as Seth: As far as I know, you can package applications to run on Mesos with "Marathon". That would be like deploying an opaque Flink standalone cluster on Mesos The implication is similar to going from an

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-23 Thread Stephan Ewen
+1 to Xintong's proposal On Wed, Jun 23, 2021 at 1:53 PM Till Rohrmann wrote: > I would first try to not introduce the exception for local builds. It makes > it quite hard for others to verify the build and to make sure that the > right things were executed. If we see that this becomes an issue

Re: Change in accumutors semantics with jobClient

2021-06-23 Thread Till Rohrmann
Yes, it should be part of the release notes where this change was introduced. I'll take a look at your PR. Thanks a lot Etienne. Cheers, Till On Wed, Jun 23, 2021 at 12:29 PM Etienne Chauchot wrote: > Hi Till, > > Of course I can update the release notes. > > Question is: this change is quite

Re: [DISCUSS] Do not merge PRs with "unrelated" test failures.

2021-06-23 Thread Till Rohrmann
I would first try to not introduce the exception for local builds. It makes it quite hard for others to verify the build and to make sure that the right things were executed. If we see that this becomes an issue then we can revisit this idea. Cheers, Till On Wed, Jun 23, 2021 at 4:19 AM Yangze

Re: [DISCUSS] Feedback Collection Jira Bot

2021-06-23 Thread Piotr Nowojski
> I don't understand why we are necessarily losing discussion/knowledge. The > tickets are still there, just in "Closed" state, which are included in > default Jira search. Finding if there already has been a ticket opened for the given issue is not always easy. Finding the right ticket among

Re: Change in accumutors semantics with jobClient

2021-06-23 Thread Etienne Chauchot
Hi Till, Of course I can update the release notes. Question is: this change is quite old (January), it is already available in all the maintained releases :1.11, 1.12, 1.13. I think I should update the release notes for all these versions no ? In case you agree, I took the liberty to update

[DISCUSS] Incrementally deprecating the DataSet API

2021-06-23 Thread Timo Walther
Hi everyone, I'm sending this email to make sure everyone is on the same page about slowly deprecating the DataSet API. There have been a few thoughts mentioned in presentations, offline discussions, and JIRA issues. However, I have observed that there are still some concerns or different

[jira] [Created] (FLINK-23121) Fix the issue that the InternalRow as arguments in Python UDAF

2021-06-23 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-23121: Summary: Fix the issue that the InternalRow as arguments in Python UDAF Key: FLINK-23121 URL: https://issues.apache.org/jira/browse/FLINK-23121 Project: Flink

Re: CONTENTS DELETED in nabble frontend

2021-06-23 Thread Robert Metzger
I've set up the nabble archives back in the stone age of Flink, when the Apache archive didn't provide a very modern user experience. Since lists.apache.org exists, we don't really need nabble anymore. I'll open a pull request to replace the links to nabble to point to lists.apache.org on the

Re: [DISCUSS] Drop Mesos in 1.14

2021-06-23 Thread Leonard Xu
+ 1 for dropping Mesos. I checked both commit history and mail list, the Mesos related issue/user question has been rarely appeared. Best, Leonard

Re: [DISCUSS] Drop Mesos in 1.14

2021-06-23 Thread Fabian Paul
+ 1 for dropping mesos. Most of the PMCs have already left the project [1] and a move to attic was barely avoided. Overall kubernetes has taken its place and it is unlikely that we will see a surge in Mesos very soon. Best, Fabian [1]

[jira] [Created] (FLINK-23120) ByteArrayWrapperSerializer.serialize should use writeInt to serialize the length

2021-06-23 Thread Dian Fu (Jira)
Dian Fu created FLINK-23120: --- Summary: ByteArrayWrapperSerializer.serialize should use writeInt to serialize the length Key: FLINK-23120 URL: https://issues.apache.org/jira/browse/FLINK-23120 Project:

Re: [DISCUSS] Drop Mesos in 1.14

2021-06-23 Thread Yang Wang
+1 for dropping mesos support. AFAIK, mesos(including marathon for the container management) is phasing out gradually and has been replaced with Kubernetes in the containerized world. Best, Yang Matthias Pohl 于2021年6月23日周三 下午2:04写道: > +1 for dropping Mesos support. There was no feedback

[jira] [Created] (FLINK-23119) Fix the issue that the exception that General Python UDAF is unsupported is not thrown in Compile Stage.

2021-06-23 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-23119: Summary: Fix the issue that the exception that General Python UDAF is unsupported is not thrown in Compile Stage. Key: FLINK-23119 URL:

Re: [DISCUSS] Feedback Collection Jira Bot

2021-06-23 Thread Konstantin Knauf
> I agree there are such tickets, but I don't see how this is addressing my concerns. There are also tickets that just shouldn't be closed as I described above. Why do you think that duplicating tickets and losing discussions/knowledge is a good solution? I don't understand why we are necessarily

[jira] [Created] (FLINK-23118) Drop mesos

2021-06-23 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23118: Summary: Drop mesos Key: FLINK-23118 URL: https://issues.apache.org/jira/browse/FLINK-23118 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Feedback Collection Jira Bot

2021-06-23 Thread Piotr Nowojski
Hi Konstantin, > In my opinion it is important that we close tickets eventually. There are a > lot of tickets (bugs, improvements, tech debt) that over time became > irrelevant, out-of-scope, irreproducible, etc. In my experience, these > tickets are usually not closed by anyone but the bot. I

Re: CONTENTS DELETED in nabble frontend

2021-06-23 Thread Dawid Wysakowicz
Hey, As far as I know the official Apache ML archive can be accessed here[1]. Personally I don't know what is the status of the nabble archives. Best, Dawid [1] https://lists.apache.org/list.html?dev@flink.apache.org On 23/06/2021 09:08, Matthias Pohl wrote: > Thanks for pointing to the

[jira] [Created] (FLINK-23117) TaskExecutor.allocateSlot is a logical error

2021-06-23 Thread zhouzhengde (Jira)
zhouzhengde created FLINK-23117: --- Summary: TaskExecutor.allocateSlot is a logical error Key: FLINK-23117 URL: https://issues.apache.org/jira/browse/FLINK-23117 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-172: Support custom transactional.id prefix in FlinkKafkaProducer

2021-06-23 Thread Piotr Nowojski
Hi, +1 from my side on this idea. I do not see any problems that could be caused by this change. Best, Piotrek śr., 23 cze 2021 o 08:59 Stephan Ewen napisał(a): > The motivation and the proposal sound good to me, +1 from my side. > > Would be good to have a quick opinion from someone who

Re: [DISCUSS] Feedback Collection Jira Bot

2021-06-23 Thread Chesnay Schepler
I would like it to not unassign people if a PR is open. These are usually blocked by the reviewer, not the assignee, and having the assignees now additionally having to update JIRA periodically is a bit like rubbing salt into the wound. On 6/23/2021 7:52 AM, Konstantin Knauf wrote: Hi

[jira] [Created] (FLINK-23116) Update documentation about TableDescriptors

2021-06-23 Thread Timo Walther (Jira)
Timo Walther created FLINK-23116: Summary: Update documentation about TableDescriptors Key: FLINK-23116 URL: https://issues.apache.org/jira/browse/FLINK-23116 Project: Flink Issue Type:

[jira] [Created] (FLINK-23115) Expose new APIs from PyFlink

2021-06-23 Thread Jira
Ingo Bürk created FLINK-23115: - Summary: Expose new APIs from PyFlink Key: FLINK-23115 URL: https://issues.apache.org/jira/browse/FLINK-23115 Project: Flink Issue Type: Sub-task

Re: CONTENTS DELETED in nabble frontend

2021-06-23 Thread Matthias Pohl
Thanks for pointing to the Nabble support forum. +1 Based on [1], the deletion of posts is not related to the switch of mailing lists becoming regular forums. But it seems to be a general issue at Nabble. But what concerns me is [2]: It looks like they are planning to remove the feature to post

Re: [DISCUSS] FLIP-172: Support custom transactional.id prefix in FlinkKafkaProducer

2021-06-23 Thread Stephan Ewen
The motivation and the proposal sound good to me, +1 from my side. Would be good to have a quick opinion from someone who worked specifically with Kafka, maybe Becket or Piotr? Best, Stephan On Sat, Jun 12, 2021 at 9:50 AM Wenhao Ji wrote: > Hi everyone, > > I would like to open this

[jira] [Created] (FLINK-23114) Address vulnerabilities in Flink UI

2021-06-23 Thread Jira
Ingo Bürk created FLINK-23114: - Summary: Address vulnerabilities in Flink UI Key: FLINK-23114 URL: https://issues.apache.org/jira/browse/FLINK-23114 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23113) Linting for Flink UI

2021-06-23 Thread Jira
Ingo Bürk created FLINK-23113: - Summary: Linting for Flink UI Key: FLINK-23113 URL: https://issues.apache.org/jira/browse/FLINK-23113 Project: Flink Issue Type: Sub-task Components:

[jira] [Created] (FLINK-23112) Formatting should be enforced for Flink UI

2021-06-23 Thread Jira
Ingo Bürk created FLINK-23112: - Summary: Formatting should be enforced for Flink UI Key: FLINK-23112 URL: https://issues.apache.org/jira/browse/FLINK-23112 Project: Flink Issue Type: Sub-task

Re: CONTENTS DELETED in nabble frontend

2021-06-23 Thread Yangze Guo
Ahh. It seems nabble has updated mailing lists to regular forums this week[1]. [1] http://support.nabble.com/Mailing-Lists-will-be-updated-to-regular-forums-next-week-td7609458.html Best, Yangze Guo On Wed, Jun 23, 2021 at 2:37 PM Yangze Guo wrote: > > It seems the post will remain iff it is

[jira] [Created] (FLINK-23111) Update Angular

2021-06-23 Thread Jira
Ingo Bürk created FLINK-23111: - Summary: Update Angular Key: FLINK-23111 URL: https://issues.apache.org/jira/browse/FLINK-23111 Project: Flink Issue Type: Sub-task Components: Runtime

[jira] [Created] (FLINK-23110) Modernize Flink UI

2021-06-23 Thread Jira
Ingo Bürk created FLINK-23110: - Summary: Modernize Flink UI Key: FLINK-23110 URL: https://issues.apache.org/jira/browse/FLINK-23110 Project: Flink Issue Type: Improvement Components:

Re: CONTENTS DELETED in nabble frontend

2021-06-23 Thread Yangze Guo
It seems the post will remain iff it is sent by a registered email. I do not register nabble in user ML and my post is deleted in [1]. [1]

[jira] [Created] (FLINK-23109) Translate "Scala API Extensions" pages into Chinese

2021-06-23 Thread Zhiliang Zhang (Jira)
Zhiliang Zhang created FLINK-23109: -- Summary: Translate "Scala API Extensions" pages into Chinese Key: FLINK-23109 URL: https://issues.apache.org/jira/browse/FLINK-23109 Project: Flink

CONTENTS DELETED in nabble frontend

2021-06-23 Thread Matthias Pohl
Hi everyone, Is it only me or does anyone else have the same problem with messages being not available anymore in the nabble frontend? I get multiple messages like the following one for individual messages: > CONTENTS DELETED > The author has deleted this message. This appears for instance in

Re: Re: [ANNOUNCE] New PMC member: Arvid Heise

2021-06-23 Thread Matthias Pohl
Congratulations, Arvid! :-) On Thu, Jun 17, 2021 at 9:02 AM Arvid Heise wrote: > Thank you for your trust and support. > > Arvid > > On Thu, Jun 17, 2021 at 8:39 AM Roman Khachatryan > wrote: > > > Congratulations! > > > > Regards, > > Roman > > > > On Thu, Jun 17, 2021 at 5:56 AM Xingbo Huang

[jira] [Created] (FLINK-23108) KafkaITCase.testCancelingEmptyTopic fails on azure

2021-06-23 Thread Xintong Song (Jira)
Xintong Song created FLINK-23108: Summary: KafkaITCase.testCancelingEmptyTopic fails on azure Key: FLINK-23108 URL: https://issues.apache.org/jira/browse/FLINK-23108 Project: Flink Issue

Re: [DISCUSS] Drop Mesos in 1.14

2021-06-23 Thread Matthias Pohl
+1 for dropping Mesos support. There was no feedback opposing the direction from the community in the most-recent discussion [1,2] on deprecating it. Matthias [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/SURVEY-Remove-Mesos-support-td45974.html [2]

Re: [ANNOUNCE] New PMC member: Xintong Song

2021-06-23 Thread Matthias Pohl
Congratulations, Xintong! On Mon, Jun 21, 2021 at 5:28 AM Zhilong Hong wrote: > Congratulations, Xintong! > > Dawid Wysakowicz 于2021年6月16日周三 下午5:23写道: > > > Hi all! > > > > I'm very happy to announce that Xintong Song has joined the Flink PMC! > > > > Congratulations and welcome Xintong! > > >