Re: [DISCUSS] Releasing Flink 1.12.5

2021-06-26 Thread Martijn Visser
Hi Jingsong, Could you still include https://issues.apache.org/jira/browse/FLINK-21229 ? Best, Martijn On Thu, 24 Jun 2021 at 11:53, Jingsong Li wrote: > Dear devs, > > As discussed in [1], I would volunteer as the release manager for 1.12.5 > and kick off the release process on next

Re: Scala 2.12 End to End Tests

2021-08-23 Thread Martijn Visser
+1 to switch to 2.12 by default On Sat, 21 Aug 2021 at 13:43, Chesnay Schepler wrote: > I'm with Till that we should switch to 2.12 by default . > > On 21/08/2021 11:12, Till Rohrmann wrote: > > Hi Danny, > > > > I think in the nightly builds we do run the e2e with Scala 2.12 [1]. The > > way

[VOTE] Drop Scala Shell

2021-09-20 Thread Martijn Visser
Hi all, I would like to start a vote on dropping the Scala Shell. This was previously discussed in the mailing list [1] The vote will be open for at least 72 hours unless there is an objection or not enough votes. Best regards, Martijn [1]

Re: [DISCUSS] Drop Scala Shell

2021-09-20 Thread Martijn Visser
scala shell. So we are still interested in scala shell > module, for us if no one want to maintain scala shell, it would be better > to have it as flink 3rd party library, maybe in flink-packages.org > > > Martijn Visser 于2021年9月16日周四 下午7:21写道: > > > Hi everyone, > > >

Re: About PR Review Help

2021-09-17 Thread Martijn Visser
Hi Roc, I'll get someone to review the PR. Thanks, Martijn On Fri, 17 Sept 2021 at 09:56, Roc Marshal wrote: > Hi, > Could someone help me to review this PR ? > https://github.com/apache/flink/pull/16962 > https://issues.apache.org/jira/browse/FLINK-15352 > > Thank you

[DISCUSS] Drop Scala Shell

2021-09-16 Thread Martijn Visser
-10911 Martijn Visser | Product Manager mart...@ververica.com <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time

Re: [DISCUSS] Address deprecation warnings when upgrading dependencies

2021-07-14 Thread Martijn Visser
With regards to JUnit 5, there was a specific proposal and vote on how to deal with that migration [1] Best regards, Martijn [1] https://lists.apache.org/thread.html/r89a2675bce01ccfdcfc47f2b0af6ef1afdbe4bad96d8c679cf68825e%40%3Cdev.flink.apache.org%3E On Wed, 14 Jul 2021 at 17:31, Chesnay

Re: [DISCUSS] Address deprecation warnings when upgrading dependencies

2021-07-14 Thread Martijn Visser
ote > or discussion thread. Please link the specific post if it was indeed > mentioned. > > On 14/07/2021 19:13, Martijn Visser wrote: > > With regards to JUnit 5, there was a specific proposal and vote on how to > deal with that migration [1] > > Best regards, > > Mar

Re: [DISCUSS] FLIP-180: Adjust StreamStatus and Idleness definition

2021-07-30 Thread Martijn Visser
Hi all, I have a couple of questions after studying the FLIP and the docs: 1. What happens when one of the readers has two splits assigned and one of the splits actually receives data? 2. If I understand it correctly the Kinesis Source uses dynamic shard discovery by default (so in case of

Re: [DISCUSS] Change Default Jira Priority from "Major" to "Minor"

2021-07-28 Thread Martijn Visser
n? > * Does it change other definitions? Do we need to migrate tickets? > * What would be the Jira Bot configuration? > > I know, this is a lot to ask, but otherwise I fear we might talk past each > other. > > Cheers, > > Konstantin > > > > On Tue, Jul 27, 2021 a

Re: Create a public open GitHub org for Flink ecosystem projects.

2021-09-23 Thread Martijn Visser
Hi all, Thanks Becket for the initiative. With the example projects mentioned, I agree that it makes a lot of sense to improve the ecosystem for projects that can be used to extend Flink. I would like to mention that Arvid and I have been working on a proposal with regards to the connectors and

Re: [VOTE] Drop Scala Shell

2021-09-23 Thread Martijn Visser
to track this. Best regards, Martijn On Tue, 21 Sept 2021 at 19:48, Robert Metzger wrote: > +1 > > On Mon, Sep 20, 2021 at 4:39 PM Seth Wiesman wrote: > >> +1 >> >> On Mon, Sep 20, 2021 at 6:04 AM Chesnay Schepler >> wrote: >> >> > +1

Re: [DISCUSS] Releasing Flink 1.13.3

2021-09-23 Thread Martijn Visser
K-23946 > > (depends > > > on https://issues.apache.org/jira/browse/FLINK-23946, which is not > > started > > > yet). This would leave FLINK-24315, which I can't judge at all. > > > > > > Cheers, > > > > > > Konstantin > > &g

Re: [DISCUSS] Releasing Flink 1.13.3

2021-10-08 Thread Martijn Visser
. I will start preparing for release first, and > > start voting after they back to office. > > > > Best, > > Leonard > > > > > > > > >> On 27/09/2021 21:26, Martijn Visser wrote: > > >>> Hi Timo, > > >>> &

Re: [ANNOUNCE] Apache Flink 1.13.3 released

2021-10-21 Thread Martijn Visser
Thank you Chesnay, Leonard and all contributors! On Thu, 21 Oct 2021 at 13:40, Jingsong Li wrote: > Thanks, Chesnay & Martijn > > 1.13.3 really solves many problems. > > Best, > Jingsong > > On Thu, Oct 21, 2021 at 6:46 PM Konstantin Knauf > wrote: > > > > Thank you, Chesnay & Martijn, for

Re: [DISCUSS] Creating an external connector repository

2021-10-19 Thread Martijn Visser
Hi all, I think it would be a huge benefit if we can achieve more frequent releases of connectors, which are not bound to the release cycle of Flink itself. I agree that in order to get there, we need to have stable interfaces which are trustworthy and reliable, so they can be safely used by

Re: [RESULT][VOTE] Release 1.13.3, release candidate #1

2021-10-19 Thread Martijn Visser
Thank you Chesnay and all the others who have contributed! Best regards, Martijn On Tue, 19 Oct 2021 at 15:03, Chesnay Schepler wrote: > We have unanimously approved this release: > > Binding votes: > - Dian > - Arvid > - Chesnay > > On 12/10/2021 19:21, Chesnay Schepler wrote: > > Hi

Re: Flink Cassandra sink connector doesn't support Apache Cassandra 4.0

2021-10-11 Thread Martijn Visser
Hi, There's already a Jira ticket [1] for this but no one is yet working on it. We'd welcome it if you would like to pick it up, I can assign it to you. Best regards, Martijn [1] https://issues.apache.org/jira/browse/FLINK-19930 On Fri, 8 Oct 2021 at 16:12, Purushotham Pushpavanthar <

Re: The Apache Flink should pay more attention to ensuring API compatibility.

2021-09-28 Thread Martijn Visser
investigate a bit what happened on our end, share that and then try to get some learnings out of it for the future. I'll get back to you in a couple of days. Best regards, Martijn Visser | Product Manager mart...@ververica.com <https://www.ververica.com/> Follow us @VervericaData -- Join

[DISCUSS] Releasing Flink 1.13.3

2021-09-22 Thread Martijn Visser
in 900 seconds) Can Yangze, David, Yun and Till give an update on the status for those? Are there any other open tickets that we should wait for? Is there a PMC member who would like to manage the release? Best regards, Martijn Visser | Product Manager mart...@ververica.com <ht

Re: [DISCUSS] Releasing Flink 1.13.3

2021-09-27 Thread Martijn Visser
t;>> Yangze Guo > >>>> > >>>> > >>>> On Wed, Sep 22, 2021 at 8:44 PM Konstantin Knauf > >>>> wrote: > >>>>> > >>>>> Hi Martijn, > >>>>> > >>>>> Thanks f

Re: [VOTE] Deprecate Java 8 support

2021-12-06 Thread Martijn Visser
ld entail a notification to our users to encourage > >> migrating to Java 11, and various efforts on our side to prepare a > >> migration to Java 11, like updating some e2e tests to actually run on > >> Java 11, performance benchmarking etc. . > >> > >>

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-20 Thread Martijn Visser
Hi all, Really like the discussion on this topic moving forward. I really think this feature will be much appreciated by the Flink users. What I still have left to answer/reply to: -- Good point. If for whatever reason the different taskmanagers can't get the latest rule, the Operator

Re: About Clickhouse connector

2021-12-27 Thread Martijn Visser
Hi, I don't have permission to give you edit rights on Confluence, but I am wondering if it's necessary to create a FLIP for a new connector. I don't think a new connector would result in any new changes in Flink, since you would use the already available APIs? There's also already a Clickhouse

Re: [DISCUSS] Releasing Flink 1.14.3

2021-12-27 Thread Martijn Visser
71 has been merged to 1.14 branch. > >> > >> Best, > >> D. > >> > >> On Wed, Dec 22, 2021 at 7:27 AM 任庆盛 wrote: > >> > >>> Hi Martjin, > >>> > >>> Thanks for the effort on Flink 1.14.3. FLINK-25132 has been m

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

2021-12-28 Thread Martijn Visser
Hi Jingsong, That sounds promising! +1 from my side to continue development under flink-dynamic-storage as a Flink subproject. I think having a more in-depth interface will benefit everyone. Best regards, Martijn On Tue, 28 Dec 2021 at 04:23, Jingsong Li wrote: > Hi all, > > After some

Re: [DISCUSS] Drop Gelly

2022-01-04 Thread Martijn Visser
/arxiv.org/abs/1403.6652 > [3] https://arxiv.org/abs/1503.03578 > [4] https://github.com/apache/spark > > Best, > Zhipeng > > Martijn Visser 于2022年1月4日周二 15:27写道: > >> Hi Zhipeng, >> >> Good that you've reached out, I wasn't aware that Gelly is bei

[DISCUSS] Looking for maintainers for Google PubSub connector or discuss next step

2022-01-03 Thread Martijn Visser
tor won't work in Flink (since it doesn't use the target interfaces) and the source code can still be found in Flink's git repo by looking back to previous versions. I'm looking forward to your thoughts. Best regards, Martijn Visser [1] https://nightlies.apache.org/flink/flink-docs-stable/docs/c

[DISCUSS] Drop Gelly

2022-01-03 Thread Martijn Visser
at the moment. Let me know your thoughts. Martijn Visser | Product Manager mart...@ververica.com [1] https://nightlies.apache.org/flink/flink-docs-stable/docs/libs/gelly/overview/ [2] https://flink.apache.org/roadmap.html [3] https://lists.apache.org/thread/b2y3xx3thbcbtzdphoct5wvzwogs9sqz <ht

Re: [DISCUSS] Deprecate MapR FS

2022-01-05 Thread Martijn Visser
ver the years. They should be able to use the 1.14 >> MapR >> > FS with later versions of Flink. >> > >> > Seth >> > >> > On Wed, Dec 8, 2021 at 10:03 AM Martijn Visser >> > wrote: >> > >> >> Hi all, >> >> &g

Re: [DISCUSS] Creating an external connector repository

2022-01-05 Thread Martijn Visser
repo per connector > approach. > > > > Cheers, > > Till > > > > On Thu, Dec 9, 2021 at 5:41 PM Till Rohrmann > wrote: > > > > > +1 for the single repo approach. > > > > > > Cheers, > > > Till > > > > > >

Re: [DISCUSS] Looking for maintainers for Google PubSub connector or discuss next step

2022-01-05 Thread Martijn Visser
> > > > If we decide to keep the connector, I can start taking a look at the next > > step: going through the existing PR, fixing conflicts with master. > > > > All my best, Ryan > > > > On Mon, Jan 3, 2022 at 3:41 PM Martijn Visser > > wr

[DISCUSS] Moving connectors from Flink to external connector repositories

2022-01-05 Thread Martijn Visser
ignoring data. * Some of the connectors haven't been maintained in a long time (for example, NiFi and Google Cloud PubSub). An argument could be made that we check if we actually want to move such a connector or make the decision to drop the connector entirely. I'm looking forward to your thoughts! Bes

Re: [DISCUSS] Disabling JNDI by default

2022-01-05 Thread Martijn Visser
Hi Till, I think it would be great if we could achieve this so that Flink would be 'hardened' by default. Hopefully someone in the community has some ideas how. Best regards, Martijn On Tue, 4 Jan 2022 at 13:19, Till Rohrmann wrote: > Hi everyone, > > With the latest CVEs around log4j, we

[DISCUSS] Releasing Flink 1.14.3

2021-12-21 Thread Martijn Visser
e backport for >> > https://issues.apache.org/jira/browse/FLINK-25132. The changes are >> > already approved I only wait for a green Azure build. >> > >> > Best, >> > Fabian >> > >> > On Thu, Dec 9, 2021 at 4:01 PM Martijn V

Re: [DISCUSS] Introduce Hash Lookup Join

2021-12-28 Thread Martijn Visser
Hi Jing, Thanks a lot for the explanation and the FLIP. I definitely learned something when reading more about `use_hash`. My interpretation would be that the primary benefit of a hash lookup join would be improved performance by allowing the user to explicitly optimise the planner. I have a

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-30 Thread Martijn Visser
better fit for this case. This would simplify > the > > > > > consistency concerns (watermarks + pushback) and the re-processing > of > > > > > historical data. > > > > > > > > > > Best, > > > > > D. > > > > > > &

Re: Re: [DISCUSS] Introduce Hash Lookup Join

2021-12-29 Thread Martijn Visser
; > candidates? > >> > >>> Things are a little different from RDBMS in the distributed world, > >> and > >> > we > >> > >>> also aim to solve the data skew problem, so all these incoming > hints > >> > names > >&g

Re: [DISCUSS] FLIP-208: Update KafkaSource to detect EOF based on de-serialized record

2022-01-03 Thread Martijn Visser
Hi Dong, Thanks for writing the FLIP. It focusses only on the KafkaSource, but I would expect that if such a functionality is desired, it should be made available for all unbounded sources (for example, Pulsar and Kinesis). If it's only available for Kafka, I see it as if we're increasing feature

Re: [DISCUSS] Drop Gelly

2022-01-03 Thread Martijn Visser
ence, I >>> would assume that it is not really used and can be removed. >>> >>> +1 for dropping Gelly. >>> >>> Cheers, >>> Till >>> >>> On Mon, Jan 3, 2022 at 2:20 PM Martijn Visser >>> wrote: >>> >>>&g

Re: [VOTE] FLIP-191: Extend unified Sink interface to support small file compaction

2022-01-04 Thread Martijn Visser
+1 (non-binding). Thanks for driving the FLIP! Best regards, Martijn On Tue, 4 Jan 2022 at 10:21, Fabian Paul wrote: > Hi everyone, > > I'd like to start a vote on FLIP-191: Extend unified Sink interface to > support small file compaction [1] that has been discussed in this > thread [2]. > >

Re: [DISCUSS] Shall casting functions return null or throw exceptions for invalid input

2021-11-23 Thread Martijn Visser
t; agree > > > > > > with > > > > > > >> Ingo that > > > > > > >> we should not just break some existing behavior, and even if > we > > > > > > introduce > > > > > > >> an > &g

Re: [VOTE][FLIP-195] Improve the name and structure of vertex and operator name for job

2021-11-23 Thread Martijn Visser
+1 (non-binding) On Tue, 23 Nov 2021 at 12:13, Aitozi wrote: > +1 (non-binding) > > Best, > Aitozi > > wenlong.lwl 于2021年11月23日周二 下午4:00写道: > > > Hi everyone, > > > > Based on the discussion[1], we seem to have consensus, so I would like to > > start a vote on FLIP-195 [2]. > > Thanks for all

Re: [ANNOUNCE] New Apache Flink Committer - Yangze Guo

2021-11-12 Thread Martijn Visser
Congratulations! Best regards, Martijn On Fri, 12 Nov 2021 at 07:49, Jark Wu wrote: > Congratulations Yangze! > > Best, > Jark > > On Fri, 12 Nov 2021 at 14:06, JING ZHANG wrote: > > > Congrats! > > > > Best, > > Jing Zhang > > > > Lijie Wang 于2021年11月12日周五 下午1:25写道: > > > > > Congrats

Re: [ANNOUNCE] New Apache Flink Committer - Leonard Xu

2021-11-12 Thread Martijn Visser
Well deserved, congratulations Leonard! Best regards, Martijn On Fri, 12 Nov 2021 at 08:44, Caizhi Weng wrote: > Congrats, Leonard! > > Shuo Cheng 于2021年11月12日周五 下午3:12写道: > > > Congrats, Leonard! > > > > On Fri, Nov 12, 2021 at 2:35 PM godfrey he wrote: > > > > > Congrats & well deserved,

Re: [DISCUSS] Update Policy for old releases

2021-11-12 Thread Martijn Visser
Thanks for bringing this up for discussion Piotr. One the one hand I think it's a good idea, because of the reasons you've mentioned. On the other hand, having an LTS version will remove an incentive for some users to upgrade, which will result in fewer Flink users who will test new features

Re: [DISCUSS] Definition of Done for Apache Flink

2021-11-16 Thread Martijn Visser
Hi all, Thanks for bringing this up for this discussion, because I think it's an important aspect. >From my perspective, a 'definition of done' serves two purposes: 1. It informs the contributor on what's expected when making a contribution in the form of a PR 2. It instructs the committer on

[DISCUSS] Releasing Flink 1.14.1

2021-11-24 Thread Martijn Visser
1.14.1%20AND%20labels%20%3D%20test-stability%20AND%20assignee%20in%20(EMPTY)%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC Martijn Visser | Product Manager mart...@ververica.com <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org

Re: [DISCUSS] Releasing Flink 1.14.1

2021-11-29 Thread Martijn Visser
FLINK-25022: I will open a PR later today, and it should be easy to > backport. > FLINK-25027: Unlikely to make it for 1.14.1; I also wouldn't consider it > a blocker > > On 24/11/2021 19:40, Martijn Visser wrote: > > Hi all, > > > > I would like to start a

Re: [DISCUSS] Definition of Done for Apache Flink

2021-11-25 Thread Martijn Visser
t; > > > it converted into checkboxes. I know it's a small detail, but it's > > much > > > > > less annoying than the current template. Something like > > > > > > > > > > ``` > > > > > - [ ] This pull requests changes the public A

Re: [DISCUSS] PR component labeling

2021-11-25 Thread Martijn Visser
Hi Nico, I'm using the labels to filter on PRs that I find interesting (for example, I look at all the new or updated PRs that are related to SQL or connectors). Best regards, Martijn On Thu, 25 Nov 2021 at 16:17, Nicolaus Weidner < nicolaus.weid...@ververica.com> wrote: > Hi all, > > since

Re: [DISCUSS] PR component labeling

2021-11-25 Thread Martijn Visser
gt; On 25/11/2021 16:28, Martijn Visser wrote: > > Hi Nico, > > > > I'm using the labels to filter on PRs that I find interesting (for > example, > > I look at all the new or updated PRs that are related to SQL or > > connectors). > > > > Best regards, >

Re: [DISCUSS] Deprecate Java 8 support

2021-11-22 Thread Martijn Visser
Hi Chesnay, Thanks for bringing this up for discussion. Big +1 for dropping Java 8 and deprecating it in 1.15, given that Java 8 support will end. We already see other dependencies that Flink use either have dropped support for Java 8 (Trino) or are going to drop it (Kafka). Best regards,

Re: [ANNOUNCE] New Apache Flink Committer - Yangze Guo

2021-11-15 Thread Martijn Visser
Congrats Yangze! On Mon, 15 Nov 2021 at 09:13, Yangze Guo wrote: > Thank you all very much! > It's my honor to work with you in such a great community. > > Best, > Yangze Guo > > On Mon, Nov 15, 2021 at 2:08 PM Yang Wang wrote: > > > > Congratulations Yangze! > > > > Best, > > Yang > > > > Yu

Re: [ANNOUNCE] New Apache Flink Committer - Jing Zhang

2021-11-15 Thread Martijn Visser
Congratulations Jing! On Mon, 15 Nov 2021 at 14:39, Timo Walther wrote: > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Jing Zhang as a new > Flink committer. > > Jing has been very active in the Flink community esp. in the Table/SQL > area for quite some time: 81 PRs [1] in

Re: [ANNOUNCE] New Apache Flink Committer - Fabian Paul

2021-11-15 Thread Martijn Visser
Congratulations Fabian! On Mon, 15 Nov 2021 at 14:27, Ingo Bürk wrote: > Congratulations, Fabian! > > On Mon, Nov 15, 2021 at 2:17 PM Arvid Heise wrote: > > > Hi everyone, > > > > On behalf of the PMC, I'm very happy to announce Fabian Paul as a new > Flink > > committer. > > > > Fabian Paul

Re: [DISCUSS] Improve the name and structure of job vertex and operator name for job

2021-11-11 Thread Martijn Visser
+1. Looks much better now On Thu, 11 Nov 2021 at 11:07, godfrey he wrote: > Thanks for driving this, this improvement solves a long-complained > problem, +1 > > Best, > Godfrey > > Jark Wu 于2021年11月11日周四 下午5:40写道: > > > > +1 for this. It looks much more clear and structured. > > > > Best, > >

Re: [DISCUSS] Shall casting functions return null or throw exceptions for invalid input

2021-11-18 Thread Martijn Visser
Hi Caizhi, Thanks for bringing this up for discussion. I think the important part is what do developers expect as the default behaviour of a CAST function when casting fails. If I look at Postgres [1] or MSSQL [2], the default behaviour of a CAST failing would be to return an error, which would

Re: [ANNOUNCE] New Apache Flink Committer - Yingjie Cao

2021-11-18 Thread Martijn Visser
Congratulations! On Thu, 18 Nov 2021 at 02:44, Leonard Xu wrote: > Congratulations!Yingjie > > Best, > Leonard > > > 在 2021年11月18日,01:40,Till Rohrmann 写道: > > > > Congratulations Yingjie! > >

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-01 Thread Martijn Visser
> > > > Hi Martijn, > > > > > > > > Thank you for your suggestion with POC. > > > > Yes I will do that and come back to this thread probably after the > > > weekend > > > > > > > > On Thu, Oct 28, 2021 at 4:38 PM Marti

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-10-28 Thread Martijn Visser
Hi Sergey, Thanks for taking the initiative to create a FLIP and propose improvements on the SQL client. All usability improvements on the SQL client are highly appreciated, especially for new users of Flink. Multi-line support is definitely one of those things I've run into myself. I do think

Re: [NOTICE] flink-streaming-java no longer depends on Scala and lost it's suffix

2021-10-26 Thread Martijn Visser
This is an awesome accomplishment, many thanks! Ideally we also manage to complete https://issues.apache.org/jira/browse/FLINK-24427 for 1.15 to get Flink in a much better shape for our Scala users. On Tue, 26 Oct 2021 at 12:34, Arvid Heise wrote: > Awesome. Thank you very much for all the hard

Re: [VOTE] FLIP-189: SQL Client Usability Improvements

2021-11-05 Thread Martijn Visser
+1 (non-binding). Looking forward! Best regards, Martijn On Fri, 5 Nov 2021 at 15:36, Timo Walther wrote: > +1 (binding) thanks for working on this. > > Regards, > Timo > > > On 05.11.21 10:14, Sergey Nuyanzin wrote: > > Also there is a short demo showing some of the features mentioned in

Re: [ANNOUNCE] New Apache Flink Committer - Ingo Bürk

2021-12-02 Thread Martijn Visser
k as a > new > >> > >> Flink > >> > >>>> committer. > >> > >>>> > >> > >>>> Ingo has started contributing to Flink since the beginning of > this > >> > >> year. > >> > >>&g

Re: [ANNOUNCE] New Apache Flink Committer - Matthias Pohl

2021-12-02 Thread Martijn Visser
t;>> Matthias has worked on Flink since August last year. He helped > review > >> >> a ton > >> >>>> of PRs. He worked on a variety of things but most notably the > tracking > >> >> and > >> >>>> reporting of concurrent excep

Re: [DISCUSS] Creating an external connector repository

2021-12-09 Thread Martijn Visser
k-core. > > >>>>>>> 3.Phase, remove old interfaces in flink-core of some connectors > > >>>>>>> (tbd > > >>>>> at a > > >>>>>>> later point). > > >>>>>>> 4.Phase, option

Re: [DISCUSS] Releasing Flink 1.14.1

2021-12-09 Thread Martijn Visser
gt; We could finish it as soon as possible. >> >> >> >> Best, >> >> Jingsong >> >> >> >> On Fri, Dec 3, 2021 at 10:25 PM Fabian Paul wrote: >> >> > >> >> > I just opened a PR for >> >> > https://issues.apache.or

Re: [DISCUSS] Strong read-after-write consistency of Flink FileSystems

2021-12-08 Thread Martijn Visser
Hi David, Just to be sure, since you've already included Azure Blob Storage, but did you deliberately skip Azure Data Lake Store Gen2? That's currently supported and also used by Flink users [1]. There's also MapR FS, but I doubt if that is still used. Best regards, [1]

[DISCUSS] Deprecate MapR FS

2021-12-08 Thread Martijn Visser
Hi all, Flink supports multiple file systems [1] which includes MapR FS. MapR as a company doesn't exist anymore since 2019, the technology and intellectual property has been sold to Hewlett Packard. I don't think that there's anyone who's using MapR anymore and therefore I think it would be

Re: [DISCUSS] Immediate dedicated Flink releases for log4j vulnerability

2021-12-12 Thread Martijn Visser
+1 to address the issue like this On Mon, 13 Dec 2021 at 07:46, Jingsong Li wrote: > +1 for fixing it in these versions and doing quick releases. Looks good to > me. > > Best, > Jingsong > > On Mon, Dec 13, 2021 at 2:18 PM Becket Qin wrote: > > > > +1. The solution sounds good to me. There

Re: [DISCUSS] GHA migration roadmap

2021-12-16 Thread Martijn Visser
Hi Nico, Thanks a lot for the proposal. I applaud the move to GHA and the planning seems great. +1 from my side. I'm assuming with the converting one CI machine, you mean that we switch convert one of the CI machines to run the Github Selfhosted Runner? Best regards, Martijn On Thu, 16 Dec

Re: [DISCUSS] Releasing Flink 1.14.1

2021-12-03 Thread Martijn Visser
gards, Martijn On Thu, 2 Dec 2021 at 14:25, Marios Trivyzas wrote: > https://issues.apache.org/jira/browse/FLINK-22113 will be merged today > (most probably) > > On Mon, Nov 29, 2021 at 10:15 AM Martijn Visser > wrote: > >> Thanks all for the updates! To summarize, these

Re: [DISCUSS] Changing the minimal supported version of Hadoop

2021-12-14 Thread Martijn Visser
Hi David, Thanks for bringing this up for discussion! Given that Hadoop 2.8 is considered EOL, shouldn't we bump the version to Hadoop 2.10? [1] Best regards, Martijn [1] https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Active+Release+Lines On Tue, 14 Dec 2021 at 10:28, Till Rohrmann

Re: [VOTE] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-14 Thread Martijn Visser
The vote will be open for at least 72 hours unless there is an objection > or not enough votes. > > [1] https://cwiki.apache.org/confluence/x/KZBnCw > [2] https://lists.apache.org/thread/n8v32j6o3d50mpblxydbz82q1q436ob4 > > Cheers, > Timo > -- Martijn Visser | Product Manager mar

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-14 Thread Martijn Visser
Hi all, > IMO, in this FLIP, we only need to introduce the general design of the Table API/SQL level. As for the design details, you can create a new FLIP. Do you think that the current section on Table/SQL API support is sufficient as a general design? > And do we need to take into account the

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-10 Thread Martijn Visser
> on a functional level and not between APIs. > > > > > > Best > > > > Ingo > > > > On 10.12.21 13:24, Martijn Visser wrote: > >> Apologies, I do see SQL mentioned at the bottom for new rules, but I > >> don't > >> think it

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-10 Thread Martijn Visser
Apologies, I do see SQL mentioned at the bottom for new rules, but I don't think it's a good idea to have these changes only for the DataStream API in the beginning. This would increase sparsity in Flink, which we should avoid. Best regards, Martijn On Fri, 10 Dec 2021 at 13:19, Martijn Visser

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-10 Thread Martijn Visser
Hi Yunfeng, Thanks for creating the FLIP. I don't see any mention of SQL's MATCH_RECOGNIZE implementation in the FLIP and I think that any change in CEP should be available to both DataStream and SQL/Table API users. Can you elaborate on that? Best regards, Martijn On Fri, 10 Dec 2021 at

Re: [ANNOUNCE] Open source of remote shuffle project for Flink batch data processing

2021-11-30 Thread Martijn Visser
Hi Yingjie, This is great, thanks for sharing. Will you also add it to https://flink-packages.org/ ? Best regards, Martijn On Tue, 30 Nov 2021 at 17:31, Till Rohrmann wrote: > Great news, Yingjie. Thanks a lot for sharing this information with the > community and kudos to all the

Re: [NOTICE] flink-runtime now scala-free

2021-07-14 Thread Martijn Visser
This is a great achievement, thank you for driving this! On Tue, 13 Jul 2021 at 18:20, Chesnay Schepler wrote: > Hello everyone, > > I just merged the last PR for FLINK-14105, with which flink-runtime is > now officially scala-free. *fireworks* > > > What does that mean in practice? > >

Re: [DISCUSS] Change Default Jira Priority from "Major" to "Minor"

2021-07-27 Thread Martijn Visser
Hi, I personally would prefer to use "Normal" as a default priority because I think a lot of people's first reaction is that their reported problem is bigger than a minor loss of function [1], resulting in them choosing the next priority which is currently "Major". Best regards, Martijn [1]

Re: [DISCUSS] Change Default Jira Priority from "Major" to "Minor"

2021-07-27 Thread Martijn Visser
quot;Normal" or would you like to introduce a new priority between Major and > Minor? > > On Tue, Jul 27, 2021 at 11:44 AM Jingsong Li > wrote: > > > I agree with Martijn. > > > > My problem is just minor, which will make me a little disappointed. > > > &

Re: Still receiving these emails.. please unsubscribe me

2021-07-26 Thread Martijn Visser
Hi, I would recommend having a look at the headers of the original email [1]. In those headers, there will be a value for 'Return-Path' with an email address. If you send an email to that address, it will unsubscribe you. Best regards, Martijn [1]

Re: [DISCUSS] Deprecate MapR FS

2022-01-12 Thread Martijn Visser
> +1 for dropping the MapR FS. > > > > Till Rohrmann 于2022年1月5日周三 18:33写道: > > > > > +1 for dropping the MapR FS. > > > > > > Cheers, > > > Till > > > > > > On Wed, Jan 5, 2022 at 10:11 AM Martijn Visser > > >

Re: OutOfMemoryError: Java heap space while implmentating flink sql api

2022-01-12 Thread Martijn Visser
Hi Ronak, I would like to ask you to stop cross-posting to all the Flink mailing lists and then also post the same question to Stackoverflow. Both the mailing lists and Stackoverflow are designed for asynchronous communication and you should allow the community some days to address your question.

[FEEDBACK] Metadata Platforms / Catalogs / Lineage integration

2022-01-13 Thread Martijn Visser
Hi everyone, I'm currently checking out different metadata platforms, such as Amundsen [1] and Datahub [2]. In short, these types of tools try to address problems related to topics such as data discovery, data lineage and an overall data catalogue. I'm reaching out to the Dev and User mailing

Re: [FEEDBACK] Metadata Platforms / Catalogs / Lineage integration

2022-01-13 Thread Martijn Visser
Hi all, @Andrew thanks for sharing that! @Tero good point, I should have clarified the purpose. I want to understand what "metadata platforms" tools are used or evaluated by the Flink community, what's their purpose for using such a tool (is it as a generic catalogue, as a data discovery tool,

Re: [DISCUSS] Releasing Flink 1.14.3

2022-01-13 Thread Martijn Visser
Hi all, We still need a couple of PMCs to validate the release and put out their vote. The vote thread can be found at https://lists.apache.org/thread/cqn8p7kv4tbc4hn4czjvzfcd905jztro Best regards, Martijn On Tue, 11 Jan 2022 at 16:29, Martijn Visser wrote: > Hi Thomas, > > Tha

Re: [DISCUSS] Creating an external connector repository

2022-01-13 Thread Martijn Visser
Hi everyone, If you have any more comments or questions, do let me know. Else I'll open up a vote thread next week. Best regards, Martijn On Tue, 11 Jan 2022 at 20:13, Martijn Visser wrote: > Good question: we want to use the same setup as we currently have for > Flink, so using the ex

Re: [DISCUSS] Moving connectors from Flink to external connector repositories

2022-01-13 Thread Martijn Visser
nector’s E2E cases. > > Best regards, > > Qingsheng Ren > > > > On Jan 5, 2022, at 9:59 PM, Martijn Visser > wrote: > > > > Hi everyone, > > > > As already mentioned in the previous discussion thread [1] I'm opening > up a > > parallel discussi

[DISCUSS] Move Flink website to privacy friendly Analytics solution

2022-01-14 Thread Martijn Visser
Hi everyone, The Flink website currently uses Google Analytics to track how visitors of the website are interacting with it. It provides insights into which documentation pages are visited, how users are using the website (what's the cycle of pages they visit before exiting the page), if they are

[VOTE] Release 1.14.3, release candidate #1

2022-01-11 Thread Martijn Visser
* website pull request listing the new release and adding announcement blog post [6]. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks on behalf of Thomas Weise and myself, Martijn Visser http://twitter.com/MartijnVi

Re: [DISCUSS] FLIP-210: Change logging level dynamically at runtime

2022-01-11 Thread Martijn Visser
Hi all, I agree with Konstantin, this feels like a problem that shouldn't be solved via Apache Flink but via the logging ecosystem itself. Best regards, Martijn On Tue, 11 Jan 2022 at 13:11, Konstantin Knauf wrote: > I've now read over the discussion on the ticket, and I am personally not in

Re: [DISCUSS] Creating an external connector repository

2022-01-11 Thread Martijn Visser
; > On 05/01/2022 11:48, Martijn Visser wrote: > > Hi everyone, > > > > I wanted to summarise the email thread and see if there are any open > items > > that still need to be discussed, before we can finalise the discussion in > > this email thread: > > >

Re: [DISCUSS] FLIP-208: Update KafkaSource to detect EOF based on de-serialized record

2022-01-11 Thread Martijn Visser
> that we > > > > > should provide API/infrastructure in Flink (as this FLIP does) to > > > support > > > > > feature consistency across connectors, I am not sure we should own > the > > > > > responsibility to actually update all connectors to achieve featu

Re: [DISCUSS] Releasing Flink 1.14.3

2022-01-11 Thread Martijn Visser
e wheels locally? > > > Alternatively, if someone can build them on their existing setup and > > > point me to the result, that would speed up things as well. > > > > > > The release branch: > > > https://github.com/apache/flink/tree/release-

Re: [VOTE] Release 1.14.3, release candidate #1

2022-01-17 Thread Martijn Visser
;>> On Wed, Jan 12, 2022 at 11:00 PM Till Rohrmann > >>>> wrote: > >>>>> > >>>>> +1 (binding) > >>>>> > >>>>> - Verified checksums and signatures > >>>>> - Build from source and

[RESULT][VOTE] [VOTE] Deprecate NiFi connector

2022-02-10 Thread Martijn Visser
Hi everyone, The Flink community has voted and accepted to deprecate the NiFi connector in Flink 1.15 and remove it in the next version. It has been unanimously approved [1] Binding +1: Fabian Paul Konstantin Knauf Seth Wiesman Chesnay Schepler Best regards, Martijn Visser https

Re: [VOTE] Deprecate Cassandra connector

2022-02-10 Thread Martijn Visser
Hi everyone, I've cancelled the vote since Marco Zühlke volunteered to help with the Cassandra contributions [1]. Best regards, Martijn [1] https://lists.apache.org/thread/1qokt5tp8dcp58dmshbwjc43ssbm1vvk On Mon, 31 Jan 2022 at 12:40, Ada Wong wrote: > +1 > > Martijn Visser 于202

Re: [VOTE] Deprecate NiFi connector

2022-02-10 Thread Martijn Visser
Hi everyone, Thanks for your votes, I'm closing the vote and I'll post the results shortly. Best regards, Martijn On Fri, 4 Feb 2022 at 11:18, Chesnay Schepler wrote: > +1 > > On 31/01/2022 11:46, Martijn Visser wrote: > > Hi everyone, > > > > I would like to open

  1   2   3   4   5   6   7   8   9   10   >