[VOTE] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-04 Thread Thomas Weise
Hi everyone, I'd like to start a vote on FLIP-212: Introduce Flink Kubernetes Operator [1] which has been discussed in [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-04 Thread Thomas Weise
Hi, Thanks for the continued feedback and discussion. Looks like we are ready to start a VOTE, I will initiate it shortly. In parallel it would be good to find the repository name. My suggestion would be: flink-kubernetes-operator I thought "flink-operator" could be a bit misleading since the

[jira] [Created] (FLINK-25964) Support shipping extra local files/jars to job/task manager classpaths

2022-02-04 Thread Chenya Zhang (Jira)
Chenya Zhang created FLINK-25964: Summary: Support shipping extra local files/jars to job/task manager classpaths Key: FLINK-25964 URL: https://issues.apache.org/jira/browse/FLINK-25964 Project:

[jira] [Created] (FLINK-25963) FLIP-212: Introduce Flink Kubernetes Operator

2022-02-04 Thread Thomas Weise (Jira)
Thomas Weise created FLINK-25963: Summary: FLIP-212: Introduce Flink Kubernetes Operator Key: FLINK-25963 URL: https://issues.apache.org/jira/browse/FLINK-25963 Project: Flink Issue Type:

[jira] [Created] (FLINK-25962) Flink generated Avro schemas can't be parsed using Python

2022-02-04 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-25962: --- Summary: Flink generated Avro schemas can't be parsed using Python Key: FLINK-25962 URL: https://issues.apache.org/jira/browse/FLINK-25962 Project: Flink

[jira] [Created] (FLINK-25961) Remove transport client from Elasticsearch 6/7 connectors (tests only)

2022-02-04 Thread Andriy Redko (Jira)
Andriy Redko created FLINK-25961: Summary: Remove transport client from Elasticsearch 6/7 connectors (tests only) Key: FLINK-25961 URL: https://issues.apache.org/jira/browse/FLINK-25961 Project:

Re: [DISCUSS] Releasing Flink 1.13.6

2022-02-04 Thread Konstantin Knauf
Hi everyone, I've created a release candidate, published all Maven artifacts and opened a PR for the website [1,2,3,4,5]. Python wheels are missing at this point as I am (like Thomas was) waiting for approval by Azure. I'll start the formal vote when the Python wheels are available. Of course,

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-04 Thread Gyula Fóra
Hi Danny, So far we have been focusing our dev efforts on the initial native implementation with the team. If the discussion and vote goes well for this FLIP we are looking forward to contributing the initial version sometime next week (fingers crossed). At that point I think we can already

Re: [DISCUSS] FLIP-212: Introduce Flink Kubernetes Operator

2022-02-04 Thread Danny Cranmer
I have been discussing this one with my team. We are interested in the Standalone mode, and are willing to contribute towards the implementation. Potentially we can work together to support both modes in parallel? Thanks, On Wed, Feb 2, 2022 at 4:02 PM Gyula Fóra wrote: > Hi Danny! > > Thanks

Re: [DISCUSS] Move Flink website to privacy friendly Analytics solution

2022-02-04 Thread Piotr Nowojski
Hi Martijn, Indeed good to hear that. Thanks for taking care of this. Best, Piotrek czw., 3 lut 2022 o 15:45 Till Rohrmann napisał(a): > Great news. Thanks for driving this effort Martijn and helping with it > Chesnay and Konstantin :-) > > Cheers, > Till > > On Thu, Feb 3, 2022 at 3:13 PM

[jira] [Created] (FLINK-25960) Distribute the data read buffers more fairly among result partitions for sort-shuffle

2022-02-04 Thread Yingjie Cao (Jira)
Yingjie Cao created FLINK-25960: --- Summary: Distribute the data read buffers more fairly among result partitions for sort-shuffle Key: FLINK-25960 URL: https://issues.apache.org/jira/browse/FLINK-25960

Re: [DISCUSS] FLIP-211: Kerberos delegation token framework

2022-02-04 Thread David Morávek
There is a really strong tendency to push many things out of Flink lately and keep the main building blocks only. However I really think that this belongs between the minimal building blocks that Flink should provide out of the box. It's also very likely that security topics will start getting

[jira] [Created] (FLINK-25959) Add a micro-benchmark for the sort-based blocking shuffle

2022-02-04 Thread Yingjie Cao (Jira)
Yingjie Cao created FLINK-25959: --- Summary: Add a micro-benchmark for the sort-based blocking shuffle Key: FLINK-25959 URL: https://issues.apache.org/jira/browse/FLINK-25959 Project: Flink

Re: [DISCUSS] FLIP-211: Kerberos delegation token framework

2022-02-04 Thread Gyula Fóra
Hi Chesnay, Thanks for the proposal for the alternative mechanism. I see the conceptual value of separating this process from Flink but in practice I feel there are a few very serious limitations with that. Just a few points that come to mind: 1. Implementing this as independent distributed

[jira] [Created] (FLINK-25958) OOME Checkpoints & Savepoints were shown as COMPLETE in Flink UI

2022-02-04 Thread Victor Xu (Jira)
Victor Xu created FLINK-25958: - Summary: OOME Checkpoints & Savepoints were shown as COMPLETE in Flink UI Key: FLINK-25958 URL: https://issues.apache.org/jira/browse/FLINK-25958 Project: Flink

Re: [VOTE] Deprecate NiFi connector

2022-02-04 Thread Chesnay Schepler
+1 On 31/01/2022 11:46, Martijn Visser wrote: Hi everyone, I would like to open up a vote to deprecate NiFi in Flink 1.15 and remove it in the next version. I've previously mentioned that we were looking for maintainers for NiFi, but so far none have come forward [1]. The vote will last for

[jira] [Created] (FLINK-25957) Cache cassandra/pulsar images

2022-02-04 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25957: Summary: Cache cassandra/pulsar images Key: FLINK-25957 URL: https://issues.apache.org/jira/browse/FLINK-25957 Project: Flink Issue Type: Technical

Re: [DISCUSS] FLIP-211: Kerberos delegation token framework

2022-02-04 Thread Chesnay Schepler
The concrete proposal would be to add a generic process startup lifecycle hook (essentially a Consumer), that is run at the start of each processs (JobManager, TaskManager, HistoryServer (, CLI?). Everything else would be left to the implementation which would live outside of Flink. For

Re: [DISCUSS] FLIP-211: Kerberos delegation token framework

2022-02-04 Thread Gabor Somogyi
Hi All, First of all sorry that I've taken couple of mails heavily! I've had an impression after we've invested roughly 2 months into the FLIP it's moving to a rejection without alternative what we can work on. That said earlier which still stands if there is a better idea how that could be

[jira] [Created] (FLINK-25956) KryoSerializer is noisy in scala-free Flink

2022-02-04 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25956: Summary: KryoSerializer is noisy in scala-free Flink Key: FLINK-25956 URL: https://issues.apache.org/jira/browse/FLINK-25956 Project: Flink Issue

[jira] [Created] (FLINK-25955) Add JobManagerRunner implementation that picks up dirty job results to be cleaned up

2022-02-04 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25955: - Summary: Add JobManagerRunner implementation that picks up dirty job results to be cleaned up Key: FLINK-25955 URL: https://issues.apache.org/jira/browse/FLINK-25955

[jira] [Created] (FLINK-25954) Add cleanup tests to BlobServerCleanupTest

2022-02-04 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25954: - Summary: Add cleanup tests to BlobServerCleanupTest Key: FLINK-25954 URL: https://issues.apache.org/jira/browse/FLINK-25954 Project: Flink Issue Type:

[jira] [Created] (FLINK-25953) Reorganizes tests around Dispatcher cleanup

2022-02-04 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-25953: - Summary: Reorganizes tests around Dispatcher cleanup Key: FLINK-25953 URL: https://issues.apache.org/jira/browse/FLINK-25953 Project: Flink Issue Type: