[jira] [Created] (FLINK-27359) Kubernetes operator throws NPE when testing with Flink 1.15

2022-04-22 Thread Yang Wang (Jira)
Yang Wang created FLINK-27359: - Summary: Kubernetes operator throws NPE when testing with Flink 1.15 Key: FLINK-27359 URL: https://issues.apache.org/jira/browse/FLINK-27359 Project: Flink Issue

[jira] [Created] (FLINK-27358) Kubernetes operator throws NPE when testing with Flink 1.15

2022-04-22 Thread Yang Wang (Jira)
Yang Wang created FLINK-27358: - Summary: Kubernetes operator throws NPE when testing with Flink 1.15 Key: FLINK-27358 URL: https://issues.apache.org/jira/browse/FLINK-27358 Project: Flink Issue

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-22 Thread Yun Gao
Very thanks Peter for reporting the problem and Matthias for tracking this issue! From my side I also agree with that we could wait for more inputs to check if it implies some blocker issues. Also during this period, let's still continue to do the verification to see if this candidate might

[jira] [Created] (FLINK-27357) In Flink HA Service, Web UI External Service should update with new elected Job Manager leader's IP

2022-04-22 Thread Jesus H Christ (Jira)
Jesus H Christ created FLINK-27357: -- Summary: In Flink HA Service, Web UI External Service should update with new elected Job Manager leader's IP Key: FLINK-27357 URL:

[jira] [Created] (FLINK-27356) Incorrect Number of Methods Listed for SplitReader

2022-04-22 Thread Austin Cawley-Edwards (Jira)
Austin Cawley-Edwards created FLINK-27356: - Summary: Incorrect Number of Methods Listed for SplitReader Key: FLINK-27356 URL: https://issues.apache.org/jira/browse/FLINK-27356 Project: Flink

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-22 Thread Matthias Pohl
An issue was reported on the user ML [1] by Peter (CC) that was observed on rc4. We couldn't find any blocker based on the description so far. But it doesn't explain the entire problem the user is describing, though. We're still waiting for the logs to investigate the cause of the JobMaster not

RE: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-04-22 Thread Александр Смирнов
Hi Qingsheng! My name is Alexander, I'm not a committer yet, but I'd really like to become one. And this FLIP really interested me. Actually I have worked on a similar feature in my company’s Flink fork, and we would like to share our thoughts on this and make code open source. I think there is a

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread godfrey he
Hi Chesnay, There is no bug fix version until now. You can find the releases in https://github.com/apache/calcite/tags Best, Godfrey Chesnay Schepler 于2022年4月22日周五 18:48写道: > > I find it a bit weird that the supposed only way to get a bug fix is to > do a big version upgrade. > Is Calcite not

Re: [DISCUSS] FLIP-220: Temporal State

2022-04-22 Thread David Morávek
> > With that in mind, we could only offer a couple of selected > temporal/sorted > state implementations that are handled internally, but not really a > generic > one - even if you let the user explicitly handle binary keys... If we want to have a generic interface that is portable between

Re: [DISCUSS] FLIP-220: Temporal State

2022-04-22 Thread David Morávek
> > Isn't allowing a TemporalValueState just a special case of b.III? So if a > user > of the state wants that, then they can leverage a simple API vs. if you > want > fancier duplicate handling, you'd just go with TemporalListState and > implement > the logic you want? Yes it is. But it IMO

Re: [DISCUSS] FLIP-220: Temporal State

2022-04-22 Thread Nico Kruber
David, 1) Good points on the possibility to make the TemporalListState generic -> actually, if you think about it more, we are currently assuming that all state backends use the same comparison on the binary level because we add an appropriate serializer at an earlier abstraction level. This

[jira] [Created] (FLINK-27355) JobManagerRunnerRegistry.localCleanupAsync does not call the JobManagerRunner.close method repeatedly

2022-04-22 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-27355: - Summary: JobManagerRunnerRegistry.localCleanupAsync does not call the JobManagerRunner.close method repeatedly Key: FLINK-27355 URL:

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread Chesnay Schepler
I find it a bit weird that the supposed only way to get a bug fix is to do a big version upgrade. Is Calcite not creating bugfix releases? On 22/04/2022 12:26, godfrey he wrote: Thanks for the feedback, guys! For Jingsong's feedback: ## Do we have the plan to upgrade calcite to 1.31? I

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread godfrey he
Thanks for the feedback, guys! For Jingsong's feedback: >## Do we have the plan to upgrade calcite to 1.31? I think we will upgrade Calcite to 1.31 only when Flink depends on some significant features of Calcite. Such as: new syntax PTF (CALCITE-4865). >## Is Cherry-pick costly? >From the

Re: [DISCUSS] FLIP-222: Support full query lifecycle statements in SQL client

2022-04-22 Thread Paul Lam
Hi Shengkai, Thanks a lot for your opinions! > 1. I think the keyword QUERY may confuse users because the statement also > works for the DML statement. I slightly lean to QUERY, because: Hive calls DMLs queries. We could be better aligned with Hive using QUERY, especially given that we plan

[jira] [Created] (FLINK-27354) JobMaster still processes requests while terminating

2022-04-22 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-27354: - Summary: JobMaster still processes requests while terminating Key: FLINK-27354 URL: https://issues.apache.org/jira/browse/FLINK-27354 Project: Flink Issue

[jira] [Created] (FLINK-27353) Update training exercises to use Flink 1.15

2022-04-22 Thread Nico Kruber (Jira)
Nico Kruber created FLINK-27353: --- Summary: Update training exercises to use Flink 1.15 Key: FLINK-27353 URL: https://issues.apache.org/jira/browse/FLINK-27353 Project: Flink Issue Type: New

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread Martijn Visser
Hi everyone, Overall I'm against the idea of setting up a Calcite fork for the same reasons that Chesnay has mentioned. We've talked extensively about doing an upgrade of Calcite during the Flink 1.15 release period, but there was a lot of pushback by the maintainers against that because of the

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-22 Thread Xingbo Huang
+1 (non-binding) - Verified checksums and signatures - Verified Python wheel package contents - Pip install apache-flink-libraries source package and apache-flink wheel package in Mac/Linux - Run the examples from Python Table API Tutorial in Python REPL - Test the Python UDF jobs in Thread Mode.

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread Xintong Song
BTW, I think this proposal sounds similar to FRocksDB, the Flink's custom RocksDB build. Maybe folks maintaining FRocksDB can share some experiences. CC @Yun Tang Thank you~ Xintong Song On Fri, Apr 22, 2022 at 4:35 PM Xintong Song wrote: > Hi Godfrey, > > >> 1. Where to put the code?

Re: [DISCUSS] FLIP-220: Temporal State

2022-04-22 Thread David Morávek
Hi Yun & Nico, few thoughts on the discussion 1) Making the TemporalListState generic This is just not possible with the current infrastructure w.r.t type serializers as the sorting key *needs to be comparable on the binary level* (serialized form). What I could imagine, is introducing some

Re: [DISCUSS] FLIP-223: Support HiveServer2 Endpoint

2022-04-22 Thread Nicholas Jiang
Hi Shengkai. Thanks for driving the proposal of HiveServer2 Endpoint support. For the "GatewayService API Change", I don't think the motivation for supporting HiveServer2 endpoint need to change the GatewayService API, in other words, integrating the Hive ecosystem should not require changing

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread Xintong Song
Hi Godfrey, > 1. Where to put the code? https://github.com/flink-extended is a good > place. Please notice that `flink-extended` is not endorsed by the Apache Flink PMC. That means if the proposed new Calcite repository is hosted there, the maintenance and release will not be guaranteed by the

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread Konstantin Knauf
I share Chesay's concerns. I fear that we will generally upgrade Calcite even fewer and - in consequence - when we upgrade it will be even more painful. Can you elaborate on why Calcite upgrades have been hard in the past? Is there anything we can do to make this better? On Fri, Apr 22, 2022 at

Re: [DISCUSS] FLIP-91: Support SQL Client Gateway

2022-04-22 Thread Nicholas Jiang
Hi Shengkai. Thanks for driving the proposal of SQL Client Gateway. I have some knowledge of Kyuubi and have some questions about the design: 1.Do the public interfaces of GatewayService refer to any service? If referring to HiveService, does GatewayService need interfaces like getQueryId etc.

Re: [DISCUSS] Docker images for SNAPSHOT

2022-04-22 Thread Yang Wang
The project flink-kubernetes-operator has already been using github packages to deliver the snapshot images[1]. [1]. https://github.com/apache/flink-kubernetes-operator/pkgs/container/flink-kubernetes-operator Best, Yang Jingsong Li 于2022年4月22日周五 10:43写道: > +1 to public Flink Docker image for

[jira] [Created] (FLINK-27352) [JUnit5 Migration] Module: flink-json

2022-04-22 Thread EMing Zhou (Jira)
EMing Zhou created FLINK-27352: -- Summary: [JUnit5 Migration] Module: flink-json Key: FLINK-27352 URL: https://issues.apache.org/jira/browse/FLINK-27352 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread Chesnay Schepler
I'm overall against the idea of creating a fork. It implies quite some maintenance overhead, like dealing with unstable tests, CI, licensing etc. and the overall release overhead. Is there no alternative where we can collaborate more with the calcite guys, like verifying new features so bugs

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-22 Thread Dawid Wysakowicz
@Konstantin: As part of this FLIP, the `AlignedSplitReader` interface (aka the stop & resume behavior) will be implemented for Kafka and Pulsar only, correct? Correct, as far as I know though, those are the only sources which consume concurrently from multiple splits and thus alignment

Re: [DISCUSS] FLIP-220: Temporal State

2022-04-22 Thread Nicholas Jiang
Hi David. Thanks for the proposal of Temporal State. I'm working for the benchmark of CEPOperator. I have question about how much performance improvement can Temporal State bring to CEPOperator and whether there is any expected performance? Best, Nicholas Jiang On 2022/04/11 12:54:15 David

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread Jingsong Li
Thanks Godfrey for driving this. Sounds good to me. I have a few questions: ## Do we have the plan to upgrade calcite to 1.31? It looks like only one major upgrade later will solve the current problem. ## Is Cherry-pick costly? If the conflict is large, this may require too much effort in

[DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-22 Thread godfrey he
Dear devs, I would like to open a discussion on the fact that currently many Flink SQL function development relies on Calcite releases, which seriously blocks some Flink SQL's features release. Therefore, I would like to discuss whether it is possible to solve this problem by creating Flink's

[jira] [Created] (FLINK-27351) TaskExecutorBuilder configures unreasonable amount of memory

2022-04-22 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27351: Summary: TaskExecutorBuilder configures unreasonable amount of memory Key: FLINK-27351 URL: https://issues.apache.org/jira/browse/FLINK-27351 Project: Flink

Re: [DISCUSS] FLIP-220: Temporal State

2022-04-22 Thread Yun Tang
Hi Nico, I did not mean that we need to support all APIs in NavigableMap, and it is indeed too heavily to implement them all. Moreover, I also prefer iterator-like API instead of the original #tailMap like API. I just use NavigableMap's API to show examples. I think we can introduce new APIs

Re: Re: [VOTE] FLIP-214: Support Advanced Function DDL

2022-04-22 Thread godfrey he
Hi, Ron Thanks for the explanation, +1 (binding) from my side Best, Godfrey 刘大龙 于2022年4月22日周五 13:45写道: > > > Hi, godfrey > > The add/delete jar syntax parse is supported in table environment side > currently, but the execution is implemented in SqlClient side. After this > FLIP, we will move