[Question] Why doesn't Flink use Calcite adapter?

I have read the design doc of the Flink planner recently. I've found the Flink only uses Calcite as an SQL optimizer. It translates an optimized RelNode to Flink（or Blink）RelNode, and then transfers it to the physical plan. Why doesn't Flink

Cancel job error ! Interrupted while waiting for buffer

Hi I use Flink 1.12.4 on yarn, job topology is. kafka -> source -> flatmap -> window 1 min agg -> sink -> kafka. Checkpoint is enable , checkpoint interval is 20s . When I cancel my job, some TM cancel success, some TM become cenceling and the TM will be kill by itself with

Re: upgrade kafka-schema-registry-client to 6.1.2 for Flink-avro-confluent-registry

Thanks Fabian. [FLINK-23160] upgrade kafka-schema-registry-client to 6.1.2 for Flink-avro-confluent-registry - ASF JIRA (apache.org) is created. I will create a private build Flink to try out the fix. If it goes well, I can contribute back.

Re: Metric for JVM Overhaed

Hi Pranjul, Currently, Flink only have the metrics shown in taskmanager UI to tell the capacity of JMV overhead. However, Flink cannot detect how much overhead memory has been occupied as those memory footprints might be asked by the third-party library via OS malloc directly instead of via

Re: How to make onTimer() trigger on a CoProcessFunction after a failure?

Sorry for the delayed response but I'm glad to hear you have solved the problem. Piotrek czw., 24 cze 2021 o 10:55 Felipe Gutierrez napisał(a): > So, just an update. > > When I used this code (My stateful watermark) on the original application > it seems that I can recover the latest

Looking for example code

I'm just starting with Flink. I've been trying all the examples online and none of them work.I am not a Java programmer but have been programming since 1982.I would like example code to read from a Pulsar topic and output to another Pulsar topic.Pulsar version 2.8.0Flink version 1.13.1Scala

Re: upgrade kafka-schema-registry-client to 6.1.2 for Flink-avro-confluent-registry

Hi, Thanks for bringing this up. It looks to me like something we definitely want to fix. Unfortunately, I also do not see an easy workaround besides building your own flink-avro-confluent-registry and bumping the dependency. Can you create a JIRA ticket for bumping the dependencies and would

Re: Flink 1.4.1 randomically responds HTTP 500 when sending job to Job Manager

Hi Burcu Could you show more logs? I could try to help find out what is happening. But to be honest the 1.4 is too old a version that the community does not support. You’d better upgrade to a newer version. Best, Guowei On Fri, Jun 25, 2021 at 2:48 PM Burcu Gül POLAT EĞRİ wrote: > Dear All, >

Re: Metric for JVM Overhaed

Hi Pranjul There are already some system metrics that track the jvm status(CPU/Memory/Threads/GC). You could find them in the [1] [1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#system-metrics Best, Guowei On Fri, Jun 25, 2021 at 2:33 PM Pranjul Ahuja wrote:

Flink 1.4.1 randomically responds HTTP 500 when sending job to Job Manager

Dear All, we are using Flink 1.4.1 one of our projects. We send some image processing jobs to our processing nodes via Flink. Flink Task Managers are installed on each processing nodes. And our main application sends jobs to Flink Job Manager and Flink Job Manager sends jobs to Flink Task Manages

Metric for JVM Overhaed

Hi, Is there any metric to track the task manager JVM overhead? Or is it the case that it is already included in the metric Status.JVM.Memory.NonHeap? Thanks, Pranjul