RE: Flink JDBC: Disable auto-commit mode

Hi Rong, We have already tried to set the fetch size with no success. According to PG documentation we have to set both configuration parameters (i.e., auto-commit to false and limit fetch) to achieve our purpose. Thanks, Konstantinos From: Rong Rong Sent: Παρασκευή, 12 Απριλίου 2019 6:50 μμ

programmaticly start yarn session for flink job

Hi everyone! Is it a good way to start a yarn session programmatically for some flink jobs under kerbores? Thank you very much!

Re: Queryable state support in Flink 1.9

Hi, I think you should check TM log first and check if there are some info like: 1430 [main] INFO org.apache.flink.queryablestate.server.KvStateServerImpl - Started Queryable State Server @ /127.0.0.1:9069. 1436 [main] INFO org.apache.flink.queryablestate.client.proxy.KvStateClientProxyImpl - Sta

Re: How would I use OneInputStreamOperator to deal with data skew?

Hi, You can checkout the bundle operator which used in Blink to perform similar thing you mentioned: https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/runtime/bundle/BundleOperator.java Best, Kurt On Fri, Apr 12, 2019 at 8:05 PM Felipe G

Re: Can back pressure data be gathered by Flink metric system?

Hi Henry, The backpressure tracking is not realized in metric framework, you could check the details via [1]. I am not sure why your requirements is showing backpressure in metrics. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/back_pressure.html Best, Zhijiang

Re: Retain metrics counters across task restarts

Hi Peter, The lifecycle of these metrics are coupled with lifecycle of task, So the metrics would be initialized after task is restarted. I think of one possible option is that you could store your required metrics into state, then the metric states would be restored from backend after task is

Re: Netty channel closed at AKKA gated status

Hi Wenrui, I think the akka gated issue and inactive netty channel are both caused by some task manager exits/killed. You should double check the status and reason of this task manager `'athena592-phx2/10.80.118.166:44177'`. Best, Zhijiang ---

Can back pressure data be gathered by Flink metric system?

Hi Experts, From the page Flink metric system(https://ci.apache.org/projects/flink/flink-docs-release-1.8/monitoring/metrics.html#system-metrics ), I do not find the info about the back p

Re: Timestamp Watermark Assigner bpund question

sorry for missing a not. :( Whether the watermark, which is generated by the AssignerWithPunctuatedWatermarks/AssignerWithPeriodicWatermarks is send to the downstream is controlled by the framework. If an operator returns a watermark going back Flink would _*not_* send it to the downstream. Best,

Re: Join of DataStream and DataSet

Hi Reminia, Currently, we can't join a DataStream with a DataSet in Flink. However, the DataSet is actually a kind of bounded stream. From the point of this view, you can use a streaming job to achieve your goal. Flink Table API & SQL support different kinds of join[1]. You can take a closer look

Re: Timestamp Watermark Assigner bpund question

Hi, Vijay >>>Then the Operator progresses to the next Watermark as a starting point for events after event time reaches currWatermark ? AFAIK, the operator that generates watermark is called by the frame work. When the operator is called depends on the operator itself. For example the operator tha

Queryable state support in Flink 1.9

I was testing with Flink 1.9. Here is how I set up mini cluster int port = 6124; int parallelism = 2; Configuration config = new Configuration(); config.setInteger(JobManagerOptions.PORT, port); config.setString(JobMan