Re: Kubernetes operator listing jobs TimeoutException

2023-06-07 Thread Shammon FY
Hi Evgeniy, >From the following exception message: at org.apache.flink.shaded.netty4.io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:123) at org.apache.flink.runtime.rest.RestClient.submitRequest(RestClient.java:469) at

Re: pyflink1.17 中文乱码

2023-06-07 Thread Shammon FY
Hi, 你是怎么运行的?是不是中文的文件编码格式不对? Best, Shammon FY On Thu, Jun 8, 2023 at 10:07 AM yidan zhao wrote: > 可以描述再详细点 > > 1 于2023年6月7日周三 19:55写道: > > > > 老师们好,pyflink运行官网例子 wordcount 。把单词改成中文 乱码 > > > > > > > > > > >

Re: pyflink1.17 中文乱码

2023-06-07 Thread yidan zhao
可以描述再详细点 1 于2023年6月7日周三 19:55写道: > > 老师们好,pyflink运行官网例子 wordcount 。把单词改成中文 乱码 > > > > >

Re: [ANNOUNCE] Apache flink-connector-pulsar v3.0.1 released

2023-06-07 Thread Neng Lu
Thank you very much for coordinating this! I think we also need to release 4.0.1 to fix the pulsar-client OAuth2 issue. On Wed, Jun 7, 2023 at 1:48 AM Leonard Xu wrote: > The Apache Flink community is very happy to announce the release of Apache > flink-connector-pulsar v3.0.1. > This release

Kubernetes operator listing jobs TimeoutException

2023-06-07 Thread Evgeniy Lyutikov
Hello. We use Kubernetes operator 1.4.0, operator serves about 50 jobs, but sometimes there are errors in the logs that are reflected in the metrics (FlinkDeployment.JmDeploymentStatus.READY.Count). What is the reason for such errors? 2023-06-07 15:28:27,601

PyFlink Error JAR files

2023-06-07 Thread Kadiyala, Ruthvik via user
Hi, Please find below the code I have been using to consume a Kafka Stream that is hosted on confluent. It returns an error regarding the jar files. Please find the error below the code snippet. Let me know what I am doing wrong. I am running this on Docker with Flink Version: 1.7.1. Code:

RE: Parquet decoding exception - Flink 1.16.x

2023-06-07 Thread Kamal Mittal via user
Hello, Metrices link given in below mail doesn’t give any way to create metrices for source function right? I am using below Flink API to read/decode parquet data, query is where exception can be caught for error like “decoding exception” from internal parquet API like “AvroParquetReader” and

Re: flink on yarn rocksdb内存超用

2023-06-07 Thread Hangxiang Yu
Hi, 目前对RocksDB使用的内存是没有严格限制住的,可以参考这个 ticket: https://issues.apache.org/jira/browse/FLINK-15532 如果要定位到内存使用情况,可以先看一些粗的Metrics: https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#rocksdb-native-metrics 如果要再细致定位到单 instance 内部 RocksDB 的详细内存使用情况,可能需要用 malloc

Re: Custom Counter on Flink File Source

2023-06-07 Thread Hang Ruan
Hi, Kirti. We could find these information in the 1.18 release wiki page[1]. Its timeline is as follows. Feature Freeze: July 11, 2023, end of business CEST Release: End of September 2023 Best, Hang [1] https://cwiki.apache.org/confluence/display/FLINK/1.18+Release#id-1.18Release-Summary

pyflink1.17 中文乱码

2023-06-07 Thread 1
老师们好,pyflink运行官网例子 wordcount 。把单词改成中文 乱码

Re: Raise alarm for corrupt records

2023-06-07 Thread Martijn Visser
Hi Kamal, Documentation on the metrics can be found at https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ Best regards, Martijn On Wed, Jun 7, 2023 at 10:13 AM Kamal Mittal via user wrote: > Hello, > > > > Thanks for quick reply. > > > > I am using parquet encoder/decoder

[ANNOUNCE] Apache flink-connector-pulsar v3.0.1 released

2023-06-07 Thread Leonard Xu
The Apache Flink community is very happy to announce the release of Apache flink-connector-pulsar v3.0.1. This release is compatible with Flink 1.16.x series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data

RE: Raise alarm for corrupt records

2023-06-07 Thread Kamal Mittal via user
Hello, Thanks for quick reply. I am using parquet encoder/decoder and during decoding if any corrupt record comes then need to raise alarm and maintain metrices visible over Flink Metrices GUI. So any custom metrices can be created in Flink? Please give some reference of any such

RE: Custom Counter on Flink File Source

2023-06-07 Thread Kirti Dhar Upadhyay K via user
Thanks Hang. Any expected date for Flink 1.18.0 release? Regards, Kirti Dhar From: Hang Ruan Sent: 07 June 2023 07:34 To: Kirti Dhar Upadhyay K Cc: user@flink.apache.org Subject: Re: Custom Counter on Flink File Source Hi, Kirti Dhar Upadhyay K. I check the FLIP-274[1]. This issue will be

Re: Raise alarm for corrupt records

2023-06-07 Thread Martijn Visser
Hi Kamal, No, but it should be straightforward to create metrics or events for these types of situations and integrate them with your own alerting solution. Best regards, Martijn On Wed, Jun 7, 2023 at 8:25 AM Kamal Mittal via user wrote: > Hello Community, > > > > Is there any way Flink

Raise alarm for corrupt records

2023-06-07 Thread Kamal Mittal via user
Hello Community, Is there any way Flink provides out of box to raise alarm for corrupt records (e.g. due to decoding failure) in between of running data pipeline and send this alarm to outside of task manager process? Rgds, Kamal