Re: flink cdc metrics 问题

你好，目前flink cdc没有提供未消费binlog数据条数这样的指标，你可以通过 currentFetchEventTimeLag 这个指标（表示消费到的binlog数据中时间与当前时间延迟）来判断当前消费情况。 [1]

Re: Combining multiple stages into a multi-stage processing pipeline

Thank you Yunfeng. Your comments gave me some insights to explore how to use consecutive windows. So, I coded up a version that looks like this and works well for me: KafkaSource => Keyby => TumblingWindows => ProcessWindowFn => WindowAll => ProcessWindowFn => (Here I will repeated keyed and

flink cdc metrics 问题

请问flink cdc对外有暴露一些监控metrics么？ 我希望能够监控到使用flink cdc的实时作业当前未消费的binlog数据条数，类似于kafka topic消费积压监控。 想通过这个监控防止flink cdc实时作业消费慢而被套圈（最大binlog条数如何获取？）

Re: Impact on using clean code and serializing everything

Hi Oscar, I assume the "dependency" in your description refers to the custom fields in the ProcessFunction's implementation. You are right that as the ProcessFunction inherits `Serializable` interface so we should make all fields either serializable or transient. As for performance, I have no

Re: 退订

Hi， If you want to unsubscribe to user-zh mailing list, please send an email with any content to user-zh-unsubscr...@flink.apache.org . 退订请发送任意内容的邮件到 user-zh-unsubscr...@flink.apache.org . Best, Biao Geng 995626544 <995626...@qq.com.invalid> 于2024年4月7日周日 16:06写道： > 退订 > > > > > 995626544 >

Re: How to enable RocksDB native metrics？

Hi Lei, You can use the "-D" option in the command line to set configs for a specific job. E.g, `flink run-application -t yarn-application -Djobmanager.memory.process.size=1024m `. See https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/deployment/cli/ for more details. Best,

Re: How to enable RocksDB native metrics？

I can enable them by adding to flink-conf.yaml, it will work. However, I don't want to edit the flink-conf.yaml file, I want to enable the configurations when submitting a job on cmd line, then it only works for the job I submitted, I have no idea how to do this? Thanks, Lei On Mon, Apr 8, 2024

Re: How to enable RocksDB native metrics？

Hi Lei, Have you tried enabling these Flink configuration properties?Configurationnightlies.apache.orgSent from my iPhoneOn Apr 7, 2024, at 6:03 PM, Lei Wang wrote:I  want to enable it only for specified jobs, how can I specify the   configurations on  cmd line when submitting a job?Thanks，LeiOn

Re: How to enable RocksDB native metrics？

I want to enable it only for specified jobs, how can I specify the configurations on cmd line when submitting a job? Thanks， Lei On Sun, Apr 7, 2024 at 4:59 PM Zakelly Lan wrote: > Hi Lei, > > You can enable it by some configurations listed in: >

Re: Debugging Kryo Fallback

Thanks Yunfeng! That is more or less what I do now when I run into the problem. This approach reports problems one at a time (an exception is raised on the first problem encountered). Instead of that, I think accumulating all the issues and presenting them all at once would be more user friendly.

Re: How to enable RocksDB native metrics？

Hi Lei, You can enable it by some configurations listed in: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/#rocksdb-native-metrics (RocksDB Native Metrics) Best, Zakelly On Sun, Apr 7, 2024 at 4:59 PM Zakelly Lan wrote: > Hi Lei, > > You can enable it by

Re: How to enable RocksDB native metrics？

You can take a look at the document. [ https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/#rocksdb-native-metrics ] Thanks, Zbz > 2024年4月7日 13:41，Lei Wang 写道： > > > Using big state and want to do some performance tuning, how can i enable > RocksDB native