Re: Duplicate tasks for the same query

Thanks Jingsong and Kurt for more details. Yes, I'm planning to try out DeDuplication when I'm done upgrading to version 1.9. Hopefully deduplication is done by only one task and reused everywhere else. One more follow-up question, I see "For production use cases, we recommend the old planner

Best way set max heap size via env variables or program arguments?

Hey folks, we've been running a k8 flink application, using the taskmanager.sh script and passing in the -Djobmanager.heap.size=9000m and -Dtaskmanager.heap.size=7000m as options to the script. I noticed from the logs, that the Maximum heap size logged completely ignores these arguments, and just

Re: How to get kafka record's timestamp in job

> In kafka010, ConsumerRecord has a field named timestamp. It is encapsulated in Kafka010Fetcher. > How can I get the timestamp when I write a flink job? Kafka010Fetcher puts the timestamps into the StreamRecords that wrap your events. If you want to access these timestamps, you can use a

How to get kafka record's timestamp in job

In kafka010, ConsumerRecord has a field named timestamp. It is encapsulated in Kafka010Fetcher. How can I get the timestamp when I write a flink job? Thank you very much.

How to get kafka record's timestamp in job

In kafka010, ConsumerRecord has a field named timestamp. It is encapsulated in Kafka010Fetcher. How can I get the timestamp when I write a flink job? Thank you very much.

?????? flink??????????????

?? ??tm-XX:NewSize tm15G -XX:NewSize=2442764288 tm20G ?? -XX:NewSize=2442764288 ?? ---- ??:"Xintong Song"

Re: flink内存分配的问题

FLINK TM 中是用到了大量的堆外内存的，除了通常意义的 JVM 的栈空间、方法区等堆外开销外，还包括网络 buffer、batch 缓存、RocksDB等。 默认配置是相对保守，为了保证大多数情况下预留出足够的堆外内存。具体是否设置过大了，要看具体运行的作业的情况。可以尝试通过配置'containerized.heap-cutoff-ratio'进行调整。 另外，即将发布的flink 1.10版本中对TM的内存计算进行了优化，不再采用cutoff而是根据用途列出了更具体的配置项，欢迎试用 Thank you~ Xintong Song On Tue, Dec 31,

Does Flink SQL support stop with savepoint draining?

Hi, I tried running a stop with savepoint on my Flink job, which includes numerous Flink SQL streams. While running the command I used `-d` to drain all streams with MAX_WATERMARK. Looking at the Flink UI, all sources successfully finished, yet all Flink SQL streams were in a Running state, and

flink??????????????

taskmanager15Gheap10G tm??1.cutoff(15GB * 0.25) 2.heap(heap15GB - cutoff) 3.offheap(offheap??15GB-heap) offheap??-XX:MaxDirectMemorySize?? MaxDirectMemorySize??

如何获取算子处理一条数据记录的时间

Hi， 我是刚学习flink的菜鸟，现在想对一个算子，获取其处理一条数据记录的时间。请问这个应该怎么做？ 谢谢 | | 张江 | | 邮箱：zjkingdom2...@163.com | 签名由 网易邮箱大师 定制

Re: StreamTableEnvironment.registerDatastream() 开放用户自定义的schemaDescriptionh和DeserializationSchema

Hi aven, 这是个合理的需求。 现在的问题是： - Flink table只支持Row, Pojo, Tuple, CaseClass作为结构化的数据类型。 - 而你的类型是JSONObject，它其实也是一个结构化的数据类型，但是目前Flink不支持它，所以可以考虑有这样的DeserializationSchema机制来支持它。 但是我理解其实没有差别多少，比如你提供RowDeserializationSchema，其实就是JSONObject到Row的转换，那你完全可以把这个套在DataStream.map中，把它转换成Flink table支持的结构化类型。

回复: -yD Kerberos 认证问题

好的非常感谢，现在确实用的只是1.8,只关注了jira的状态，没有关注相关源码的改动 -邮件原件- 发件人: Terry Wang 发送时间: 2019年12月31日 14:42 收件人: user-zh@flink.apache.org 主题: Re: -yD Kerberos 认证问题 Hi ~ 这个问题在最新的代码上已经修复了，在flink 1.9 上应该也是不存在这个问题的，你可以用下看看~ Best, Terry Wang > 2019年12月31日 14:18， > 写道： > > 大家好 > >