Re: Sharing data among flink operator pipeline

[shenzhongwei]

unsubscribe


         原始邮件
         
       
发件人：Nikola Milutinovic <n.milutino...@levi9.com&gt;
发件时间：2025年8月25日 17:26
收件人：user@flink.apache.org <user@flink.apache.org&gt;
主题：Re: Sharing data among flink operator pipeline



 Hi Kamal.

 
 When you say “Kafka Headers”, are you referring to “Kafka Message Key”? In 
Kafka, messages come as Key/Value pairs. You can omit the key, but if you do 
have it, you can read it in the job and then use it when writing to the other 
topic.

 
 Nix.

 
From: Kamal Mittal via user <user@flink.apache.org&gt;
 Date: Thursday, August 21, 2025 at 11:35 AM
 To: user@flink.apache.org <user@flink.apache.org&gt;
 Subject: Sharing data among flink operator pipeline
 
 

 Hello,

 &nbsp;

 I have a scenario like for kafka source where kafka headers also come along 
with kafka record/event. Kafka headers fetched need to share/pass to 
next/parallel operators in pipeline.

 &nbsp;

 So, is there any way to share data across operator pipeline?

 &nbsp;

 Explored keyed state which has limitation that only with keyby() it will work.

 Explored broadcast state which limitation that it can only work with process 
functions and not with map/flatmap/filter etc.

 &nbsp;

 Rgds,

 Kamal