Re: Flink Job across Data Centers

[Hang Ruan]

Hi, Chirag,

I am not sure whether this FLIP-268[1] is what you want.

Best,
Hang

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-268%3A+Kafka+Rack+Awareness

Andrew Otto  于2023年4月12日周三 22:12写道：

> Hi, I asked a similar question in this thread
> , which
> might have some relevant info.
>
> On Wed, Apr 12, 2023 at 7:23 AM Chirag Dewan via user <
> user@flink.apache.org> wrote:
>
>> Hi,
>>
>> Can anyone share any experience on running Flink jobs across data centers?
>>
>> I am trying to create a Multi site/Geo Replicated Kafka cluster. I want
>> that my Flink job to be closely colocated with my Kafka multi site cluster.
>> If the Flink job is bound to a single data center, I believe we will
>> observe a lot of client latency by trying to access the broker in another
>> DC.
>>
>> Rather if I can make my Flink Kafka collectors as rack aware and start
>> fetching data from the closest Kafka broker, I should get better results.
>>
>> I will be deploying Flink 1.16 on Kubernetes with Strimzi managed Apache
>> Kafka.
>>
>> Thanks.
>>
>>

Re: Flink Job across Data Centers

[Andrew Otto]

Hi, I asked a similar question in this thread
, which
might have some relevant info.

On Wed, Apr 12, 2023 at 7:23 AM Chirag Dewan via user 
wrote:

> Hi,
>
> Can anyone share any experience on running Flink jobs across data centers?
>
> I am trying to create a Multi site/Geo Replicated Kafka cluster. I want
> that my Flink job to be closely colocated with my Kafka multi site cluster.
> If the Flink job is bound to a single data center, I believe we will
> observe a lot of client latency by trying to access the broker in another
> DC.
>
> Rather if I can make my Flink Kafka collectors as rack aware and start
> fetching data from the closest Kafka broker, I should get better results.
>
> I will be deploying Flink 1.16 on Kubernetes with Strimzi managed Apache
> Kafka.
>
> Thanks.
>
>

Flink Job across Data Centers

[Chirag Dewan via user]

Hi,
Can anyone share any experience on running Flink jobs across data centers?
I am trying to create a Multi site/Geo Replicated Kafka cluster. I want that my 
Flink job to be closely colocated with my Kafka multi site cluster. If the 
Flink job is bound to a single data center, I believe we will observe a lot of 
client latency by trying to access the broker in another DC.
Rather if I can make my Flink Kafka collectors as rack aware and start fetching 
data from the closest Kafka broker, I should get better results.
I will be deploying Flink 1.16 on Kubernetes with Strimzi managed Apache Kafka.
Thanks.