HHoflittlefish777 commented on issue #31531: URL: https://github.com/apache/doris/issues/31531#issuecomment-1975804469
> > > > > so I think add new configuration into BE is inevitable > > > > > > > > > > > > In fact, the consumption task of routinue load is scheduled by FE, and the consumption task corresponding to a job may be on different BE, the consumption of tasks is also determined by FE, not BE. So I think it is better which region should routine load task consume when job was created, it can be a job property or FE config. > > > > > > > > > FE assign topic-partitions to BEs, but BE is the one send fetch request to kafka, even FE want to assign partition to BE without cross network flow, it still need know BE's region, so a parameter specify the region of BE is requisite. maybe I still couldn't understand what you mean, could you provide a more detail description? > > > > > > It needs to know the region of BE, different BE may be in different regions, and then determine which partition of Kafka this BE should consume based on this region. Do you mean this? > > yes that's what I mean We have no disagreement on this point. My previous question was that the job may send consumption tasks from different partitions to different BE, and it is possible that the region of this BE and the region of the partition are different. Is there a possibility of this issue？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
