[jira] [Commented] (KAFKA-13029) FindCoordinators batching can break consumers during rolling upgrade
[ https://issues.apache.org/jira/browse/KAFKA-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373991#comment-17373991 ] Rajini Sivaram commented on KAFKA-13029: [~mimaison] [~dajac] Consumer and transaction coordinator always have single group, so the batch flag in those with failover are unnecessary. As David mentioned, we just need to set the appropriate data based on versions. The plan is to do the same for batching OffsetFetchRequest as well. For Admin Client, the PR currently switches to unbatched mode if there is an error. But the unbatched mode now works with any version (by setting appropriate data). So while performance could be improved in case an admin client gets into this state using the version of individual brokers, it didn't seem worthwhile making that change now for admin clients. > FindCoordinators batching can break consumers during rolling upgrade > > > Key: KAFKA-13029 > URL: https://issues.apache.org/jira/browse/KAFKA-13029 > Project: Kafka > Issue Type: Bug > Components: consumer >Reporter: Rajini Sivaram >Assignee: Rajini Sivaram >Priority: Blocker > Fix For: 3.0.0 > > > The changes made under KIP-699 assume that it is always safe to use unbatched > mode since we move from batched to unbatched and cache the value forever in > clients if a broker doesn't support batching. During rolling upgrade, if a > request is sent to an older broker, we move from batched to unbatched mode. > The consumer (admin client as well I think) disables batching and future > requests to upgraded brokers will fail because we attempt to use unbatched > requests with a newer version of the request. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-13029) FindCoordinators batching can break consumers during rolling upgrade
[ https://issues.apache.org/jira/browse/KAFKA-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373909#comment-17373909 ] David Jacot commented on KAFKA-13029: - For the consumer, I think a simpler version would be to have all the logic in the builder. As the request is always for one group, we could simply build the correct request data based on the version passed to ‘Builder#build’. > FindCoordinators batching can break consumers during rolling upgrade > > > Key: KAFKA-13029 > URL: https://issues.apache.org/jira/browse/KAFKA-13029 > Project: Kafka > Issue Type: Bug > Components: consumer >Reporter: Rajini Sivaram >Assignee: Rajini Sivaram >Priority: Blocker > Fix For: 3.0.0 > > > The changes made under KIP-699 assume that it is always safe to use unbatched > mode since we move from batched to unbatched and cache the value forever in > clients if a broker doesn't support batching. During rolling upgrade, if a > request is sent to an older broker, we move from batched to unbatched mode. > The consumer (admin client as well I think) disables batching and future > requests to upgraded brokers will fail because we attempt to use unbatched > requests with a newer version of the request. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-13029) FindCoordinators batching can break consumers during rolling upgrade
[ https://issues.apache.org/jira/browse/KAFKA-13029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373734#comment-17373734 ] Mickael Maison commented on KAFKA-13029: Right, I missed that! For the Consumer/Producer, I guess we could reset the batch flag in AbstractCoordinator each time the coordinator changes. For AdminClient, we create a new CoordinatorStrategy for each call but I think this could still happen if the coordinator changes during a call while we upgrade brokers. It's a bit more tricky because the strategy doesn't know explicitly if the coordinator just changed. > FindCoordinators batching can break consumers during rolling upgrade > > > Key: KAFKA-13029 > URL: https://issues.apache.org/jira/browse/KAFKA-13029 > Project: Kafka > Issue Type: Bug > Components: consumer >Reporter: Rajini Sivaram >Assignee: Rajini Sivaram >Priority: Blocker > Fix For: 3.0.0 > > > The changes made under KIP-699 assume that it is always safe to use unbatched > mode since we move from batched to unbatched and cache the value forever in > clients if a broker doesn't support batching. During rolling upgrade, if a > request is sent to an older broker, we move from batched to unbatched mode. > The consumer (admin client as well I think) disables batching and future > requests to upgraded brokers will fail because we attempt to use unbatched > requests with a newer version of the request. -- This message was sent by Atlassian Jira (v8.3.4#803005)
