[GitHub] kafka pull request: KAFKA-2975

[MayureshGharat]

Github user MayureshGharat closed the pull request at:

https://github.com/apache/kafka/pull/658


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request: KAFKA-2975

[MayureshGharat]

GitHub user MayureshGharat opened a pull request:

https://github.com/apache/kafka/pull/658

KAFKA-2975

The newtorkClient should request a metadata update after it gets an error 
in the handleResponse().

Currently in data pipeline,
1) Lets say Mirror Maker requestTimeout is set to 2 min and metadataExpiry 
is set to 5 min
2) We delete a topic, the Mirror Maker get UNKNOWN_TOPIC_PARTITION and 
tries torefresh its Metadata.
3) It gets LeaderNotAvailableException, may be because the topic is not 
created yet.
4) Now its metadata does not have any information about that topic.
5) It will wait for 5 min to do the next refresh.
6) In the mean time the batches sitting in the accumulator will expire and 
the mirror makers die to avoid data loss.

To overcome this we need to refresh the metadata after 3).

Well there is an alternative solution to have the metadataExpiry set to be 
less then requestTimeout, but this will mean we make more metadataRequest over 
the wire in normal scenario as well.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/MayureshGharat/kafka kafka-2975

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/658.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #658


commit 8c7534b5ff89960e26db22a27e183d10002aeb01
Author: Mayuresh Gharat 
Date:   2015-12-10T03:04:37Z

The newtorkClient should request a metadata update after it gets an error 
in response




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---