Ben, Please see the inlined answers below.
Thanks, Jun On Fri, Nov 23, 2012 at 7:51 AM, ben fleis <ben.fl...@gmail.com> wrote: > Hello all, > > I am writing a producer client for the v0.8 protocol. I have the wire > format encoding working, and much of the functionality works well. > However, I have made too many assumptions along the way, and it's time > that I figure out the answers to these things. Sorry for the 'laundry > list' -- most of them are related, and can possibly be answered in batch > form. > > TopicMetadata updates > > - is there a particular expectation of polling frequency? (per topic, > per broker, etc.?) > You just need to do the update the first time you send a message to a topic/partition or you get an error or an exception while sending a message (likely the leader has moved to a different broker). In general, you should be updating the metadata infrequently. > - should a particular topic's metadata request only go to the "current" > leader of that topic? > Metadata request can be answered by any broker. > - if such a request fails to receive response, is there a particular > expected behavior? > Then you send the same metadata request to another broker. Also, if the leader in the metadata response is null, it means that the leader is not available yet. You will need to wait a bit and retry the metadata request. > - I am assuming that producers always push to the leader (feels like > it's in the docs someplace, but can't confirm) > That's right. Producer/Fetch request can only be served at the leader of a partition. > - if the leader changes, what is the protocol for a producer? must the > producer await ACK from outstanding requests before making a change, > and if > so, presumeably re-push those requests? > > If the leader changes, current producer will receive an error code or an exception. This means that the produced data is not guaranteed to be in the broker. Depending on the application, the producer may choose to re-send. > I'm sure there will be more, but this opens the door. > > Cheers, > > ben >
