Is there any 0.8 documentation available yet for all this? On Sun, Nov 25, 2012 at 9:30 PM, Jun Rao <jun...@gmail.com> wrote:
> Ben, > > Please see the inlined answers below. > > Thanks, > > Jun > > On Fri, Nov 23, 2012 at 7:51 AM, ben fleis <ben.fl...@gmail.com> wrote: > > > Hello all, > > > > I am writing a producer client for the v0.8 protocol. I have the wire > > format encoding working, and much of the functionality works well. > > However, I have made too many assumptions along the way, and it's time > > that I figure out the answers to these things. Sorry for the 'laundry > > list' -- most of them are related, and can possibly be answered in batch > > form. > > > > TopicMetadata updates > > > > - is there a particular expectation of polling frequency? (per topic, > > per broker, etc.?) > > > You just need to do the update the first time you send a message to a > topic/partition or you get an error or an exception while sending a message > (likely the leader has moved to a different broker). In general, you should > be updating the metadata infrequently. > > > > - should a particular topic's metadata request only go to the > "current" > > leader of that topic? > > > > Metadata request can be answered by any broker. > > > > - if such a request fails to receive response, is there a particular > > expected behavior? > > > > Then you send the same metadata request to another broker. Also, if the > leader in the metadata response is null, it means that the leader is not > available yet. You will need to wait a bit and retry the metadata request. > > > > - I am assuming that producers always push to the leader (feels like > > it's in the docs someplace, but can't confirm) > > > > That's right. Producer/Fetch request can only be served at the leader of a > partition. > > > > - if the leader changes, what is the protocol for a producer? must > the > > producer await ACK from outstanding requests before making a change, > > and if > > so, presumeably re-push those requests? > > > > > If the leader changes, current producer will receive an error code or an > exception. This means that the produced data is not guaranteed to be in the > broker. Depending on the application, the producer may choose to re-send. > > > > I'm sure there will be more, but this opens the door. > > > > Cheers, > > > > ben > > >
