Hey Joe, Actually this API was asked for before, and we have several use cases in LinkedIn as well. I thought we have added that in KAFKA-1650 but obviously I forgot to do that.
My understanding is that we won¹t really deprecate high level consumer until we move to 0.9.0. So we can have this API either in 0.8.3 or 0.8.2.2. Do you mean we only add them to those releases but not put it into trunk? Any specific concern on that? Considering this API has already been provided in new consumer. Adding this method probably won¹t cause any API compatibility issue even if people move to new consumer later. Given it is both backward and forward compatible and is a one line change, I think it is probably OK to have it added. Thanks, Jiangjie (Becket) Qin On 5/13/15, 3:18 PM, "Joe Stein" <joe.st...@stealth.ly> wrote: >My gut reaction is that this isn't that important for folks otherwise they >would have complained already. If it is a blocker for folks upgrading to >0.8.2.1 then we should do a 0.8.2.2 release with this fix in it. For >0.9.0. >we are pushing for folks to start using the new consumer and that is the >upgrade path we should continue on, imho. If we are going to phase out the >scala clients then we need to strive to not be making changes to them on >trunk. > >~ Joe Stein >- - - - - - - - - - - - - - - - - > > http://www.stealth.ly >- - - - - - - - - - - - - - - - - > >On Wed, May 13, 2015 at 6:01 PM, Jiangjie Qin <j...@linkedin.com.invalid> >wrote: > >> Add the DISCUSS prefix to the email title : ) >> >> From: Jiangjie Qin <j...@linkedin.com<mailto:j...@linkedin.com>> >> Date: Tuesday, May 12, 2015 at 4:51 PM >> To: "dev@kafka.apache.org<mailto:dev@kafka.apache.org>" < >> dev@kafka.apache.org<mailto:dev@kafka.apache.org>> >> Subject: Add missing API to old high level consumer >> >> Hi, >> >> I just noticed that in KAFKA-1650 (which is before we use KIP) we added >>an >> offset commit method in high level consumer that commits offsets using a >> user provided offset map. >> >> public void commitOffsets(Map<TopicPartition, OffsetAndMetadata> >> offsetsToCommit, boolean retryOnFailure); >> >> This method was added to all the Scala classes but I forgot to add it to >> Java API of ConsumerConnector. (Already regretting now. . .) >> This method is very useful in several cases and has been asked for from >> time to time. For example, people have several threads consuming >>messages >> and processing them. Without this method, one thread will unexpectedly >> commit offsets for another thread, thus might lose some messages if >> something goes wrong. >> >> I created KAFKA-2186 and hope we can add this missing method into the >>Java >> API of old high level consumer (literarily one line change). >> Although this method should have been there since KAFKA-1650, adding >>this >> method to Java API now is a public API change, just want to see if >>people >> think we need a KIP for this. >> >> Thanks. >> >> Jiangjie (Becket) Qin >>
