> On Aug. 16, 2016, 8:42 a.m., Balázs Donát Bessenyei wrote: > > flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java, > > line 301 > > <https://reviews.apache.org/r/51112/diff/1/?file=1474463#file1474463line301> > > > > Do you think we could use the Kafka Consumer Configs ( > > https://kafka.apache.org/08/configuration.html#consumerconfigs ) instead of > > the hard-coded 30000-s?
The new consumer doesn't have zookeeper configurations I can leverage. What configurations should I use? Tying it to some configuration isn't a bad idea. Though this client isn't used for long and 30 seconds is pretty liberal. So I am not sure its neccessary. > On Aug. 16, 2016, 8:42 a.m., Balázs Donát Bessenyei wrote: > > flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java, > > lines 325-326 > > <https://reviews.apache.org/r/51112/diff/1/?file=1474463#file1474463line325> > > > > Is this catch required here? Not explicitly. I can remove it. - Grant ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51112/#review145846 ----------------------------------------------------------- On Aug. 16, 2016, 6:12 p.m., Grant Henke wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51112/ > ----------------------------------------------------------- > > (Updated Aug. 16, 2016, 6:12 p.m.) > > > Review request for Flume. > > > Bugs: FLUME-2972 > https://issues.apache.org/jira/browse/FLUME-2972 > > > Repository: flume-git > > > Description > ------- > > Offsets tracking the position in Kafka consumers change from using zookeeper > for offset storage to Kafka when moving from 0.8.x to 0.9.x. > FLUME-2823 makes the client change in the Kafka Channel but does not ensure > existing offsets get migrated in order to continue consuming where it left > off. > > This patch adds automated logic on startup to check if Kafka offsets exist, > if not and migration is enabled (by default) then the offsets from Zookeeper > are copied and committed to Kafka. > > > Diffs > ----- > > flume-ng-channels/flume-kafka-channel/pom.xml 587b4b4 > > flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java > 90e3288 > > flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannelConfiguration.java > ccf46d9 > > flume-ng-channels/flume-kafka-channel/src/test/java/org/apache/flume/channel/kafka/TestKafkaChannel.java > b63ac9b > flume-ng-doc/sphinx/FlumeUserGuide.rst fde9ff7 > > Diff: https://reviews.apache.org/r/51112/diff/ > > > Testing > ------- > > Unit tests so far. > > > Thanks, > > Grant Henke > >
