Sure On Sep 1, 2011, at 1:02 PM, Jun Rao wrote: > Sam, > > Could you provide a patch for that? Thanks, > > Jun > > On Thu, Sep 1, 2011 at 10:56 AM, Sam William <sa...@stumbleupon.com> wrote: > >> Ben, >> There is a bug in KafkaETLContext.java in where the iterator is created >> multiple times. You ll have this issue only when kafka.request.limit is set >> to -1 in your test.properties. >> >> Line 120 Iterator<ByteBufferMessageSet> iter = _response.iterator(); >> >> >> This is what is causing the loops . In my setup , I fixed it by having a >> _respIterator attribute in the class and assigning >> it to _response.iterator() just once in the fetchMore() method. I use the >> class attribute in the rest of the places. >> >> Hope this helps. >> >> Sam >> >> >> On Sep 1, 2011, at 9:18 AM, Ben Ciceron wrote: >> >>> hello, >>> >>> i am able to run the job now but it loops indefinitely. >>> >>> can someone send a valid sample of 1.dat as generated by >>> kafka.etl.impl.DataGenerator in the README, please ? >>> mine seems awfully short (see below). >>> >>>>> 3. 1.dat is a sequence file, so if it isn't compressed, you should be >>>>> able to see its contents in plain text. The contents will essentially >> list >>>>> the kafka server url, the partition number and the topic as well as >> the >>>>> offset. >>>> >>>> >>>> mine has only 1 line (some encrypted in the middle of it is shown as >> is): >>>> >> SEQkafka.etl.KafkaETLKey"org.apache.hadoop.io.BytesWritable$?ړ??tl?3aC+tcp://<ip_of_my_hostA>:9092 >> SimpleTestEvent 0 -1 >> >> Sam William >> sa...@stumbleupon.com >> >> >> >>
Sam William sa...@stumbleupon.com
