Had a few questions around the Hadoop Consumer. - We have event data under the topic "foo" written to the kafka Server/Broker in avro format and want to write those events to HDFS. Does the Hadoop consumer expect the data written to HDFS already? Based on the doc looks like the DataGenerator is pulling events from the broker and writing to HDFS. In our case we only wanted to utilize the SimpleKafkaETLJob to write to HDFS. I am surely missing something here? - Is there a version of consumer which appends to an existing file on HDFS until it reaches a specific size?
Thanks, murtaza
