Nope. You can put anything you want, just set parseAsFlumeEvent to false and the channel won't attempt to convert it into a Flume event. It just stashes the whole thing into the body of the returned event.
Thanks, Hari On Thu, Aug 27, 2015 at 5:53 PM, Roshan Naik <[email protected]> wrote: > My understanding is that the Kafka channel expects "Flume Event" objects > to be stored in the Kafka topic. > Isn't that right ? > -roshan > > > On 8/27/15 5:47 PM, "Hari Shreedharan" <[email protected]> wrote: > > >So one of the things that the already existing Kafka channel can do is to > >run without a source. Does this outperform that as well? I have already > >seen people use it this way. > > > > > >Thanks, > >Hari > > > >On Thu, Aug 27, 2015 at 4:11 PM, Roshan Naik <[email protected]> > >wrote: > > > >> Wanted to give a heads-up on this idea I have been working on Š > >> > >> Using Flume as a Kafka producer or consumer has been gaining popularity > >> thanks to the Flafka components that were recently introduced. > >> > >> For the use case of Flume as a Kafka consumer, it appears we can > >>sidestep > >> the compromise between Mem channel (which is fast but can lose data) and > >> File channel (which is slow but won't lose data) and get the best of > >>both > >> worlds. > >> > >> I have a prototype of this idea for a "Kafka Consumer" channel. It is > >> designed to enable the use of Flume as a really light weight and very > >>fast > >> Kafka consumer without the data loss potential of mem channel. My > >> measurements indicate it easily outperforms memory channel. > >> > >> Additional info here Š > >> https://github.com/roshannaik/kafka-consumer-channel > >> > >> I think the same idea could be applied for "Kafka producer channel". > >> > >> -roshan > >> > >
