The explanation could be clearer but it boils down to this: -If you use parseAsEvent=true (default) Then the events in the channel are avro FlumeEvents. So you have to read/write avro but you have all the object metadata (timestamp, headers, etc)
-If you use parseAsEvent=false Then the events in the channel is just text (no metadata) You have to choose either to use pure text or have a richer FlumeEvent object. On 3 June 2016 at 09:48, George M. <[email protected]> wrote: > Hi Gonzalo, > > > Seems the morphline is transforming the event into one without "body", are > you converting the event body into headers? > > > Yes. I am converting the event body into headers with Kite grok command > with regex patterns. For example, for the event commented in previous > messages, i'am creating unixtime, address, size, hostip, line_type and > event_route headers. > > > parseAsFlumeEvent only handles the body, the headers are lost. > > > I do not find in the user guide any reference about that. > > "Expecting Avro datums with FlumeEvent schema in the channel. This should > be true if Flume source is writing to the channel And false if other > producers are writing into the topic that the channel is using Flume source > messages to Kafka can be parsed outside of Flume by using > org.apache.flume.source.avro.AvroFlumeEvent provided by the flume-ng-sdk > artifact" > > Maybe, is the description a little confusing? > > Are the headers lost? In the kafka message below, the headers appear but > in a 'curious' way. > > Regards. > > > > > On 2 June 2016 at 16:59, George M. <[email protected]> wrote: > > With the morphlines and without multiplexing > ============================ > ------------------------------------------ > parseAsFlumeEvent = true > ------------------------------------------ > unixtime 1420119195000address,[email protected] > <[email protected]> > hostip > 10.1.0.170 line_type sender id 938223BD77C event_route SENDERmessage� 2015 > Jan 1 14:33:15 10.1.0.170 postfix-dsp/qmgr[5605]: 938223BD77C: from=< > [email protected]>, size=1060, nrcpt=1 (queue active) > > ------------------------------------------- > parseAsFlumeEvent = false > ------------------------------------------- > Only new lines > > > > >
