[
https://issues.apache.org/jira/browse/FLUME-860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Jarcec Cecho updated FLUME-860:
-------------------------------------
Attachment: input
> Combination of ExecSource and HDFS DataStream is removing end line characters
> -----------------------------------------------------------------------------
>
> Key: FLUME-860
> URL: https://issues.apache.org/jira/browse/FLUME-860
> Project: Flume
> Issue Type: Bug
> Affects Versions: NG alpha 2
> Reporter: Jarek Jarcec Cecho
> Attachments: hddev01.log, hddev01.properties, hddev02.log,
> hddev02.properties, input, output
>
>
> I've noticed that combination of ExecSource and HDFS Sink configured to use
> DataStream is removing end line characters and thus is creating one line
> output file.
> I've used two centos boxes where first was acting as an agent, reading local
> log file using ExecSource. Second machine was acting as a collector, waiting
> for input events and storing them on HDFS. Both machines were connected
> using AVRO sink+source combination. You can find configuration files for both
> machines with their logs as well attached to this JIRA bug.
> I've executed both flume-ng instances using following commands:
> ./bin/flume-ng node --conf conf/ --classpath flume-ng.jar --f
> conf/configuration.properties -n hddev01 > hddev01.log 2>&1
> ./bin/flume-ng node --conf conf/ --classpath flume-ng.jar --f
> conf/configuration.properties -n hddev02 > hddev02.log 2>&1
> Input file was created using following small bash script (it was executed
> after flume-ng was successfully started):
> for i in `seq -w 01 10`; do echo $i; echo Yoda-$i >> /var/log/jarcec; sleep
> 1s; done
> Please note that I had to apply patch from FLUME-858 in order to get
> DataStream file type working.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
