file_roll is detailed in the Flume user guide. It's generated in
target/docs when you do a full build. I've put it up at
http://people.apache.org/~juhanic/flume-docs/FlumeUserGuide.html#file-roll-sink
<http://people.apache.org/%7Ejuhanic/flume-docs/FlumeUserGuide.html#file-roll-sink>
for convenience, but the version may not match yours.
If the docs are insufficient let us know and we'll try to address it.
On 07/12/2012 02:38 PM, Young Kim wrote:
After some debugging, I found out that the syslogTcp is working as
intended. I didn't realize that debug no longer shows the resulting
syslog messages on success. On that note, this may be a bit off-topic,
but is there documentation showing how file_roll should be used to
write to local disk?
Thanks,
Young
On Wednesday, July 11, 2012 at 11:12 AM, Young Kim wrote:
Hey Alex,
The IP is correct, but I've altered it to say localhost. As for the
syslog tests, I've ran it verbatim, but it doesn't register. However,
when I alter /tmp/foo to include two new lines, flume registers it as
a partial event.
It still seems odd to me that flume-ng isn't detecting the syslogtcp
whereas flume-og is doing it perfectly. Is it possible that there is
a bug in the way flume-ng is implementing it?
Thanks,
Young
On Tuesday, July 10, 2012 at 11:22 PM, alo alt wrote:
HI,
try these for syslog tests:
echo "<13>Jun 20 12:12:12 host foo[345]: a syslog message" > /tmp/foo
nc -v FLUME SYSLOG AGENT 5140 < /tmp/foo
syslog-agent.sources.syslog.host = 172.17.1.92
Means, will only listen only on this interface, is the IP correct?
cheers,
Alex
On Jul 10, 2012, at 10:59 PM, Young Kim wrote:
Hey guys,
I've been trying to use flume-ng (build in trunk) to read syslogs
through tcp then write to a flat file, as seen in this configuration:
# Source, sink, and channel name below
syslog-agent.channels = rmc
syslog-agent.sources = syslog
syslog-agent.sinks = flat
# Configure channels
syslog-agent.channels.rmc.type = memory
# Configure sources
syslog-agent.sources.syslog.type = syslogTcp
syslog-agent.sources.syslog.port = 5140
syslog-agent.sources.syslog.host = 172.17.1.92
syslog-agent.sources.syslog.channels = rmc
# Configure sinks
syslog-agent.sinks.flat.type = FILE_ROLL
syslog-agent.sinks.flat.directory = /Users/ykim/Desktop/test
syslog-agent.sinks.flat.rollInterval = 0
syslog-agent.sinks.flat.channels = rmc
After a bit of wrangling with rsyslog configurations, it appeared
as if rsyslog wasn't sending any packets. However, I decided to try
out flume-og with a simple dump syslogTcp, and it correctly reads
the syslog. At this point, I'm a bit lost and not sure what else to
look at. Hopefully, someone on here can shed some light on what
could be wrong.
Thanks,
Young Kim
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
