sorry to bump this old one but i saw something interesting with dashes, when i send a syslog message like the following:
*"my string - with - dashes inside - of it"* flume output is like *"- dashes inside - of it"* in the SyslogUDPSource class, when the event is extracted it loses some part of the message.. Any ideas? Regards On Wed, Jul 25, 2012 at 12:50 PM, mete <[email protected]> wrote: > Hello Hari, > > I tried to correlate two logs and here is a sample event: > > *Http Connection Event On Syslog NG:* > > SSG550: NetScreen device_id=SSG550 > [Root]system-notification-00257(traffic): start_time="2012-07-11 19:06:51" > duration=0 policy_id=10 service=http proto=6 src zone=Trust dst zone=Trust > action=Permit sent=0 rcvd=0 src=IP1 dst=IP2 src_port=57829 dst_port=80 > src-xlated ip=IP1 port=57829 dst-xlated ip=IP2 port=80 session_id=254877 * > reason=Creation* > > SSG550: NetScreen device_id=SSG550 > [Root]system-notification-00257(traffic): start_time="2012-07-11 19:06:51" > duration=14 policy_id=10 service=http proto=6 src zone=Trust dst zone=Trust > action=Permit sent=2402 rcvd=8364 src=IP1 dst=IP2 src_port=57829 > dst_port=80 src-xlated ip=IP1 port=57829 dst-xlated ip=IP2 port=80 > session_id=254877 *reason=Close - TCP FIN* > > > > *Same Event on Flume:* > > SSG550: NetScreen device_id=SSG550 > [Root]system-notification-00257(traffic): start_time="2012-07-11 19:06:51" > duration=0 policy_id=10 service=http proto=6 src zone=Trust dst zone=Trust > action=Permit sent=0 rcvd=0 src=IP1 dst=IP2 src_port=57829 dst_port=80 > src-xlated ip=IP1 port=57829 dst-xlated ip=IP2 port=80 session_id=254877 * > reason=Creation^@* > > FIN^@ > > In general, on the flume side, i cannot see any close,age out,finish logs > properly. For this event, i cannot find an end event, and i assumed one of > the FIN message belongs to that. For the end events i see lines like: > > FIN^@ > FIN^@ > Unreach^@ > FIN^@ > OUT^@ > OUT^@ > FIN^@ > RST^@ > > Any ideas? > Thanks in advance. > > Mete > > > > > > > On Wed, Jul 25, 2012 at 10:55 AM, Hari Shreedharan < > [email protected]> wrote: > >> It would be helpful if you could send the original messages as well. >> >> Thanks >> Hari >> >> -- >> Hari Shreedharan >> >> On Wednesday, July 25, 2012 at 12:49 AM, mete wrote: >> >> Hello folks, >> >> I am using flume-ng for cdh4 (1.10), and i am redirecting syslog output >> from a network device to flume-ng. My config is as follows: >> test1.channels.mem-chan-1.type = memory >> test1.channels.mem-chan-1.capacity = 100000 >> test1.channels.mem-chan-1.transactionCapacity = 1000 >> >> test1.sources.syslog-traffic.channels = mem-chan-1 >> test1.sources.syslog-traffic.type = syslogudp >> test1.sources.syslog-traffic.port = 5140 >> test1.sources.syslog-traffic.bind = test1 >> test1.sources.syslog-traffic.eventSize = 10000 >> >> test1.sinks.file-sink-1.channel = mem-chan-1 >> test1.sinks.file-sink-1.type = file_roll >> test1.sinks.file-sink-1.sink.directory = /home/cloudera-user/tmp/ >> test1.sinks.file-sink-1.rollInterval = 86400 >> >> test1.channels = mem-chan-1 >> test1.sources = syslog-traffic >> test1.sinks = file-sink-1 >> >> i have a pretty straightforward config with one syslogudp source , a >> memory channel and a file sink. >> >> However, some of the messages i see on the file is like this: >> >> DEVICE: "some syslog content"@ >> DEVICE: "some syslog content"@ >> OUT^@ >> FIN^@ >> RST^@ >> RST^@ >> OUT^@ >> FIN^@ >> RST^@ >> FIN^@ >> FIN^@ >> OUT^@ >> RST^@ >> RST^@ >> RST^@ >> >> As you can see, some lines are somehow trimmed and does not contain the >> entire message. When i redirect same device to syslog-ng there are no >> issues like this. >> I tried increasing the event size on the syslog source but that did not >> change anything at all. >> Any ideas on what might be the problem? >> Thanks in advance. >> >> Mete >> >> >> >
