On Wed, 3 Jun 2015, Muhammad Asif wrote:
@David Lang
We are sending these logs to fluent (Testing fluent to replace logstash) to
parse logs.
by the way, reading the Fluent website a bit, it's design seems rather similar
to rsyslog. Input modules, use and support of JSON, output modules (and in
addition, rsyslog has message modification modules). I'll note that where Fluent
says in their faq that one box can do 18K logs/sec, rsyslog routinely pushes
100K+ logs/sec.
Now, it may be that the pre-written modules give you some things that would be
more work to setup in rsyslog. If this is the case, we'd be interested in
learning about the use cases.
I haven't yet been able to find out the preferred protocol for delivering
messages to Fluent. It implies some form of JSON, but that's not really clear
enough. You can have multiple JSON messages being delivered that use different
names to mean the same thing (i.e. the time the log was generated and the
host/IP that generated the log), unless the correct names are known for each
message (or better still, a set of names is defined as being authoritive and the
incoming logs required to conform to this set), you won't make sense of te logs
and deliver them correctly.
David Lang
Is there any good webinar on rsyslog parsing and template. There should be
a series of about 5 webinars on rsyslog. Rsyslog is lacking on youtube.
*.* @127.0.0.1:5120;msgonly
On Wed, Jun 3, 2015 at 3:58 AM, David Lang <[email protected]> wrote:
On Tue, 2 Jun 2015, Otis Gospodnetic wrote:
To answer your question about rsyslog in place of Logstash. Short answer:
yes.
See
http://blog.sematext.com/2015/05/18/tuning-elasticsearch-indexing-pipeline-for-logs/
for some info about rsyslog and Logstash comparison when it comes to
performance and footprint.
a question about this link. Why is it showing logstash performance getting
better as it starts doing more complicated things?
David Lang
This may also be of interest to you:
http://blog.sematext.com/2015/04/13/monitoring-rsyslogs-performance-with-impstats-and-elasticsearch/
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
On Tue, Jun 2, 2015 at 8:33 AM, Muhammad Asif <[email protected]>
wrote:
Is there any default queue mechanism in rsyslog. I did not configure any
main and action queue but when i checked imstat statistics. It shows the
following output.
Tue Jun 2 17:30:17 2015: main Q: size=0 enqueued=60933 full=0
discarded.full=0 discarded.nf=0 maxqsize=409
Tue Jun 2 17:31:17 2015: imuxsock: submitted=4 ratelimit.discarded=0
ratelimit.numratelimiters=3
Tue Jun 2 17:31:17 2015: action 1: processed=60933 failed=0
Tue Jun 2 17:31:17 2015: action 2: processed=0 failed=0
Tue Jun 2 17:31:17 2015: action 3: processed=20 failed=0
Tue Jun 2 17:31:17 2015: action 4: processed=60913 failed=0
Tue Jun 2 17:31:17 2015: action 5: processed=0 failed=0
Tue Jun 2 17:31:17 2015: action 6: processed=0 failed=0
Tue Jun 2 17:31:17 2015: action 7: processed=0 failed=0
Tue Jun 2 17:31:17 2015: action 8: processed=0 failed=0
Tue Jun 2 17:31:17 2015: action 9: processed=0 failed=0
Tue Jun 2 17:31:17 2015: action 10: processed=0 failed=0
Tue Jun 2 17:31:17 2015: action 11: processed=0 failed=0
*Tue Jun 2 17:31:17 2015: action 12: processed=60932 failed=60932*
Tue Jun 2 17:31:17 2015: action 13: processed=60933 failed=0
Tue Jun 2 17:31:17 2015: imudp(*:514): submitted=0
Tue Jun 2 17:31:17 2015: imudp(*:514): submitted=0
Tue Jun 2 17:31:17 2015: imtcp(514): submitted=0
Tue Jun 2 17:31:17 2015: main Q: size=0 enqueued=60933 full=0
discarded.full=0 discarded.nf=0 maxqsize=409
Why it is showing as many failed messages as many processed.
Can we use rsyslog in place of logstash.
Thanks
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
