[rsyslog] two patches for imjournal

2013-05-14 Thread Tomas Heinrich
Hi, here are two bugfixes for imjournal. The first one deals with an api change in a systestemd lib, the other is from Milan Bartos and adds a PID field that was missing. Tomas From 4662184f709a9f389188c0e9635fb4da228248bd Mon Sep 17 00:00:00 2001 From: Tomas Heinrich thein...@redhat.com

Re: [rsyslog] RELP / Load Balance

2013-05-14 Thread Mark Olliver
Hi, Thank you all for your replies, so for now it probably looks best for me to stick with tcp to enable the load balancing, but it would be great it in the future there was an option to split the load between servers using REPL. What is the current max throughput expected for rsyslog?

Re: [rsyslog] two patches for imjournal

2013-05-14 Thread Rainer Gerhards
thanks, right in time :-) I guess I'll do a new stable next week. Will see that I merge later today or tomorrow morning! Rainer On Tue, May 14, 2013 at 1:04 PM, Tomas Heinrich thein...@redhat.com wrote: Hi, here are two bugfixes for imjournal. The first one deals with an api change in a

Re: [rsyslog] two patches for imjournal

2013-05-14 Thread Tomas Heinrich
On 05/14/2013 01:16 PM, Rainer Gerhards wrote: thanks, right in time :-) I guess I'll do a new stable next week. Will see that I merge later today or tomorrow morning! Rainer Thanks. Actually, I'll probably send a couple more. Some related to journal, some not. Tomas On Tue, May 14, 2013

Re: [rsyslog] RELP / Load Balance

2013-05-14 Thread David Lang
People have done 1M logs/sec I haven't had access to 10G ethernet to test with, so my tests have been limited to Gig-E wire speed (~380K log/sec with 250 byte messages) now, a more complex config may slow this down (slower output modules, lots of filter rules, regex matches, complex output

Re: [rsyslog] RELP / Load Balance

2013-05-14 Thread David Lang
On Tue, 14 May 2013, Rainer Gerhards wrote: I am not sure if it is such a good idea to do this for RELP. While of course it is possible to do (and probably not so much work), the performance profile is quite different. There is much more overhead in creating and shutting down these sessions and

Re: [rsyslog] RELP / Load Balance

2013-05-14 Thread Rainer Gerhards
On Tue, May 14, 2013 at 12:47 PM, Rainer Gerhards rgerha...@hq.adiscon.comwrote: On Mon, May 13, 2013 at 9:03 PM, David Lang da...@lang.hm wrote: On Mon, 13 May 2013, Gary Foster wrote: I believe that you are correct across the board. It should not be that difficult a task to get this

[rsyslog] Does rsyslog use ephemeral ports?

2013-05-14 Thread Boylan, James
Greetings! I'm trying to get a better idea on the number of connections a single Rsyslog server can support. I currently have it running in our pre-production environment as a smoke test for performance, but I want to check on this prior to moving it into our production environment. Does it

Re: [rsyslog] Does rsyslog use ephemeral ports?

2013-05-14 Thread David Lang
On Tue, 14 May 2013, Boylan, James wrote: Greetings! I'm trying to get a better idea on the number of connections a single Rsyslog server can support. I currently have it running in our pre-production environment as a smoke test for performance, but I want to check on this prior to moving

[rsyslog] missing configure check

2013-05-14 Thread Tomas Heinrich
When compiling with --enable-usertools and either --enable-libgcrypt (default) or --enable-guardtime , the build will fail with: GENrscryutil.1 /bin/sh: rscryutil.rst: command not found make[2]: *** [rscryutil.1] Error 127 The patch adds a check for the rst2man binary to configure.ac.

Re: [rsyslog] Does rsyslog use ephemeral ports?

2013-05-14 Thread Rainer Gerhards
If you don't need TLS, I recommend imptcp as an input. It handles even larger loads and uses less CPU (but is linux specific). Rainer On Tue, May 14, 2013 at 3:38 PM, Boylan, James james.boy...@orbitz.comwrote: I'm investigating issues with the current syslog server that is running in our

Re: [rsyslog] Does rsyslog use ephemeral ports?

2013-05-14 Thread Boylan, James
We'll definitely be using imtcp in the short term, though we are considering imrhelp in the long term. -- James -Original Message- From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards Sent: Tuesday, May 14, 2013 8:45 AM To:

Re: [rsyslog] two patches for imjournal

2013-05-14 Thread Rainer Gerhards
On Tue, May 14, 2013 at 1:44 PM, Tomas Heinrich thein...@redhat.com wrote: On 05/14/2013 01:16 PM, Rainer Gerhards wrote: thanks, right in time :-) I guess I'll do a new stable next week. Will see that I merge later today or tomorrow morning! Rainer Thanks. Actually, I'll probably send a

Re: [rsyslog] Does rsyslog use ephemeral ports?

2013-05-14 Thread Rainer Gerhards
im*p*tcp I said, not imtcp ;) On Tue, May 14, 2013 at 3:47 PM, Boylan, James james.boy...@orbitz.comwrote: We'll definitely be using imtcp in the short term, though we are considering imrhelp in the long term. -- James -Original Message- From: rsyslog-boun...@lists.adiscon.com

Re: [rsyslog] missing configure check

2013-05-14 Thread Rainer Gerhards
merged, thx Rainer On Tue, May 14, 2013 at 3:42 PM, Tomas Heinrich thein...@redhat.com wrote: When compiling with --enable-usertools and either --enable-libgcrypt (default) or --enable-guardtime , the build will fail with: GENrscryutil.1 /bin/sh: rscryutil.rst: command not found

Re: [rsyslog] RELP / Load Balance

2013-05-14 Thread Mark Olliver
Thanks for that - I will try and give it a go shortly. Regards -- Mark Olliver On 14 May 2013 at 14:08:01, Rainer Gerhards (rgerha...@hq.adiscon.com) wrote: On Tue, May 14, 2013 at 12:47 PM, Rainer Gerhards rgerha...@hq.adiscon.comwrote: On Mon, May 13, 2013 at 9:03 PM, David Lang

Re: [rsyslog] Does rsyslog use ephemeral ports?

2013-05-14 Thread David Lang
On Tue, 14 May 2013, Boylan, James wrote: I'm investigating issues with the current syslog server that is running in our environment. The information I had found was they were running into issues with the instances reaching a connection limit on the port that resulted in them running multiple

Re: [rsyslog] Does rsyslog use ephemeral ports?

2013-05-14 Thread David Lang
On Tue, 14 May 2013, Boylan, James wrote: Oh! I missed the P. I'll have to look into that. Our log servers are handling considerably more than 1mb/sec so I'm going to have to figure out a means of testing this overall performance. David was saying that he had heard of people sending 1mb/sec

Re: [rsyslog] two patches for imjournal

2013-05-14 Thread Tomas Heinrich
On 05/14/2013 03:50 PM, Rainer Gerhards wrote: On Tue, May 14, 2013 at 1:44 PM, Tomas Heinrichthein...@redhat.com wrote: On 05/14/2013 01:16 PM, Rainer Gerhards wrote: thanks, right in time :-) I guess I'll do a new stable next week. Will see that I merge later today or tomorrow morning!

Re: [rsyslog] two patches for imjournal

2013-05-14 Thread Tomas Heinrich
On 05/14/2013 05:00 PM, Rainer Gerhards wrote: OK, that's great to hear. My goal for 7.4 stable is end of month, so this aligns nicely (also I have a really good feeling about that release). While we are at it: the 7.4 branch has as an important feature the security package, which is log

Re: [rsyslog] two patches for imjournal

2013-05-14 Thread Rainer Gerhards
On Tue, May 14, 2013 at 5:39 PM, Tomas Heinrich thein...@redhat.com wrote: On 05/14/2013 05:00 PM, Rainer Gerhards wrote: OK, that's great to hear. My goal for 7.4 stable is end of month, so this aligns nicely (also I have a really good feeling about that release). While we are at it: the

[rsyslog] Action Queue cfg discarding local messages?

2013-05-14 Thread Alan Ross
Hi, I have a centos 6.3 system with rsyslogd 5.8.10 running. I have a config that forwards logs to a collector using tcp. I have action queues set-up in case the collector is not available. I previously had an issue where the queues filled up and any app using syslog slowed down. I

[rsyslog] Very high throughput options

2013-05-14 Thread Chris Bartram
We are in the planning stages of setting up a rsyslog server pool to accommodate syslog streams from a couple thousand *nix servers; including auditd type data and potentially some application logs (so it's going to be a VERY high volume of data) and we're looking to archive this data