Re: [rsyslog] Timeline for 8.18 release

[David Lang]

I believe it's due out this week.

David Lang

On Mon, 11 Apr 2016, Alec Swan wrote:


Is there any approximate date when version 8.18 is expected to go out?

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] message getting written to /var/log/message (Again! after forwarding)

[David Lang]

On Mon, 11 Apr 2016, Ashish Barmase wrote:


Rsyslog version v.5.8.10 (sorry using amazon linux)

I have below setup to monitor the log file /var/log/mongodb/mongod.conf

The message to remote graylog server goes fine, however, the same message is 
get written to /var/log/message. Not sure why, when I have a disacard option 
set (~).? Please help!!


When you use include files, the results are not always obvious. Using include 
files does the same thing as if you had done a cut-n-paste of the contents of 
those files into /etc/rsyslog.conf. There have been some versions of rsyslog 
that had a bug that included the files in the wrong order.


In this case, what's happening is that it's being written out to 
/var/log/messages before the /var/rsyslog.d/22-graylog-mongod.conf stuff gets 
invoked. Without seeing all your files and knowing if the ancient 5.8.10 version 
is one that had the reverse order bug, it's impossible to know exactly what is 
going to happen with a particular log message.


you can run rsyslog in debug mode (rsyslogd -dn) and from the output there you 
can trace exactly what happens to a particular meesage, but I hope that the info 
above gives you enough hints to be able to understand your config.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] Timeline for 8.18 release

[Alec Swan]

Is there any approximate date when version 8.18 is expected to go out?

Thanks,

Alec
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] message getting written to /var/log/message (Again! after forwarding)

[Ashish Barmase]

Hi,

Rsyslog version v.5.8.10 (sorry using amazon linux)

I have below setup to monitor the log file /var/log/mongodb/mongod.conf

The message to remote graylog server goes fine, however, the same message is 
get written to /var/log/message. Not sure why, when I have a disacard option 
set (~).? Please help!!

Contents of /var/rsyslog.d/22-graylog-mongod.conf
$ModLoad imfile
$InputFilePollInterval 10
$WorkDirectory /var/spool/rsyslog

# Input for mongod.log
$InputFileName /var/log/mongodb/mongod.log
$InputFileTag mongo
$InputFileStateFile stat-mongo
$InputFileSeverity info
$InputFilePersistStateInterval 2
$InputRunFileMonitor

# Add a tag for file events
$template GraylogMongo,"<%pri%>%protocol-version% %timestamp:::date-rfc3339% 
%HOSTNAME% %app-name% %procid% %msgid% [tag=\"mongo\"] %msg%\n"

# Send to Graylog then discard
if $programname == 'mongo' then @@graylog-s2.tloapps.com:20514;GraylogMongo
if $programname == 'mongo' then ~


the rsyslog.conf is the default one.

Thanks,
Ashish
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] dynastats problems with user defined variables?

[David Lang]

At this point I have things working. I found that in some cases I had logic 
errors where the variable being used in the update did not exist, or was 
otherwise not a valid json tag (spaces, etc)


David Lang

On Mon, 11 Apr 2016, singh.janmejay 
wrote:



Date: Mon, 11 Apr 2016 23:04:48 +0530
From: singh.janmejay 
Reply-To: rsyslog-users 
To: rsyslog-users 
Subject: Re: [rsyslog] dynastats problems with user defined variables?

David,

Did a few runs with those lines present / commented out with
rsyslog-master. I see no difference in either on-cpu or off-cpu
performance. The left-hand-side is with edge-relay and core-relay
lines being active, and the right-side has them commented out. Here is
the cpu profile:
https://drive.google.com/open?id=0B_XhUZLNFT4dWDFOZlB6VW1hc0E
The hypercall on top is cpu-idle call. This was on a c1.4large box on ec2.

Both were on the same version of Rsyslog (built off master). I'll do
another round of tests with the version I run on production (which is
patched fork of 8.12, afair) to identify any large change in cpu
profile.

This is a much more simplified version of config though. I extracted
valid (non garbled) json messages from debug log(got 9089 lines) and
pumped them in 100 times using tcpflood.

tcpflood call was timed, it took between 19.7 - 20.5 seconds for each
run regardless of edge and core relay counters being incremented. It
was ~ 45k / sec (but again, neither the load profile, not the machine
is not exactly like yours), so its not apples to apples.


On Fri, Apr 8, 2016 at 12:15 PM, singh.janmejay
 wrote:

It doesn't treat them any different. It takes the key and puts it in a
hash-table and set an accumulator as its value. Then it'll look it up
every time you ask it to increment the counter followed by actual
increment. That is about it.

The only way I can foresee the contents affecting its behavior is the
hash-fn takes more CPU time when given a large string. But that should
be on-cpu and not off-cpu time.

Today im working on a getting an Rsyslog build running with config
into which I inject messages from your debug-log (i'll extract
messages and tcpflood them in) and try to understand why it behaves
different between those lines commented out or not.

If I fail to reproduce it in this simple setup, we'll have to track
backtraces that are taking Rsyslog off-cpu in your environment.

On Fri, Mar 25, 2016 at 12:57 AM, David Lang  wrote:

doing a little more digging (and some accidental stuff), is it possible that
it's running into grief if the contents of the variable are not a simple
word (spaces or other funny characters in the value)?

David Lang

On Wed, 23 Mar 2016, singh.janmejay wrote:


Date: Wed, 23 Mar 2016 01:07:49 +0530

From: singh.janmejay 
Reply-To: rsyslog-users 
To: rsyslog-users 
Subject: Re: [rsyslog] dynastats problems with user defined variables?

Yep, it looked like that. Its interesting though, its 100% off CPU
(very unique).

On Wed, Mar 23, 2016 at 1:02 AM, David Lang  wrote:


Yes, the first one is with the user variables commented out, exactly what
I
pasted in from the grep.

the second is enabling the dyn_inc for edge relays.

David Lang

On Wed, 23 Mar 2016, singh.janmejay wrote:


Date: Wed, 23 Mar 2016 00:51:18 +0530
From: singh.janmejay 
Reply-To: rsyslog-users 
To: rsyslog-users 
Subject: Re: [rsyslog] dynastats problems with user defined variables?

You meant the first one was with "commented out" version right? (you
said uncommenting edge_relay, so double-checking).

On Wed, Mar 23, 2016 at 12:38 AM, David Lang  wrote:



here is vmstat 1 running

# vmstat 1
procs ---memory-- ---swap-- -io -system--
--cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy
id
wa st
 2  0  62280 2363804584 272803760121   59601 46
3
51  0 0
 1  0  62280 2349476584 2728061200 0 87042  979  891  4
0
95  0 0
 1  0  62280 2330488584 2728090800 0 0  388  258  4
0
96  0 0
 1  0  62280 2314744584 2728114800 0 0  396  255  4
0
96  0 0
 1  0  62280 2297736584 2728138800 0 0  376  245  4
0
96  0 0
 2  0  62280 2546260584 2724527200 0 4  639  568  6
1
94  0 0
 2  0  62280 2464300584 2724592800 0  1716  936 1146  8
0
91  0 0
 2  0  62280 2394452584 2724644400 0 8  687  333  8
0
92  0 0


here is vmstat 1 uncommenting edge_relay

# vmstat 1
procs ---memory-- ---swap-- -io -system--
--cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy
id
wa st
 1  0  62280 3198672584 2677062401   

Re: [rsyslog] lognorm mailing list

[David Lang]

there are people listening and responding there. It's not very active.

David Lang

On Mon, 11 Apr 2016, Adam Williams wrote:


Date: Mon, 11 Apr 2016 12:37:56 -0400
From: Adam Williams 
Reply-To: rsyslog-users 
To: rsyslog@lists.adiscon.com
Subject: [rsyslog] lognorm mailing list

Hello,

Does anyone know if the lognorm mailing list has anyone listening? I
see there are no archives since 2015 sometime. I've got a couple of
questions about lognorm there and want to be sure I'm not posting to
the wrong place.

Thanks!

Adam
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] rsync mirror

[David Lang]

On Mon, 11 Apr 2016, Giulio Vaccari wrote:


First of all, thanks to all that answer to my question in these days
(specially David Lang), I finally reach to create a system that works
quite well.

Now I want implement this system in all my machine but some of them are
had an old version of rsyslog so I must update it.
Unfortunately I can't do that using an external repository because the
company rule strictly forbid this.

So my question is, can I create an internal mirror of your repository
(http://rpms.adiscon.com/) using rsync?


I'm not sure if rsync is allowed to that destination, but there are many 
programs out there that will mirror a portion of a website.


Creating an internal repo is FAR better than updating directly from the 
Internet. You don't want your systems updating just because someone 'out there' 
decided to push an update, and you sure don't want all your systems to update 
with no warning.


The normal documentation on setting up an internal repo for your distro should 
work just fine.


David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] dynastats problems with user defined variables?

[singh.janmejay]

David,

Did a few runs with those lines present / commented out with
rsyslog-master. I see no difference in either on-cpu or off-cpu
performance. The left-hand-side is with edge-relay and core-relay
lines being active, and the right-side has them commented out. Here is
the cpu profile:
https://drive.google.com/open?id=0B_XhUZLNFT4dWDFOZlB6VW1hc0E
The hypercall on top is cpu-idle call. This was on a c1.4large box on ec2.

Both were on the same version of Rsyslog (built off master). I'll do
another round of tests with the version I run on production (which is
patched fork of 8.12, afair) to identify any large change in cpu
profile.

This is a much more simplified version of config though. I extracted
valid (non garbled) json messages from debug log(got 9089 lines) and
pumped them in 100 times using tcpflood.

tcpflood call was timed, it took between 19.7 - 20.5 seconds for each
run regardless of edge and core relay counters being incremented. It
was ~ 45k / sec (but again, neither the load profile, not the machine
is not exactly like yours), so its not apples to apples.


On Fri, Apr 8, 2016 at 12:15 PM, singh.janmejay
 wrote:
> It doesn't treat them any different. It takes the key and puts it in a
> hash-table and set an accumulator as its value. Then it'll look it up
> every time you ask it to increment the counter followed by actual
> increment. That is about it.
>
> The only way I can foresee the contents affecting its behavior is the
> hash-fn takes more CPU time when given a large string. But that should
> be on-cpu and not off-cpu time.
>
> Today im working on a getting an Rsyslog build running with config
> into which I inject messages from your debug-log (i'll extract
> messages and tcpflood them in) and try to understand why it behaves
> different between those lines commented out or not.
>
> If I fail to reproduce it in this simple setup, we'll have to track
> backtraces that are taking Rsyslog off-cpu in your environment.
>
> On Fri, Mar 25, 2016 at 12:57 AM, David Lang  wrote:
>> doing a little more digging (and some accidental stuff), is it possible that
>> it's running into grief if the contents of the variable are not a simple
>> word (spaces or other funny characters in the value)?
>>
>> David Lang
>>
>> On Wed, 23 Mar 2016, singh.janmejay wrote:
>>
>>> Date: Wed, 23 Mar 2016 01:07:49 +0530
>>>
>>> From: singh.janmejay 
>>> Reply-To: rsyslog-users 
>>> To: rsyslog-users 
>>> Subject: Re: [rsyslog] dynastats problems with user defined variables?
>>>
>>> Yep, it looked like that. Its interesting though, its 100% off CPU
>>> (very unique).
>>>
>>> On Wed, Mar 23, 2016 at 1:02 AM, David Lang  wrote:

 Yes, the first one is with the user variables commented out, exactly what
 I
 pasted in from the grep.

 the second is enabling the dyn_inc for edge relays.

 David Lang

 On Wed, 23 Mar 2016, singh.janmejay wrote:

> Date: Wed, 23 Mar 2016 00:51:18 +0530
> From: singh.janmejay 
> Reply-To: rsyslog-users 
> To: rsyslog-users 
> Subject: Re: [rsyslog] dynastats problems with user defined variables?
>
> You meant the first one was with "commented out" version right? (you
> said uncommenting edge_relay, so double-checking).
>
> On Wed, Mar 23, 2016 at 12:38 AM, David Lang  wrote:
>>
>>
>> here is vmstat 1 running
>>
>> # vmstat 1
>> procs ---memory-- ---swap-- -io -system--
>> --cpu-
>>  r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy
>> id
>> wa st
>>  2  0  62280 2363804584 272803760121   59601 46
>> 3
>> 51  0 0
>>  1  0  62280 2349476584 2728061200 0 87042  979  891  4
>> 0
>> 95  0 0
>>  1  0  62280 2330488584 2728090800 0 0  388  258  4
>> 0
>> 96  0 0
>>  1  0  62280 2314744584 2728114800 0 0  396  255  4
>> 0
>> 96  0 0
>>  1  0  62280 2297736584 2728138800 0 0  376  245  4
>> 0
>> 96  0 0
>>  2  0  62280 2546260584 2724527200 0 4  639  568  6
>> 1
>> 94  0 0
>>  2  0  62280 2464300584 2724592800 0  1716  936 1146  8
>> 0
>> 91  0 0
>>  2  0  62280 2394452584 2724644400 0 8  687  333  8
>> 0
>> 92  0 0
>>
>>
>> here is vmstat 1 uncommenting edge_relay
>>
>> # vmstat 1
>> procs ---memory-- ---swap-- -io -system--
>> --cpu-
>>  r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy
>> id
>> wa st
>>  1  0  62280 3198672584 267706240121   

[rsyslog] lognorm mailing list

[Adam Williams]

Hello,

Does anyone know if the lognorm mailing list has anyone listening? I
see there are no archives since 2015 sometime. I've got a couple of
questions about lognorm there and want to be sure I'm not posting to
the wrong place.

Thanks!

Adam
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] rsync mirror

[Giulio Vaccari]

First of all, thanks to all that answer to my question in these days
(specially David Lang), I finally reach to create a system that works
quite well.

Now I want implement this system in all my machine but some of them are
had an old version of rsyslog so I must update it.
Unfortunately I can't do that using an external repository because the
company rule strictly forbid this.

So my question is, can I create an internal mirror of your repository 
(http://rpms.adiscon.com/) using rsync?

thank you
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Certificate usage

[Thomas D.]

No, rsyslog currently only do basic certificate verification using
GnuTLS' gnutls_certificate_verify_peers2 function. In other words it is
checked that the presented certificate is valid and matches up to your
specified CA trust anchor.

No OIDs (purpose) will be checked.

If you are currently using OIDs to limit hosts/apply some kind of ACLs
please have a look at rsyslog's peer list feature which allows you to
specify hostnames and/or fingerprints to control which hosts are allowed
to send logs. See $ActionSendStreamDriverPermittedPeer or
StreamDriverPermittedPeers parameter when using nsd (omfwd) or
tls.permittedPeer parameter when using imrelp.


-Thomas

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.