Re: [rsyslog] If a ruleset is bound to a specific input, will rsyslog check a message arriving on that input against a different ruleset?

2017-07-10 Thread mostolog--- via rsyslog
It's implicit, AFAIK On 11/07/17 07:48, deoren wrote: Here is some pseducode based off of another recent thread: ruleset(name="remote-rules"){ action( ... ) action( ... ) stop } input(type="imudp" port="1514" address="127

[rsyslog] If a ruleset is bound to a specific input, will rsyslog check a message arriving on that input against a different ruleset?

2017-07-10 Thread deoren
Here is some pseducode based off of another recent thread: ruleset(name="remote-rules"){ action( ... ) action( ... ) stop } input(type="imudp" port="1514" address="127.0.0.1" ruleset="remote-rules") input(type="imptcp" port=

Re: [rsyslog] Split messages options

2017-07-10 Thread Scot Kreienkamp
>From my config text, I'm not writing a disk queue so that won't be a problem, >and my network queues I'm assuming are single threaded by default so that >shouldn't be a problem either. ruleset(name="RMS-Ecomm-1514-1531"){ action( name="omfile-Ecomm.log" t

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread deoren
On 7/10/17 6:58 PM, David Lang wrote: On Mon, 10 Jul 2017, deoren wrote: Without an override in place, the permissions are being reset on boot for the /var/log directory to 0755, ownership of root:syslog. The /usr/lib/tmpfiles.d/00rsyslog.conf file overrides the systemd default configuration

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread David Lang
On Mon, 10 Jul 2017, deoren wrote: Without an override in place, the permissions are being reset on boot for the /var/log directory to 0755, ownership of root:syslog. The /usr/lib/tmpfiles.d/00rsyslog.conf file overrides the systemd default configuration to set 0775 on boot. that seems like

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread deoren
On 7/10/17 5:28 PM, David Lang wrote: On Mon, 10 Jul 2017, deoren wrote: Hi David, My comment wasn't as clear as it should have been. The /usr/lib/tmpfiles.d/var.conf file targets specific directories and one of those is /var/log. By default /var/log is set to 0755, so if rsyslog is to gene

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread David Lang
On Mon, 10 Jul 2017, deoren wrote: On 7/10/17 5:13 PM, David Lang wrote: On Mon, 10 Jul 2017, deoren wrote: I checked Debian 9 and that release appears to be handling the rsyslog configuration the same way as CentOS 7: run as root, do not attempt to drop privileges. Ubuntu's rsyslog package

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread deoren
On 7/10/17 5:13 PM, David Lang wrote: On Mon, 10 Jul 2017, deoren wrote: I checked Debian 9 and that release appears to be handling the rsyslog configuration the same way as CentOS 7: run as root, do not attempt to drop privileges. Ubuntu's rsyslog package on the other hand does make the atte

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread David Lang
On Mon, 10 Jul 2017, deoren wrote: I checked Debian 9 and that release appears to be handling the rsyslog configuration the same way as CentOS 7: run as root, do not attempt to drop privileges. Ubuntu's rsyslog package on the other hand does make the attempt, as does the package provided by th

Re: [rsyslog] Any estimate re availability of the 8.28.0 version via the Ubuntu PPA?

2017-07-10 Thread deoren
On 7/10/17 3:00 PM, David Lang wrote: no, there are sometimes delays between the release and the PPA, but they are not the result of a deliberate policy I'm a new user of the PPA, so I wasn't sure what to expect re release timeline. Thanks for confirming that it's variable. __

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread deoren
On 7/10/17 3:07 PM, David Lang wrote: > I've been using rsyslog since 2006 and never seen /usr/lib/tempfiles.d From what I've learned it's systemd specific and really came into play with Ubuntu 15.04, but I only learned about with recent system upgrades from 14.04 to 16.04 LTS. > I don't thi

Re: [rsyslog] Split messages options

2017-07-10 Thread David Lang
If you have multiple threads working to process messages, thread 1 will grab messages 1-10 and start processing them, thread 2 will grab messages 11-20 and start processing them in parallel, so ordering will be lost. avoid using multiple threads when processing them, and you avoid that problem.

Re: [rsyslog] Split messages options

2017-07-10 Thread Scot Kreienkamp
Yep, understood on the ordering. Log4j is sending to rsyslog@localhost via UDP, rsyslog is relaying via TCP. For my usage, vast majority of the time is just fine. I would have assumed that rsyslog would attempt processing of messages from any queue in the order received though... No? Scot

Re: [rsyslog] Split messages options

2017-07-10 Thread David Lang
note that there is some potential for messages to get out of order (over the network with UDP, and inside rsyslog if you use disk queues or multiple threads) UDP messages can be dropped if the network is busy as well (by any router/firewall or receiving host) but the vast majority of the time

Re: [rsyslog] Split messages options

2017-07-10 Thread Scot Kreienkamp
That makes sense, I wasn't aware of the limitation in log4j. And yes, it is a very old implementation and getting them to update would likely take years. :-) In my case I'm transporting the logfile from the prod servers to a common collector server internally that the devs can have access to;

Re: [rsyslog] Split messages options

2017-07-10 Thread David Lang
ahh, if this is a very old log4j, it will refuse to send UDP messages >1K in size, so it splits things before they get to rsyslog, and your maxmessagesize isn't going to help. normally I am not in favor of writing to disk and then reading them, but if you are stuck with an old log4j, that may

Re: [rsyslog] Split messages options

2017-07-10 Thread Scot Kreienkamp
Hi David, I'm not bringing them in with imfile, log4j is using the builtin syslog appender to submit it directly to syslog over UDP 514, the default UDP listener port. I was assuming rsyslog is doing the splitting of the messages due to size. Scot Kreienkamp | Senior Systems Engineer | La

Re: [rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread David Lang
I've been using rsyslog since 2006 and never seen /usr/lib/tempfiles.d I don't think any adiscon created package has ever had it. That looks like something that is unique to the ubuntu packaging (it's not in the adiscon packages or in the debian packages) David Lang On Mon, 10 Jul 2017, deo

Re: [rsyslog] Any estimate re availability of the 8.28.0 version via the Ubuntu PPA?

2017-07-10 Thread David Lang
no, there are sometimes delays between the release and the PPA, but they are not the result of a deliberate policy David Lang On Mon, 10 Jul 2017, deoren wrote: Date: Mon, 10 Jul 2017 12:23:33 -0500 From: deoren Reply-To: rsyslog-users To: rsyslog@lists.adiscon.com Subject: [rsyslog] Any es

Re: [rsyslog] Split messages options

2017-07-10 Thread David Lang
you don't show us your imfile config, check to see if the java program is outputting multi-line messages (and if so, are you handling them correctly) with current rsyslo versions, I have set the maxmessagesize larger than 64k unfortunantly, rsyslog processes each message it sees independently,

Re: [rsyslog] Log messages held when using FQDN as omrelp target and (/etc/hosts entry not present or 'After=network.target' not present in Unit file)

2017-07-10 Thread deoren
On 7/8/17 10:19 PM, deoren wrote: Running the same command on the SSD copy of that VM I see about 220ms startup time. I'm also new to systemd, so I might be misinterpreting the values, but it appears that the slower load time for rsyslog is giving the system sufficient time to load all require

[rsyslog] Ubuntu >= 16.04: Upgrading stock rsyslog package via PPA results in /usr/lib/tmpfiles.d/00rsyslog.conf file being removed

2017-07-10 Thread deoren
While researching permission issues on /var/log, I realized that the rsyslog package from the Ubuntu PPA doesn't provide an override for /usr/lib/tmpfiles.d/var.conf (which has a rule that sets /var/log to 0755). I submitted #1655 via GitHub for that, thinking that the problem was likely limite

[rsyslog] Weird disk caching that is only dumping on full system crash

2017-07-10 Thread David Turner via rsyslog
Installed Packages rsyslog.x86_64 7.4.7-12.el7 @OS $ cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) $ uname -r 3.10.0-327.36.3.el7.x86_64 On June 16th I upgraded Ceph on a couple Ceph clusters and due to some stale crons tha

[rsyslog] Any estimate re availability of the 8.28.0 version via the Ubuntu PPA?

2017-07-10 Thread deoren
Is the policy to wait a bit between releasing a new stable release and then building it for the PPA to make sure there are no issues with the new release? https://launchpad.net/~adiscon/+archive/ubuntu/v8-stable Thanks. ___ rsyslog mailing list http:

[rsyslog] Split messages options

2017-07-10 Thread Scot Kreienkamp
Hi everyone, I have a java program that is using log4j to write into rsyslog. The messages are so large they are getting split, which causes the sorting rule (if $syslogtag == 'RMS-Tomcat:' then) to write the first part to the correct file, but the second part goes into messages because it has