The queue errors are bad. Anything else in regard to that queue?
The thread messages are informational and part of the better status
indicators. They say that the load was so high an additional worker thread
was started... And later shut down as it was no longer needed.
Rainer
Sent from phone, thus brief.
Am 18.10.2017 22:15 schrieb "David Lang" <da...@lang.hm>:
On Wed, 18 Oct 2017, deoren wrote:
On 10/18/2017 1:36 PM, David Lang wrote:
>
>> On Wed, 18 Oct 2017, deoren wrote:
>>
>>> Since the sender and receiver in this are both the latest versions of
>>> rsyslog (with the plan for the setup to remain that way), can I scale the
>>> accepted message size values to properly accommodate non-standard message
>>> sizes (delivered via JSON payloads)?
>>>
>>
>> up to 128K should not be a problem, I believe that to scale the message
>> size >128K you need to change a setting in the source.
>>
>> The particular box I'm having trouble with runs a Redmine instance that
>>> is heavily used and has lots of activity within its production log and
>>> development log files (large entries). I'd like to deliver entire log
>>> entries instead of truncated versions if possible.
>>>
>>
>> how large are the lines?
>>
>
> At present, the development log file has a line with a length of 2997
> characters and the production.log file has a line with a length of 8608
> characters.
>
those are quite reasonable
Yesterday I muddied the waters a bit when I opted to test using a regex
> with imfile to treat a large batch of content between a marker as part of a
> single message. I just picked two samples and found that one was 25K and
> the other was 97K.
>
that's starting to get large enough to worry.
I should note that I have this setting enabled on all of our Ubuntu boxes:
>
> maxMessageSize="64k"
>
bump this up if you are going to be using the large size.
I forget which article I found that mentioned it, but the idea was that
> JSON message content is larger than standard syslog messages and the
> increased max message size would help accommodate that.
>
yes, json is more verbose than an unformatted text log, it also depends on
if you duplicate data in the json version (do you have both the plain text
message and the data parsed from it?)
one thing you can do is to have your senders check the size of the log (set
$.x = exec_template("template"); test for lenght($.x)), and if the message
is too large, write it locally to a 'oversized' file and then either figure
a way to trim the message before it's sent, or send a placeholder saying
that you had a message that was too large to send, so you are storing it
locally.
It could be completely unrelated, but I found these messages when I checked
> back on the /var/log/rsyslog.log file:
>
> 2017-10-18T13:50:15.134882-05:00 redminebox rsyslogd: main Q:Reg: high
> activity - starting 1 additional worker thread(s), currently 1 active
> worker threads. [v8.30.0 try http://www.rsyslog.com/e/2439 ]
> 2017-10-18T13:50:15.446207-05:00 redminebox rsyslogd: ForwardToSawmill1
> queue[DA]: qDeqDisk error happened at around offset 4152 [v8.30.0 try
> http://www.rsyslog.com/e/2059 ]
> 2017-10-18T13:50:15.446464-05:00 redminebox rsyslogd: ForwardToSawmill1
> queue[DA]: error dequeueing element - ignoring, but strange things may
> happen [v8.30.0 try http://www.rsyslog.com/e/2059 ]
> 2017-10-18T13:50:15.525897-05:00 redminebox rsyslogd: ForwardToSawmill1
> queue[DA]: qDeqDisk error happened at around offset 12289 [v8.30.0 try
> http://www.rsyslog.com/e/2059 ]
> 2017-10-18T13:50:15.526146-05:00 redminebox rsyslogd: ForwardToSawmill1
> queue[DA]: error dequeueing element - ignoring, but strange things may
> happen [v8.30.0 try http://www.rsyslog.com/e/2059 ]
> 2017-10-18T13:51:15.847344-05:00 redminebox rsyslogd: main Q:Reg: worker
> thread 1a8d180 terminated, now 1 active worker threads [v8.30.0 try
> http://www.rsyslog.com/e/2439 ]
>
> This is the first time I've seen those log entries.
>
these sound significant, but Rainer will need to comment on them.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
