On Thu, 9 Jan 2014, DuyLong Le wrote:

Hi,


Hi,

I didn't get this problem but I've tried with 7.4 or something.

Can you reproduce this problem in isolation (i.e. except for your
production machines)? If you do, then you can:
a) easily check if the bug was fixed in a later version
b) help somebody else reproduce the problem and fix it or maybe even a
workaround for your version.

Very strange, I'm testing with Rsyslog 7.4.8 and my problem is always
present.

setup the impstats module, it will give you far more information about
what's happening.

I tested with impstats, my output :

RabbitMQ logs :

## Start
=INFO REPORT==== 9-Jan-2014::16:41:12 ===
accepting AMQP connection <0.29745.207> (X.X.X.X:33400 -> X.X.X.X:5672)

=ERROR REPORT==== 9-Jan-2014::16:41:12 ===
error on AMQP connection <0.29745.207>: enotconn (socket is not connected)

=INFO REPORT==== 9-Jan-2014::16:41:38 ===
accepting AMQP connection <0.29904.207> (X.X.X.X:33452 -> X.X.X.X:5672)

=WARNING REPORT==== 9-Jan-2014::16:41:38 ===
closing AMQP connection <0.29904.207> (X.X.X.X:33452 -> X.X.X.X:5672):
connection_closed_abruptly

=INFO REPORT==== 9-Jan-2014::16:41:52 ===
accepting AMQP connection <0.29990.207> (X.X.X.X:52565 -> X.X.X.X:5671)
...
## End

Rsyslog impstats :

## End
...
rabbitmq REPORT==== 9-Jan-2014::16:41:52 ===#012accepting AMQP connection <0.29990.207> (X.X.X.X:52565 -> X.X.X.X:5671) rsyslogd-pstats main Q: size=2 enqueued=334 full=0 discarded.full=0 discarded.nf=0 maxqsize=193
rsyslogd-pstats action 1: processed=333 failed=0
rsyslogd-pstats imuxsock: submitted=0 ratelimit.discarded=0 ratelimit.numratelimiters=0 rabbitmq REPORT==== 9-Jan-2014::16:41:52 ===#012accepting AMQP connection <0.29990.207> (X.X.X.X:52565 -> X.X.X.X:5671) rsyslogd-pstats main Q: size=2 enqueued=334 full=0 discarded.full=0 discarded.nf=0 maxqsize=193
rsyslogd-pstats action 1: processed=333 failed=0
rsyslogd-pstats imuxsock: submitted=0 ratelimit.discarded=0 ratelimit.numratelimiters=0 rabbitmq 9-Jan-2014::16:41:38 ===#012accepting AMQP connection <0.29904.207> (X.X.X.X:33452 -> X.X.X.X:5672) rsyslogd-pstats main Q: size=2 enqueued=330 full=0 discarded.full=0 discarded.nf=0 maxqsize=193 rsyslogd-pstats imuxsock: submitted=0 ratelimit.discarded=0 ratelimit.numratelimiters=0
rsyslogd-pstats action 1: processed=329 failed=0
rsyslogd-pstats imuxsock: submitted=0 ratelimit.discarded=0 ratelimit.numratelimiters=0
rsyslogd-pstats action 1: processed=326 failed=0
rsyslogd-pstats main Q: size=2 enqueued=327 full=0 discarded.full=0 discarded.nf=0 maxqsize=193
rsyslogd-pstats action 1: processed=323 failed=0
rsyslogd-pstats main Q: size=2 enqueued=324 full=0 discarded.full=0 discarded.nf=0 maxqsize=193 rsyslogd-pstats imuxsock: submitted=0 ratelimit.discarded=0 ratelimit.numratelimiters=0 rabbitmq =INFO REPORT==== 9-Jan-2014::16:41:12 ===#012accepting AMQP connection <0.29745.207> (X.X.X.X:33400 -> X.X.X.X:5672)
## Start

How to understand this output?

Dle


David Lang

Examples for a period of 1 minute :

RabbitMQ log :

###
=ERROR REPORT==== 9-Jan-2014::10:00:03 ===
error on AMQP connection <0.13019.203>: {ssl_upgrade_error,closed}
(unknown POSIX error)

=INFO REPORT==== 9-Jan-2014::10:00:16 ===
accepting AMQP connection <0.13132.203> (X.X.X.X:60843 -> X.X.X.X:5671)

=INFO REPORT==== 9-Jan-2014::10:00:18 ===
accepting AMQP connection <0.13145.203> (X.X.X.X:60847 -> X.X.X.X:5671)

=ERROR REPORT==== 9-Jan-2014::10:00:21 ===
error on AMQP connection <0.13132.203>: {ssl_upgrade_error,closed}
(unknown POSIX error)

=ERROR REPORT==== 9-Jan-2014::10:00:23 ===
error on AMQP connection <0.13145.203>: {ssl_upgrade_error,closed}
(unknown POSIX error)
###

RSyslog output :

###
=ERROR REPORT==== 9-Jan-2014::10:00:21 ===#012error on AMQP connection
<0.13132.203>: {ssl_upgrade_error,closed} (unknown POSIX error)
=INFO REPORT==== 9-Jan-2014::10:00:16 ===#012accepting AMQP connection
<0.13132.203> (X.X.X.X:60843 -> X.X.X.X:5671)
###

tcpdump gives the same result that rsyslog. It seems Rsyslog drops
some multiline...

With valgrant I saw some "deleted", is this normal?
###
5245.516127514:9692700: stream checking for file change on
'/var/log/rabbitmq/[email protected]', inode 23229/23229
5245.516272268:7689700: we deleted 3 objects and enqueued 0 objects
5245.516353536:7689700: delete batch from store, new sizes: log 0, phys 0
5245.516436979:7689700: regular consumer finished, iret=4, szlog 0 sz
phys 0
5245.516536782:7689700: main Q:Reg/w0: worker IDLE, waiting for work.
###

Best regards,
Dle


Best regards,
Radu


2014/1/7 DuyLong Le <[email protected]
<mailto:[email protected]>>

    Hi,

    I use "ReadMode" option in order to parse RabbitMQ logs.

    I have a problem because I lose a lot of messages. All multiline
    events are not forwarding and tcpdump can confirm it.

    My configuration is very basic :

    ###
    $ModLoad imfile.so

    $InputFileName /var/log/rabbitmq/rabbit@plop.__log
    $InputFileTag rabbitmq:
    $InputFileStateFile stat-rabbitmq
    $InputFileSeverity info
    $InputFileFacility local1
    $InputFileReadMode 1
    $InputRunFileMonitor

    *.*  @@(o)forward_host;RSYSLOG___ForwardFormat
    ###

    OS : Redhat 6.2
    Rsyslog version : 5.8.10

    Somebody has already this problem or reproduce it ? I have the same
    problem on all machines.
    No problem when I remove the ReadMode option but I need multiline.
    Moreover, I cannot update the Rsyslog version.

    Thanks for your help.

    Dle
    _________________________________________________
    rsyslog mailing list
    http://lists.adiscon.net/__mailman/listinfo/rsyslog
    <http://lists.adiscon.net/mailman/listinfo/rsyslog>
    http://www.rsyslog.com/__professional-services/
    <http://www.rsyslog.com/professional-services/>
    What's up with rsyslog? Follow https://twitter.com/rgerhards
    NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
    myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
    POST if you DON'T LIKE THAT.


_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Reply via email to