Hi,
I don't seem to be making as much progress as I would hope for. I now have
the following client config:
action( type="omfwd"
target="192.168.8.134"
port="514"
protocol="tcp"
queue.size="50000"
queue.type="LinkedList"
queue.spoolDirectory="/var/lib/rsyslog"
queue.filename="testforwardingqueue"
queue.lowwatermark="20000"
queue.highwatermark="35000"
queue.discardmark="50000"
queue.maxfilesize="1g"
queue.saveonshutdown="on"
# queue.fulldelaymark="2500"
)
I have made the numbers smaller to be able to do testing that will finish
in a reasonable amount of time.
My process is:
1. Start firewall on master server to block all access.
2. On client server start logging using: for i in $(seq 1 40000); do
logger -p error "############### TESTING ########### SEQ001-${i}"; done
3. Wait for for loop to finish and check if all entries are contained
locally
4. watch 'ls -l /var/lib/rsyslog' to see if queue gets created.
5. Disable firewall on master
6. run logger -p error with one message to get logs going again
7. Wait for logs to arrive on master server
8. grep 'SEQ001' /var/log/messages -c
The problem I am getting is that the number of messages I receive on the
remote server does not correspond with what I get on the master server.
I only get 24768 messages on the master server.
Is there anything else that I have missed out of the configuration? I can't
see anything that would cause my messages to not arrive. The queue file I
specified grows in size but interestingly when I grep'ed it for SEQ005 I
only get 15232 matches. I did the grep after re-enabling the firewall.
I thought that it might be because syslog thinks my messages are the same
and then marks them as duplicates. I then added:
for i in $(seq 1 40000); do logger -p error "############### TESTING
########### SEQ005-${i}-$(openssl rand -base64 32)"; done
to try and prevent that. However if the firewall is not enabled then all
40k messages arrive on the master server.
Any help would be appreciated.
Regards
On 13 July 2015 at 14:20, Rainer Gerhards <[email protected]> wrote:
> 2015-07-13 15:18 GMT+02:00 Gerhardus Geldenhuis
> <[email protected]>:
> > Hi,
> > Thanks for the replies. I think the bulk of my problem was mixing old and
> > new config and I much further along to get something working. I have
> > discovered a few other niggles but will report back once I have something
> > working properly. As far as pull requests go, I would really consider
> doing
> > so but as always time is a factor.
>
> yup - for everyone ;) And that is what it is like it currently is ;)
>
> > It does bug me so much that I must just
> > end up doing it for the documentation. I will give debug mode a go.
> >
> > Regards
> >
> > On 13 July 2015 at 13:17, Rainer Gerhards <[email protected]>
> wrote:
> >
> >> 2015-07-10 13:40 GMT+02:00 Gerhardus Geldenhuis
> >> <[email protected]>:
> >> > Hi
> >> > I am struggling a bit to get rsyslog to work as described.
> >> >
> >> > <rant>
> >> > Firstly the documentation is a struggle. There is some reference to
> old
> >> and
> >> > new style configuration but no clear differentiation between the two.
> >> What
> >> > makes it more confusing is that documents like
> >> > http://www.rsyslog.com/doc/queues.html then refers to what looks like
> >> the
> >> > old style of config and none of the examples contains new syntax
> >> examples.
> >> >
> >> > There was also an expectation that the new rsyslog package would
> install
> >> a
> >> > new style config but that turned out to be not the case. I deleted the
> >> > config file and did a yum reinstall just to be sure.
> >> > </rant>
> >>
> >> well, this is open source. Pull requests are always appreciated,
> >> anything else happens as time permits ;)
> >>
> >> >
> >> > OS: CentOS 7
> >> > RSyslog:
> >> > rsyslog-8.11.0-1.el7.x86_64
> >> > rsyslog-relp-8.11.0-1.el7.x86_64
> >> > rsyslog-gnutls-8.11.0-1.el7.x86_64
> >> >
> >> > So basically what I am trying to achieve is the following:
> >> >
> >> > - Log remotely to a rsyslog server
> >> > - Turn off the remote server ( via firewall )
> >> > - Have logs be cached locally and saved to disk
> >> > - Restart client server
> >> > - Turn remote server back on
> >> > - See cached logs appear in the remote server
> >> >
> >> > It does not work...
> >> >
> >> > - So more specifically, if I turn the firewall off, log a few
> messages
> >> > and turn it back on then the caching works and I get the messages.
> >> > - If however I restart the client server, the logs never make it to
> >> the
> >> > remote sever, I can see the logs in the cached file but it does not
> >> get
> >> > send to the remote server.
> >> >
> >> > My config on the client:
> >> > #### MODULES ####
> >> > module(load="imuxsock") # provides support for local system logging
> (e.g.
> >> > via logger command)
> >> > module(load="imklog") # provides kernel logging support (previously
> >> done
> >> > by rklogd)
> >> >
> >> > #### GLOBAL DIRECTIVES ####
> >> > $IncludeConfig /etc/rsyslog.d/*.conf
> >> >
> >> > #### RULES ####
> >> >
> >> > *.info;mail.none;authpriv.none;cron.none
> /var/log/messages
> >> > authpriv.*
> /var/log/secure
> >> > mail.*
> /var/log/maillog
> >> > cron.* /var/log/cron
> >> > *.emerg :omusrmsg:*
> >> > uucp,news.crit
> /var/log/spooler
> >> > local7.*
> /var/log/boot.log
> >> >
> >> > # ### begin forwarding rule ###
> >> > $WorkDirectory /var/lib/rsyslog # where to place spool files
> >> > $MainMsgQueueFileName LocalCaching # unique name prefix for spool
> files
> >> > $MainMsgQueueSaveOnShutdown on # save messages to disk on shutdown
> >> > # $MainMsgQueueType LinkedList
> >> > $MainMsgQueueType Disk
> >> > $MainMsgResumeRetryCount -1 # infinite retries if host is down
> >> >
> >> > *.* @@192.168.8.253:514
> >> >
> >> > # ### end of the forwarding rule ###
> >> >
> >> > My config on the remote server:
> >> > module(load="imuxsock") # provides support for local system logging
> (e.g.
> >> > via logger command)
> >> > module(load="imklog") # provides kernel logging support (previously
> >> done
> >> > by rklogd)
> >> > module(load="imtcp") # needs to be done just once
> >> > input(type="imtcp" port="514")
> >> > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
> >> > $IncludeConfig /etc/rsyslog.d/*.conf
> >> > *.info;mail.none;authpriv.none;cron.none
> /var/log/messages
> >> > authpriv.*
> /var/log/secure
> >> > mail.*
> /var/log/maillog
> >> > cron.* /var/log/cron
> >> > *.emerg :omusrmsg:*
> >> > uucp,news.crit
> /var/log/spooler
> >> > local7.*
> >> >
> >> > Any pointers would be appreciated. I am hoping I am missing something
> >> > obvious or misunderstanding what I am suppose to be doing.
> >> >
> >>
> >> You should run rsyslog in such a situation in debug mode.From the
> >> debug log, we can see why it thinks it can't deliver to the remote
> >> system (well, hopefully ;)).
> >>
> >> HTH
> >> Rainer
> >>
> >> > Regards
> >> >
> >> > --
> >> > Gerhardus Geldenhuis
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com/professional-services/
> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >> DON'T LIKE THAT.
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> >> DON'T LIKE THAT.
> >>
> >
> >
> >
> > --
> > Gerhardus Geldenhuis
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
--
Gerhardus Geldenhuis
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
