Thanks Botond,
Unfortunately, I haven't been able to reproduce it successfully outside my
production environment. We have however moved to renaming our logs -1 day
instead of -2 days and it seems to have helped significantly. I'll try and
reproduce it in my test env this weekend. Looks to be related to number of
large files as my initial test with 50 small files did not reproduce the
problem.
On Thu, Oct 3, 2013 at 12:31 AM, Botond Botyanszki <b...@nxlog.org> wrote:
> Hi,
>
> There is no maximum limit. The number in the DEBUG log below is only to
> limit files which can be open at the same time. Even if there are more
> files than this which were appended to, all should be detected and read.
> There is some work in progress which should address situations where the
> current im_file implementation seems to have problems.
>
> Regards,
> Botond
>
>
> On Tue, 1 Oct 2013 20:34:14 -0700
> DMF Lists <dmfli...@gmail.com> wrote:
>
> > Hi Botond,
> >
> > I finally had some time to try and dig into this. I was able to trace it
> to
> > the client nxlog process not sending the logs. I set the log level to
> debug
> > and noticed several lines with:
> >
> > 2013-10-01 20:09:57 DEBUG opening
> > C:\Inetpub\logs\W3SVC123456789\ex131002.log
> > 2013-10-01 20:09:57 DEBUG maximum number (21) of files open, closing
> current
> >
> > Is there a maximum limit of log files that can be opened by nxlog? I'm
> > renaming my logs to miss the wildcard path but I still have roughly 50
> iis
> > files that nxlog would have to keep open at any given time.
> >
> > Thanks.
> >
> >
> >
> >
> > On Fri, Aug 2, 2013 at 10:28 AM, DMF Lists <dmfli...@gmail.com> wrote:
> >
> > > Right, I am renaming the files to avoid the wildcard masking every
> couple
> > > of days. I'll try and reproduce the scenario and submit a bug.
> > >
> > >
> > > On Fri, Aug 2, 2013 at 8:56 AM, Botond Botyanszki <b...@nxlog.org>
> wrote:
> > >
> > >> Hi,
> > >>
> > >> Rotated log files (including IIS) should work ok using a wildcarded
> File
> > >> directive. Actually by default IIS always starts a new file daily and
> > >> there is no rotation, but it might be configured differently in your
> > >> setup. If you have rotated files make sure to have them rotated to
> > >> another directory or at least in such a way that the wildcarded name
> > >> doesn't match those.
> > >>
> > >> Regards,
> > >> Botond
> > >>
> > >> On Fri, 2 Aug 2013 01:20:25 -0700
> > >> DMF Lists <dmfli...@gmail.com> wrote:
> > >>
> > >> > Thanks Botond,
> > >> >
> > >> > That's why I haven't been using the pm_buffer module. However, now
> that
> > >> I
> > >> > think of it, it's generally the iis logs that seem to stop being
> > >> forwarded.
> > >> > Is it possible that if the flow control pauses the clients and the
> file
> > >> > gets rotated while it's paused, nxlog doesn't know how to continue?
> What
> > >> > would you suggest for iis logs that are rolled daily?
> > >> >
> > >> > I was asking about redis/ampq support for the same reason, use it
> as a
> > >> > queue. I'll take a look at those options you mentioned. Thanks.
> > >> >
> > >> >
> > >> > On Fri, Aug 2, 2013 at 1:02 AM, Botond Botyanszki <b...@nxlog.org>
> > >> wrote:
> > >> >
> > >> > > Hi,
> > >> > >
> > >> > > Ok, I think some misconceptions needs to be cleared up because a
> lot
> > >> > > of people tend to think nxlog will drop messages, probably due to
> > >> > > experience with other log collector software that behaves this
> way.
> > >> > >
> > >> > > See the note at
> > >> > >
> > >> > >
> > >>
> http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html#pm_buffer
> > >> > > Also see the FlowControl directive:
> > >> > >
> > >> > >
> > >>
> http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html#config_module_flowcontrol
> > >> > >
> > >> > > In short:
> > >> > > if you are using im_msvistalog => om_tcp you will not need to use
> > >> > > pm_buffer or any other external buffering solution such as redis.
> When
> > >> > > om_tcp detects that it cannot send, the input module
> (im_msvistalog)
> > >> will
> > >> > > stop reading logs. When the tcp connection is resumed, the input
> > >> module
> > >> > > will resume reading from the same position and will properly
> collect
> > >> all
> > >> > > messages that were stored in the windows eventlog during that
> time.
> > >> This
> > >> > > works the same way with im_file. This should also work if you
> restart
> > >> > > nxlog during this time as it will remember the last position read
> in
> > >> the
> > >> > > input.
> > >> > >
> > >> > > The only situation this can go wrong is when there is log rotation
> > >> set up
> > >> > > for the windows eventlog, or the log file read by im_file. When
> nxlog
> > >> > > tries to resume, the input source is no longer there. Normally
> this
> > >> > > shouldn't be a problem.
> > >> > >
> > >> > > Redis and AMPQ support would be nice to have, there are plans to
> have
> > >> > > support for these , unfortunately there are always higher priority
> > >> > > issues/features. If someone would volunteer, I'd be happy to help.
> > >> > >
> > >> > > Interestingly the exact same issue has been discussed on #nxlog
> > >> yesterday:
> > >> > >
> > >> > > (04:52:10 PM) nemish: b0ti: does nxlog support redis output?
> > >> > > (04:53:39 PM) b0ti: nemish: not directly
> > >> > > (04:54:58 PM) b0ti: there are different options: use some script
> with
> > >> > > om_exec
> > >> > > (04:55:01 PM) b0ti:
> > >> > > https://github.com/blakeblackshear/nxlog-to-redis-proxy
> > >> > > (04:55:24 PM) b0ti: nxlog => logstash => redis
> > >> > > (04:57:10 PM) b0ti: another option might be om_http + webdis
> > >> > > 04:57:02 PM) nemish: b0ti: yeah i was trying to do nxlog -> redis
> ->
> > >> > > logstash or some intermediary queue in case LS is down or being
> > >> recycled
> > >> > > (04:58:12 PM) b0ti: so you only want to use redis as a buffer?
> > >> > >
> > >> > > Regards,
> > >> > > Botond
> > >> > >
> > >> > >
> > >> > > On Fri, 2 Aug 2013 00:30:23 -0700
> > >> > > DMF Lists <dmfli...@gmail.com> wrote:
> > >> > >
> > >> > > > I was expecting that even without a buffer that nxlog would
> > >> reconnect to
> > >> > > > it's outputs once the target started accepting connections
> again.
> > >> > > >
> > >> > > > Botond, do you have any plans to support Redis or
> RabbitMQ(AMPQ) as
> > >> an
> > >> > > > output?
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Aug 1, 2013 at 6:12 PM, Mark D. Nagel <
> > >> mna...@willingminds.com
> > >> > > >wrote:
> > >> > > >
> > >> > > > > On 8/1/2013 5:49 PM, DMF Lists wrote:
> > >> > > > > > I'm setup to ship windows logs to a central linux host
> running
> > >> nxlog
> > >> > > via
> > >> > > > > native
> > >> > > > > > transport which then ships it off to logstash via generic
> TCP.
> > >> If
> > >> > > > > logstash backs up or
> > >> > > > > > dies and it's caught immediately and restarted, the nxlog
> > >> transport
> > >> > > > > chain seems to
> > >> > > > > > survive. However, if logstash is stopped for a long period
> of
> > >> time
> > >> > > ie a
> > >> > > > > few hours, not
> > >> > > > > > all windows nxlog clients will reconnect properly. The nxlog
> > >> logs
> > >> > > just
> > >> > > > > show the
> > >> > > > > > following mutiple times:
> > >> > > > > >
> > >> > > > > > 2013-03-27 14:49:59 ERROR couldn't connect to tcp socket on
> > >> > > logger1:514
> > >> > > > > No connection
> > >> > > > > > could be made because the target machine actively refused
> it.
> > >> > > > > > 2013-03-27 14:50:00 INFO reconnecting in 10 seconds
> > >> > > > > >
> > >> > > > > > Once I restart the nxlog client on the windows side, it
> starts
> > >> > > logging
> > >> > > > > again. Any thoughts?
> > >> > > > >
> > >> > > > > I believe this is a use case for pm_buffer -
> > >> > > > >
> > >> > > > >
> > >> > >
> > >>
> http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html#pm_buffer
> > >> > > .
> > >> > > > > You
> > >> > > > > just need to figure out how much disk storage to reserve for
> your
> > >> worst
> > >> > > > > case.
> > >> > > > >
> > >> > > > > Regards,
> > >> > > > > Mark
> > >> > > > >
> > >> > > > > --
> > >> > > > > Mark D. Nagel, CCIE #3177 <mna...@willingminds.com>
> > >> > > > > Principal Consultant, Willing Minds LLC (
> > >> http://www.willingminds.com)
> > >> > > > > cell: 949-279-5817, desk: 714-495-4001, fax: 714-646-8277
> > >> > > > >
> > >> > > > > ** For faster support response time, please
> > >> > > > > ** email supp...@willingminds.com or call 714-495-4000
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > >
> > >>
> ------------------------------------------------------------------------------
> > >> > > > > Get your SQL database under version control now!
> > >> > > > > Version control is standard for application code, but
> databases
> > >> havent
> > >> > > > > caught up. So what steps can you take to put your SQL
> databases
> > >> under
> > >> > > > > version control? Why should you start doing it? Read more to
> find
> > >> out.
> > >> > > > >
> > >> > >
> > >>
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> > >> > > > > _______________________________________________
> > >> > > > > nxlog-ce-users mailing list
> > >> > > > > nxlog-ce-users@lists.sourceforge.net
> > >> > > > > https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
> > >> > > > >
> > >> > >
> > >> > >
> > >> > >
> > >>
> ------------------------------------------------------------------------------
> > >> > > Get your SQL database under version control now!
> > >> > > Version control is standard for application code, but databases
> havent
> > >> > > caught up. So what steps can you take to put your SQL databases
> under
> > >> > > version control? Why should you start doing it? Read more to find
> out.
> > >> > >
> > >>
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> > >> > > _______________________________________________
> > >> > > nxlog-ce-users mailing list
> > >> > > nxlog-ce-users@lists.sourceforge.net
> > >> > > https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
> > >> > >
> > >>
> > >
> > >
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
> _______________________________________________
> nxlog-ce-users mailing list
> nxlog-ce-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
>
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________
nxlog-ce-users mailing list
nxlog-ce-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
