Hey Cornelius,
On 29.08.2014, at 10:32, [email protected] wrote:
> Hello,
> any 2 or three days I get the following message within graylog2:
>
> Processing of a stream has been disabled due to excessive processing time. 16
> minutes ago
> The processing of stream SVP has taken too long for 0 times. To protect the
> stability of message processing, this stream has been disabled. Please
> correct the stream rules and reenable the stream.
>
> Corresponding to this I see something like this in graylog2-server.log
>
> 2014-08-29 09:09:20,795 WARN : org.graylog2.streams.StreamRouter - Processing
> of stream <53e238930cf22c9d5ed90244> failed to return within 2000ms.
>
> I have some questions on this
> a) what is the reason for this or the other way round: does the detection of
> long processing time depend on message size? (We sometimes have messages with
> some MegaByte of payload, of course analasis takes some time...)
Yes. Especially if you are using one or more regular expressions to filter your
streams, it is depending very much on your message size.
> b) is there a way to increase the limit to e.g. 10000 ms?
There is a configuration paramater named "stream_processing_timeout" which
defines the upper runtime limit in ms.
> c) is there a way to skip only messages with problems - and resume the stream
> with the next message - or at least e.g. within the next minute? Today I have
> to observe the error - and resume the stream by hand - and the resume was
> successful anytime...
It is possible that it does not completely disable the stream even after
continuously hitting the runtime limit for individual messages when you are
setting "stream_processing_max_faults" to 0 in the server's configuration file.
The default for this is 3. If you set it to 0, then streams will never get
disabled, but if you have defined a processing timeout other than 0, individual
messages which exceed this runtime limit will be dropped.
Hope this helps,
Dennis
--
TORCH GmbH
Steckelhörn 11
20457 Hamburg
Tel +49 (0)40-60945200
https://www.torch.sh
Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann
--
You received this message because you are subscribed to the Google Groups
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
