Is there any documentation that describes when the input and output cache are used?
Both of mine grow very large, and I just restarted graylog2-server to see if it "compacts/deletes" these files. The output cache shrunk to almost nothing, but the input cache is still very large. Tristan On Thu, Dec 4, 2014 at 8:42 AM, Jochen Schalanda <[email protected]> wrote: > Hi Chris, > > On Wednesday, 3 December 2014 19:06:42 UTC+1, Chris Tresco wrote: >> >> I am wondering why this file gets so large and what I can do to keep the >> size down. It being that big, it seems to me it would be a problem with >> feeding messages to elasticsearch for indexing but I am not sure how to >> troubleshoot. >> > > The spool files will currently be compacted (which means that old messages > will be physically removed and not only marked as deleted) when starting > Graylog2. There is no "on-line" compaction at the moment. > > The output cache should usually be empty if Elasticsearch can index > messages fast enough. If you're using a queued input (e. g. an AMQP or > Kafka input) you might want to disable the output cache (possible since > Graylog2 0.92.0, see > https://github.com/Graylog2/graylog2-server/blob/0.92/misc/graylog2.conf#L343-347) > to generate proper back-pressure. Otherwise messages will just be written > into the output cache, even if the actual backend is not able to keep up > with indexing). > > > Cheers, > Jochen > > -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Tristan Rhodes -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
