[
https://issues.apache.org/jira/browse/SOLR-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16414979#comment-16414979
]
Shawn Heisey commented on SOLR-11934:
-------------------------------------
I wrote a lengthy comment on SOLR-7887, with some details that might help with
deciding what the default logfile rotation size should be. See the comment on
that issue for full gory details. Erick reminded me about this issue as the
correct place to discuss it.
A summary, part of that long comment:
The 4MB log covered times from 18:51:26.552 to 18:57:58.139, about six and a
half minutes. The 32MB log covered times from 18:01:23.375 to 18:58:13.886,
which is almost an hour.
This server, running version 4.7.2, has a pretty low query rate, but each user
query is quite large in terms URL parameter size -- filters are added by the
application to limit what the user can see according to their permission level.
The log is dominated by the ping requests from the load balancer and those
very large user queries. It is my opinion that those logs should NOT be moved
out of the INFO level.
I don't have any newer versions with production usage, so I can't comment on
what will happen when running a version where attempts have been made to reduce
the noise level in the logs.
Based on what I found when I examined my logs, I think the default logfile
rotation size needs to be fairly large. The current 4MB is certainly too
small. Perhaps 64MB or 128MB? We'll need to see if we can ask some users
varying traffic levels to provide us some information about their logfiles, to
try and come up with a number that will preserve a decent amount of history for
a "typical" install, without consuming REALLY large amounts of disk space.
> Visit Solr logging, it's too noisy.
> -----------------------------------
>
> Key: SOLR-11934
> URL: https://issues.apache.org/jira/browse/SOLR-11934
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Priority: Major
>
> I think we have way too much INFO level logging. Or, perhaps more correctly,
> Solr logging needs to be examined and messages logged at an appropriate level.
> We log every update at an INFO level for instance. But I think we log LIR at
> INFO as well. As a sysadmin I don't care to have my logs polluted with a
> message for every update, but if I'm trying to keep my system healthy I want
> to see LIR messages and try to understand why.
> Plus, in large installations logging at INFO level is creating a _LOT_ of
> files.
> What I want to discuss on this JIRA is
> 1> What kinds of messages do we want log at WARN, INFO, DEBUG, and TRACE
> levels?
> 2> Who's the audience at each level? For a running system that's functioning,
> sysops folks would really like WARN messages that mean something need
> attention for instance. If I'm troubleshooting should I turn on INFO? DEBUG?
> TRACE?
> So let's say we get some kind of agreement as to the above. Then I propose
> three things
> 1> Someone (and probably me but all help gratefully accepted) needs to go
> through our logging and assign appropriate levels. This will take quite a
> while, I intend to work on it in small chunks.
> 2> Actually answer whether unnecessary objects are created when something
> like log.info("whatever {}", someObjectOrMethodCall); is invoked. Is this
> independent on the logging implementation used? The SLF4J and log4j seem a
> bit contradictory.
> 3> Maybe regularize log, logger, LOG as variable names, but that's a nit.
> As a tactical approach, I suggest we tag each LoggerFactory.getLogger in
> files we work on with //SOLR-(whatever number is assigned when I create
> this). We can remove them all later, but since I expect to approach this
> piecemeal it'd be nice to keep track of which files have been done already.
> Finally, I really really really don't want to do this all at once. There are
> 5-6 thousand log messages. Even at 1,000 a week that's 6 weeks, even starting
> now it would probably span the 7.3 release.
> This will probably be an umbrella issue so we can keep all the commits
> straight and people can volunteer to "fix the files in core" as a separate
> piece of work (hint).
> There are several existing JIRAs about logging in general, let's link them in
> here as well.
> Let the discussion begin!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
