[
https://issues.apache.org/jira/browse/SOLR-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464534#comment-16464534
]
Shawn Heisey commented on SOLR-11453:
-------------------------------------
The rollover size value in the default config used to be 4MB, so 32MB is a huge
improvement! I agree, it's a low value. But we can't make it TOO big, or Solr
may have trouble when somebody begins using it on a very small system. (Think
raspberry pi, or a super small container/VM). For the same reason, the default
heap size is only 512MB, which means that almost all users have to eventually
adjust their config.
When I deploy Solr for real usage, I change the rollover size to 4GB (on older
versions where it's log4j.properties). The systems I run it on have terabytes
of disk space, and I want to be sure that I WILL have something to look at
(without checking lots of log files) when somebody tells me about a problem
they had two or three days ago.
My dev server is running an older version (6.6.3). The production servers are
running even older versions. The current logfile on dev is 720MB and covers
these times:
2018-05-02 15:26:59.556
2018-05-05 00:22:35.226
I haven't modified the logging config much, so that should be fairly typical
for a server matching the size and activity level.
A fair amount of effort has been expended to reduce how much Solr logs in an
out-of-the-box configuration. When starting the Solr example on the most
current release (7.3.0) with one index that has nothing in it, the logfile gets
to 6KB. Here's the contents:
{noformat}
2018-05-05 00:21:48.416 INFO (main) [ ] o.e.j.s.Server
jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash:
82b8fb23f757335bb3329d540ce37a2a2615f0a8
2018-05-05 00:21:49.133 INFO (main) [ ] o.a.s.s.SolrDispatchFilter ___
_ Welcome to Apache Solr™ version 7.3.0
2018-05-05 00:21:49.133 INFO (main) [ ] o.a.s.s.SolrDispatchFilter / __|
___| |_ _ Starting in standalone mode on port 8983
2018-05-05 00:21:49.133 INFO (main) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _
\ | '_| Install dir: C:\Users\sheisey\Downloads\solr-7.3.0
2018-05-05 00:21:49.148 INFO (main) [ ] o.a.s.s.SolrDispatchFilter
|___/\___/_|_| Start time: 2018-05-05T00:21:49.135Z
2018-05-05 00:21:49.150 INFO (main) [ ] o.a.s.u.StartupLoggingUtils Property
solr.log.muteconsole given. Muting ConsoleAppender named CONSOLE
2018-05-05 00:21:49.238 INFO (main) [ ] o.a.s.c.SolrResourceLoader Using
system property solr.solr.home:
C:\Users\sheisey\Downloads\solr-7.3.0\server\solr
2018-05-05 00:21:49.245 INFO (main) [ ] o.a.s.c.SolrXmlConfig Loading
container configuration from
C:\Users\sheisey\Downloads\solr-7.3.0\server\solr\solr.xml
2018-05-05 00:21:50.197 INFO (main) [ ] o.a.s.c.CorePropertiesLocator Found
1 core definitions underneath C:\Users\sheisey\Downloads\solr-7.3.0\server\solr
2018-05-05 00:21:50.198 INFO (main) [ ] o.a.s.c.CorePropertiesLocator Cores
are: [foo]
2018-05-05 00:21:50.205 INFO (coreLoadExecutor-6-thread-1) [ ]
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647
transient cores
2018-05-05 00:21:50.323 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.c.SolrResourceLoader [foo] Added 57 libs to classloader, from paths:
[/C:/Users/sheisey/Downloads/solr-7.3.0/contrib/clustering/lib,
/C:/Users/sheisey/Downloads/solr-7.3.0/contrib/extraction/lib,
/C:/Users/sheisey/Downloads/solr-7.3.0/contrib/langid/lib,
/C:/Users/sheisey/Downloads/solr-7.3.0/contrib/velocity/lib,
/C:/Users/sheisey/Downloads/solr-7.3.0/dist]
2018-05-05 00:21:53.197 INFO (main) [ ] o.e.j.s.Server Started @6060ms
2018-05-05 00:21:53.898 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
2018-05-05 00:21:54.191 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.s.IndexSchema [foo] Schema name=default-config
2018-05-05 00:21:54.630 WARN (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class
[solr.SynonymFilterFactory]. Please consult documentation how to replace it
accordingly.
2018-05-05 00:21:55.357 INFO (qtp818403870-15) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/info/system params={wt=json} status=0 QTime=51
2018-05-05 00:21:55.371 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
2018-05-05 00:21:55.438 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.c.CoreContainer Creating SolrCore 'foo' using configuration from
instancedir C:\Users\sheisey\Downloads\solr-7.3.0\server\solr\foo, trusted=true
2018-05-05 00:21:55.485 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
2018-05-05 00:21:55.492 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.c.SolrCore [[foo] ] Opening new SolrCore at
[C:\Users\sheisey\Downloads\solr-7.3.0\server\solr\foo],
dataDir=[C:\Users\sheisey\Downloads\solr-7.3.0\server\solr\foo\data\]
2018-05-05 00:21:55.534 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
2018-05-05 00:21:55.895 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.u.UpdateHandler Using UpdateLog implementation:
org.apache.solr.update.UpdateLog
2018-05-05 00:21:55.896 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
2018-05-05 00:21:55.915 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
2018-05-05 00:21:55.915 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.u.CommitTracker Soft AutoCommit: disabled
2018-05-05 00:21:56.010 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.s.SolrIndexSearcher Opening [Searcher@67d0efb2[foo] main]
2018-05-05 00:21:56.050 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir:
C:\Users\sheisey\Downloads\solr-7.3.0\server\solr\foo\conf
2018-05-05 00:21:56.101 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.h.c.SpellCheckComponent Initializing spell checkers
2018-05-05 00:21:56.111 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.s.DirectSolrSpellChecker init:
{name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
2018-05-05 00:21:56.130 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
2018-05-05 00:21:56.270 INFO (coreLoadExecutor-6-thread-1) [ x:foo]
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using
new clock 1599581418967531520
2018-05-05 00:21:56.282 INFO (searcherExecutor-7-thread-1-processing-x:foo) [
x:foo] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to
Searcher@67d0efb2[foo]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
2018-05-05 00:21:56.282 INFO (searcherExecutor-7-thread-1-processing-x:foo) [
x:foo] o.a.s.c.QuerySenderListener QuerySenderListener done.
2018-05-05 00:21:56.282 INFO (searcherExecutor-7-thread-1-processing-x:foo) [
x:foo] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker:
default
2018-05-05 00:21:56.293 INFO (searcherExecutor-7-thread-1-processing-x:foo) [
x:foo] o.a.s.c.SolrCore [foo] Registered new searcher Searcher@67d0efb2[foo]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
{noformat}
That version is still using log4j1.2, but I would not anticipate much change
with the next version using log4j2.
> Create separate logger for slow requests
> ----------------------------------------
>
> Key: SOLR-11453
> URL: https://issues.apache.org/jira/browse/SOLR-11453
> Project: Solr
> Issue Type: Improvement
> Components: logging
> Affects Versions: 7.0.1
> Reporter: Shawn Heisey
> Assignee: Shawn Heisey
> Priority: Minor
> Attachments: SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch
>
>
> There is some desire on the mailing list to create a separate logfile for
> slow queries. Currently it is not possible to do this cleanly, because the
> WARN level used by slow query logging within the SolrCore class is also used
> for other events that SolrCore can log. Those messages would be out of place
> in a slow query log. They should typically stay in main solr logfile.
> I propose creating a custom logger for slow queries, similar to what has been
> set up for request logging. In the SolrCore class, which is
> org.apache.solr.core.SolrCore, there is a special logger at
> org.apache.solr.core.SolrCore.Request. This is not a real class, just a
> logger which makes it possible to handle those log messages differently than
> the rest of Solr's logging. I propose setting up another custom logger
> within SolrCore which could be org.apache.solr.core.SolrCore.SlowRequest.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
