Hello
We are using Solr 8.9.0. We have configured Solr cloud like 2 shards and each
shard has one replica. We have used 5 zoo keepers for Solr cloud.
We have used the below schema field in employee collection.
<field name="id" type="string" indexed="true" stored="true" required="true"
multiValued="false" docValues="true"/>
<dynamicField name="/*" type="text" indexed="true" stored="true"
multiValued="true"/>
Total no of record: 8562099
Size of instance:
solrgnrls2r1----67GB
solrgnrls1----66 GB
solrgnrls1r1----66 GB
solrgnrls2----68 GB
Solr logs:
2023-09-14 10:04:30.705 DEBUG (qtp1984975621-8805766) [c:employee s:shard1
r:core_node3 x:employee_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2
updateDocuments(add{_version_=1777003156686766080,id=EMP5487098118986160})
2023-09-14 10:04:30.710 INFO (qtp1984975621-8805766) [c:employee s:shard1
r:core_node3 x:employee_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory
[forms_shard1_replica_n1] webapp=/solr path=/update
params={wt=javabin&version=2}{add=[EMP5487098118986160 (1777003156686766080)]}
0 5
2023-09-14 10:04:30.807 DEBUG (commitScheduler-930-thread-1) [c:employee
s:shard1 r:core_node3 x:employee_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 start
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
2023-09-14 10:04:35.134 DEBUG (commitScheduler-930-thread-1) [c:employee
s:shard1 r:core_node3 x:employee_shard1_replica_n1] o.a.s.s.SolrIndexSearcher
Opening [Searcher@796ab9b9[employee_shard1_replica_n1] main]
2023-09-14 10:04:35.413 DEBUG (commitScheduler-930-thread-1) [c:employee
s:shard1 r:core_node3 x:employee_shard1_replica_n1]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
Why is the commitScheduler thread taking 5 seconds to complete? Due to this, we
cannot see the latest update for id EMP5487098118986160. We also have another
collection with an index size of 120 GB and the number of documents is
744620373 but still, there is no slowness in the soft commit.
When we checked Solr source code we found that time spent in
ExitableDirectoryReader.wrap(UninvertingReader.wrap(reader,
core.getLatestSchema().getUninversionMapper()),SolrQueryTimeoutImpl.getInstance());
this.leafReader = SlowCompositeReaderWrapper.wrap(this.reader);
Is this issue due to dynamicField? because we have stored lots of fields in
dynamicField. How can we reduce the soft commit time?
Regards,
Vishal
