lehong.ding created SOLR-15408:
----------------------------------
Summary: ConcurrentUpdate issue of solr 8.8.1
Key: SOLR-15408
URL: https://issues.apache.org/jira/browse/SOLR-15408
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 8.8.1
Reporter: lehong.ding
h4. *Background:*
Before we moving to solr 8.8.1 from 7.7.2, we performed some performance test
on solr 8.8.1. We met a lot of concurrent update error in solr log.
*Envrironment:*
{{3 cluster nodes with 500 collections, each has about 1m documents.}}
h4. *Threads:*
30 update/add threads + 10 deleteByQuery threads
h4. *Results:*
During deleteByQuery thread runing, only one node (lead node) has update
transactions, but other two node has none .
h4. Errrors:
java.io.IOException: Request processing has stalled for 20091ms with 100
remaining elements in the queue.java.io.IOException: Request processing has
stalled for 20091ms with 100 remaining elements in the queue. at
org.apache.solr.client.solrj.impl.ConcurrentUpdateHttp2SolrClient.request(ConcurrentUpdateHttp2SolrClient.java:449)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290) at
org.apache.solr.update.SolrCmdDistributor.doRequest(SolrCmdDistributor.java:345)
at
org.apache.solr.update.SolrCmdDistributor.submit(SolrCmdDistributor.java:338)
at
org.apache.solr.update.SolrCmdDistributor.distribAdd(SolrCmdDistributor.java:244)
at
org.apache.solr.update.processor.DistributedZkUpdateProcessor.doDistribAdd(DistributedZkUpdateProcessor.java:300)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:230)
at
org.apache.solr.update.processor.DistributedZkUpdateProcessor.processAdd(DistributedZkUpdateProcessor.java:245)
at
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:106)
at
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:110)
at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:343)
at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readIterator(JavaBinUpdateRequestCodec.java:291)
at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:338)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:283) at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readNamedList(JavaBinUpdateRequestCodec.java:244)
h4. Temporary Solution:
adding -Dsolr.http1=1 in solr start parameters
There are still some error in error log but the number is much small.
h4. My Questions:
1 We found solr cluster will eventually get the data consistent. What’s the
concurrent update error mainly impacted?
2 Adding -Dsolr.http1=1 in solr start parameters can reduce the error number.
Do we realy need add this parameter? And does this parameter will be kept in
later version?
Many Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
