Your soft commit time of 3 seconds is quite aggressive, I'd lengthen it to as long as possible.
Ugh, looked at your query more closely. Adding commit=true to every update request is horrible performance wise. Let your autocommit process handle the commits is the first thing I'd do. Second, I'd try going to SolrJ and batching up documents (I usually start with 1,000) or using the post.jar tool rather than sending them via a raw URL. I agree with Upayavira, 100 concurrent threads is a _lot_. Also, what version of Solr? There was a 2x speedup in Solr 5.2, see: http://lucidworks.com/blog/indexing-performance-solr-5-2-now-twice-fast/ One symptom was that the followers were doing waaaaay more work than the leader (BTW, using master/slave when talking SolrCloud is a bit confusing...) which will affect query response rates. Basically, if query response is paramount, you really need to throttle your indexing, there's just a whole lot of work going on here.. Best, Erick On Fri, Aug 7, 2015 at 11:23 AM, Upayavira <u...@odoko.co.uk> wrote: > How many CPUs do you have? 100 concurrent indexing calls seems like > rather a lot. You're gonna end up doing a lot of context switching, > hence degraded performance. Dunno what others would say, but I'd aim for > approx one indexing thread per CPU. > > Upayavira > > On Fri, Aug 7, 2015, at 02:58 PM, Nitin Solanki wrote: >> Hello Everyone, >> I have indexed 16 million documents in Solr >> Cloud. Created 4 nodes and 8 shards with single replica. >> I am trying to make concurrent indexing and searching on those indexed >> documents. Trying to make 100 concurrent indexing calls along with 100 >> concurrent searching calls. >> It *degrades searching and indexing* performance both. >> >> Configuration : >> >> "commitWithin":{"softCommit":true}, >> "autoCommit":{ >> "maxDocs":-1, >> "maxTime":60000, >> "openSearcher":false}, >> "autoSoftCommit":{ >> "maxDocs":-1, >> "maxTime":3000}}, >> >> "indexConfig":{ >> "maxBufferedDocs":-1, >> "maxMergeDocs":-1, >> "maxIndexingThreads":8, >> "mergeFactor":-1, >> "ramBufferSizeMB":100.0, >> "writeLockTimeout":-1, >> "lockType":"native"}}} >> >> AND <maxWarmingSearchers>2</maxWarmingSearchers> >> >> I don't have know that how master and slave works. Normally, I created 8 >> shards and indexed documents using : >> >> >> >> >> *http://localhost:8983/solr/test_commit_fast/update/json?commit=true >> <http://localhost:8983/solr/test_commit_fast/update/json?commit=true> -H >> 'Content-type:application/json' -d ' [ JSON_Document ]'*And Searching >> using >> *: http://localhost:8983/solr/test_commit_fast/select >> <http://localhost:8983/solr/test_commit_fast/select>*?q=< field_name: >> search_string> >> >> Please any help on it. To make searching and indexing fast concurrently. >> Thanks. >> >> >> Regards, >> Nitin
