Some time ago, I did some testing with SolrCloud (mostly 5.0 and
branch_5x) handling thousands of collections.

How that testing went is documented in SOLR-7191.  That issue has been
marked as resolved in version 6.3, but no commits were made in the
issue, and I haven't seen any evidence to suggest that recent SolrCloud
versions can handle thousands of collections any better than older
versions did.  In fact, what evidence I *have* seen suggests that newer
versions are worse than older versions in this regard.

More recently, I tried to re-run the testing with version 6.4.0, but
never could get all the collections created, so I couldn't do the same
tests.  Between SOLR-10130 and a number of SolrCloud fixes included in
later 6.x releases, I had hoped that a new version would fare better.

I have begun some new testing with 7.0.1.  This has been plagued by a
bunch of problems.  I am still having trouble creating all of the
collections for the test, and I'm seeing evidence of very slow and
problematic restarts even though I haven't yet created all the
collections I'm after.

One persistent problem I've encountered is that every collection
creation (using the Collections API) is taking longer than expected to
complete.  Just creating the "gettingstarted" collection while setting
up the cloud example had a QTime for the Collections API of over seven
seconds, and subsequent collections each take at least four seconds. 
Creating those initial collections should take a LOT less time.

Because a number of scalability-related fixes have been made since
SOLR-7191, I will likely file a new issue to cover problems I've
encountered with 7.0.1.

I wanted to add some temporary logging to the collection creation code,
so I could see how long each part of the creation was taking.  I hope to
locate the bottlenecks that cause creation to take several seconds. 
Unfortunately, I was unable to figure out *where* that code is, so I've
got no idea where to add the logging.  Can anyone point me to the right
place in the code?

Thanks,
Shawn

Reply via email to