Hi guys, I have a problem with data import (based on database sql) in Solr Cloud. I'm trying to import ~500 000 000 of documents and I've created 30 logical shards on 2 physical machines. Documents are distributed by composite id. After some time (5-10 minutes; about 400 000 documents) Solr Cloud stops indexing documents. This is because indexing thread parks and waits on semaphore: org.apache.solr.update.SolrCmdDistributor#semaphore.acquire() in method submit.
While indexing I see jdbc calls in stack trace but after it parks on semaphore I don't see any jdbc calls (I see only Solr and JDK method calls). Version of Solr: 4.4 Version of Lucene: 4.4 *With one shard and one physical machines everything is OK* *With one shard and two physical machines (one leader, one replica) everything is OK* This is really big problem for us because of large number of documents we have to shard index. We have unique queries with sorting so it leads to 1 minute long response times without sharding. Best, Kowish -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Cloud-dataimport-freezes-tp4090812.html Sent from the Solr - User mailing list archive at Nabble.com.
