Re: Slow qTime for distributed search

2013-04-12 Thread Furkan KAMACI
Manuel Le Normand, I am sorry but I want to learn something. You said you have 40 dedicated servers. What is your total document count, total document size, and total shard size? 2013/4/11 Manuel Le Normand > Hi, > We have different working hours, sorry for the reply delay. Your assumed > number

Re: Slow qTime for distributed search

2013-04-11 Thread Manuel Le Normand
Hi, We have different working hours, sorry for the reply delay. Your assumed numbers are right, about 25-30Kb per doc. giving a total of 15G per shard, there are two shards per server (+2 slaves that should do no work normally). An average query has about 30 conditions (OR AND mixed), most of them

Re: Slow qTime for distributed search

2013-04-09 Thread Shawn Heisey
On 4/9/2013 3:50 PM, Furkan KAMACI wrote: Hi Shawn; You say that: *... your documents are about 50KB each. That would translate to an index that's at least 25GB* I know we can not say an exact size but what is the approximately ratio of document size / index size according to your experiences

Re: Slow qTime for distributed search

2013-04-09 Thread Furkan KAMACI
Hi Shawn; You say that: *... your documents are about 50KB each. That would translate to an index that's at least 25GB* I know we can not say an exact size but what is the approximately ratio of document size / index size according to your experiences? 2013/4/9 Shawn Heisey > On 4/9/2013 2:

Re: Slow qTime for distributed search

2013-04-09 Thread Shawn Heisey
On 4/9/2013 2:10 PM, Manuel Le Normand wrote: Thanks for replying. My config: - 40 dedicated servers, dual-core each - Running Tomcat servlet on Linux - 12 Gb RAM per server, splitted half between OS and Solr - Complex queries (up to 30 conditions on different fields), 1 qps rate

Re: Slow qTime for distributed search

2013-04-09 Thread Manuel Le Normand
Thanks for replying. My config: - 40 dedicated servers, dual-core each - Running Tomcat servlet on Linux - 12 Gb RAM per server, splitted half between OS and Solr - Complex queries (up to 30 conditions on different fields), 1 qps rate Sharding my index was done for two reasons, based

Re: Slow qTime for distributed search

2013-04-08 Thread Shawn Heisey
On 4/8/2013 12:19 PM, Manuel Le Normand wrote: It seems that sharding my collection to many shards slowed down unreasonably, and I'm trying to investigate why. First, I created "collection1" - 4 shards*replicationFactor=1 collection on 2 servers. Second I created "collection2" - 48 shards*replic

Re: Slow qTime for distributed search

2013-04-08 Thread Manuel Le Normand
After taking a look on what I'd wrote earlier, I will try to rephrase in a clear manner. It seems that sharding my collection to many shards slowed down unreasonably, and I'm trying to investigate why. First, I created "collection1" - 4 shards*replicationFactor=1 collection on 2 servers. Second I

Slow qTime for distributed search

2013-04-07 Thread Manuel Le Normand
Hello After performing a benchmark session on small scale i moved to a full scale on 16 quad core servers. Observations at small scale gave me excellent qTime (about 150 ms) with up to 2 servers, showing my searching thread was mainly cpu bounded. My query set is not faceted. Growing to full scale