Hi, all!
I have a problem with distributed search. I downloaded one shard from my
production. It has:
* ~29M docs
* 11 fields
* ~105M terms
* size of shard is: 13GB
On production there are near 30 the same shards. I split this shard to 4
more smaller shards, so now I have:
small shard1:
docs: 6.2M
terms: 27.2M
size: 2.89GB
small shard2:
docs: 6.3M
terms: 28.7M
size: 2.98GB
small shard3:
docs: 7.9M
terms: 32.8M
size: 3.60GB
small shard4:
docs: 8.2M
terms: 32.6M
size: 3.70GB
My machine confguration:
ABIT AX-78
AMD Athlon 64 X2 5200+
DDR2 Kingston 2x2G+2x1G = 6G
WDC WD2500JS (System here)
WDC WD20EARS (6 partitions = 30 GB for shards at begin of drive, and other
empty, all partitions are well aligned)
GNU/Linux Debian Squeeze
Tomcat 6.0.32 with JAVA_OPTS:
JAVA_OPTS="$JAVA_OPTS -XX:+DisableExplicitGC -server \
-XX:PermSize=512M -XX:MaxPermSize=512M -Xmx4096M -Xms4096M
-XX:NewSize=128M -XX:MaxNewSize=128M \
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled \
-XX:CMSInitiatingOccupancyFraction=50 -XX:GCTimeRatio=9
-XX:MinHeapFreeRatio=25 -XX:MaxHeapFreeRatio=25 \
-verbose:gc -XX:+PrintGCTimeStamps -Xloggc:$CATALINA_HOME/logs/gc.log"
Solr 3.5
I configured 4 cores and start Tomcat. I write a bash script. It's runing
during 300 seconds and sending every 6 seconds queries like
http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(assistants)&rows=2000&start=0&fl=*,score&qt=requestShards
where qt=requestShards is my 4 shards. After test I have the results:
Elapsed time: 299 secs
--- solr ---
Queries processed: 21 << this is full response file
Queries cancelled: 29 << this is number of killed curls
Average QTime is: 59645.6 ms
Average RTime is: 59.7619 sec(s) << this is average time difference between
start and stop the curl. There is a part of script:
# >>dcs=`date +%s`
# >>curl ${url} -s -H 'Content-type:text/xml; charset=utf-8' >
${F_DATADIR}/$dest.fdata
# >>dce=`date +%s`
# >>dcd=$(echo "$dce - $dcs" | bc)
Size of data-dir is: 3346766 bytes << this is response dir size
I'm using nmon to to monitor R/W disk speed, and I was surprised that read
speed of my shards volumes WDC20EAR's drive was nearly 3 MB/s when script is
working. After this I run benchmark test from disk utility. Here is results:
Minimum read rate: 53.2MB/s
Maximum Read rate: 126.4 MB/s
Average Read rate: 95.8 MB/s
But from the other side I tested queries like
http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(assistants)&rows=2000&start=0&fl=*,score
results is:
Elapsed time: 299 secs
--- solr ---
Queries processed: 50
Queries cancelled: 0
Average QTime is: 139.76 ms
Average RTime is: 2.2 sec(s)
Size of data-dir is: 6819259 bytes
and quesries like
http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(assistants)&rows=2000&start=0&fl=*,score&shards=127.0.0.1:8080/solr/shard1
and result is:
Elapsed time: 299 secs
--- solr ---
Queries processed: 49
Queries cancelled: 1
Average QTime is: 1878.37 ms
Average RTime is: 1.95918 sec(s)
Size of data-dir is: 4274099 bytes
So we see the results are the same.
My big question is: why is so slow drive read speed when Solr is working?
Thanks for any replies
P.S. And maybe my general problem is too much terms in shard, for example,
query
http://127.0.0.1:8080/solr/shard1/terms?terms.fl=field1
shows:
<lst name="field1">
<int name="a">58641</int>
<int name="the">45022</int>
<int name="i">36339</int>
<int name="s">35637</int>
<int name="d">34247</int>
<int name="m">33869</int>
<int name="b">28961</int>
<int name="r">28147</int>
<int name="e">27654</int>
<int name="n">26940</int>
</lst>
Thanks.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3590028.html
Sent from the Solr - User mailing list archive at Nabble.com.
