I have switched entirely to using Katta in an elastic computing setting. This allows me to have very fine grained control over the hardware size/response time trade-off. Katta can easily incorporate update shards or broadcast deletions. In the elastic setting, I also don't have to worry about deploying a new full index since I can just allocate a secondary search cluster and flip the load balancer to it when it is ready. By having a moderately large number (10-25) of index shards per compute node, I can vary the response time in very fine increments. Throughput can be adjusted the same way. On a 300G index, I easily get 10 requests per second with sub-second response time.
On Thu, Apr 2, 2009 at 8:09 AM, sunnyfr <[email protected]> wrote: > > Hi, > > My target would be to have 20req/sec, I just don't get what's wrong there : > at the very start it's very fast then it's getting very slow...? and I > can't > have more than 6.3req/sec > > Apr 2 17:05:46 search-06 jsvc.exec[6423]: 358.744: [GC > Apr 2 17:05:46 search-06 jsvc.exec[6423]: 358.744: [ParNew > Apr 2 17:05:46 search-06 jsvc.exec[6423]: : 69648K->1796K(76672K), > 0.0119440 secs] 3603202K->3535363K(4185792K), 0.0120160 secs] [Times: > user=0.04 sys=0.00, real=0.01 secs] > > Apr 2 17:05:47 search-06 jsvc.exec[6423]: 359.022: [GC > Apr 2 17:05:47 search-06 jsvc.exec[6423]: 359.022: [ParNew > Apr 2 17:05:47 search-06 jsvc.exec[6423]: 69956K->1898K(76672K), > 0.0175830 > secs] 3603523K->3535539K(4185792K), 0.0176580 secs] [Times: user=0.03 > sys=0.01, real=0.02 secs] > > This is my conf: (ive 8G ram) my index is just 11G > > JAVA_OPTS="-Xmx4500M -Xms4G > -XX:+UseConcMarkSweepGC \ > -XX:+CMSClassUnloadingEnabled \ > -XX:-CMSParallelRemarkEnabled \ > -XX:+UseCMSCompactAtFullCollection \ > -XX:+UseParNewGC \ > -Xloggc:\data\gc.log -XX:+PrintGCDetails -verbose:gc" > > > Any suggestion would be excellent, > Thanks > -- > View this message in context: > http://www.nabble.com/GC-configuration-tp22849848p22849848.html > Sent from the Lucene - General mailing list archive at Nabble.com. > > -- Ted Dunning, CTO DeepDyve 111 West Evelyn Ave. Ste. 202 Sunnyvale, CA 94086 www.deepdyve.com 858-414-0013 (m) 408-773-0220 (fax)
