Re: It cost some many memory with solrj 3.5 how to decrease it?
Study the update examination more deeply,i logged all elapsetime value of Updateresponse, the result list following: It seems that it spent almost 20 ms on adding/updating one document in general, thus, i called which spend less than 20ms on adding one docs as normal log,and the others were abnormal logs. i can`t get a correct suit of solr 1.4, i use solr3.2 which has same performance as solr 1.4 during the test. solr3.5 vs solr 3.2 solr3.5 sum of docs:31998 sum of elapsetime:1218344 ms average: 38.0744 ms /doc sum of normal docs:28409 sum of normal elapsetime:442258 average=15.5675 ms/doc normal percentage:28409/31998 = 88.78% abnormal docs: 3590 solr 3.2 sum of docs:31998 sum of elapsetime:852935 ms average:26.6559 ms /doc sum of normal docs:28416 sum of normal elapsetime:443045 average=15.5914 ms/doc normal percentage:28409/31998 = 88.80% abnormal docs: 3160 What can be analyzed from them? B.R. murphy On Fri, Apr 6, 2012 at 10:28 AM, a sd liurx.cn@gma il.com wrote: hi,Erick. thanks at first. I had watched the status of JVM at runtime helped by jconsole and jmap. 1,When the Xmx was not assigned, then, the Old Gen area was full whose size was up to 1.5Gb and whose major content are instances of String , when the whole size of heap was up to the maximum ( about 2GB), the JVM run gc() ,which wasted the CPU time,then, the performance was degraded sharply, which was from 100,000 docs per minute to 10,000 docs per minute, as a examination, i assigned Xmx=1024m purposely, the amount was down to 1000 docs per minute. 2,When assigned Xmx=4096m, i found that the Old Gen was up to 2.1 GB and the all size of JVM was up to 3GB, but, the performance with 100,000 docs per minute can attained. During all of the test above, i only adjust the setting of client, which connect to the identical solr server and i empty the data directory of solr home before every test. By the way, i know the client code was very ugly occupied so many heap too, but, i wan`t permitted to promote them before i obtain a benchrank using solrj 3.5 as much as which the old version did using solrj 1.4. B.R murphy On Fri, Apr 6, 2012 at 5:54 AM, Erick Erickson erickerick...@gmail.comwrote: What's memory? Really, how are you measuring it? If it's virtual, you don't need to worry about it. Is this causing you a real problem or are you just nervous about the difference? Best Erick On Wed, Apr 4, 2012 at 11:23 PM, a sd liurx...@gmail.com wrote: hi,all. I have write a program which send data to solr using the update request handler, when i adopted server client library ( namely solrj ) with version 4.0 or 3.2, jvm`s heap size was up to 1.0 G about, but ,when i transfer the all of them to solr 3.5 ( both server and client libs), the size of heap was top to 3.0G ! There are the same server configuration and the identical program. What`s wrong with the new version of solrj 3.5 , i had looked the source code, there is no difference between solrj 3.2 and solrj 3.5 where my program may invoke. How can i do to decrease the memory cost by solrj 3.5? Any advice will be appreciated! murphy
Re: It cost some many memory with solrj 3.5 how to decrease it?
What's memory? Really, how are you measuring it? If it's virtual, you don't need to worry about it. Is this causing you a real problem or are you just nervous about the difference? Best Erick On Wed, Apr 4, 2012 at 11:23 PM, a sd liurx...@gmail.com wrote: hi,all. I have write a program which send data to solr using the update request handler, when i adopted server client library ( namely solrj ) with version 4.0 or 3.2, jvm`s heap size was up to 1.0 G about, but ,when i transfer the all of them to solr 3.5 ( both server and client libs), the size of heap was top to 3.0G ! There are the same server configuration and the identical program. What`s wrong with the new version of solrj 3.5 , i had looked the source code, there is no difference between solrj 3.2 and solrj 3.5 where my program may invoke. How can i do to decrease the memory cost by solrj 3.5? Any advice will be appreciated! murphy
Re: It cost some many memory with solrj 3.5 how to decrease it?
hi,Erick. thanks at first. I had watched the status of JVM at runtime helped by jconsole and jmap. 1,When the Xmx was not assigned, then, the Old Gen area was full whose size was up to 1.5Gb and whose major content are instances of String , when the whole size of heap was up to the maximum ( about 2GB), the JVM run gc() ,which wasted the CPU time,then, the performance was degraded sharply, which was from 100,000 docs per minute to 10,000 docs per minute, as a examination, i assigned Xmx=1024m purposely, the amount was down to 1000 docs per minute. 2,When assigned Xmx=4096m, i found that the Old Gen was up to 2.1 GB and the all size of JVM was up to 3GB, but, the performance with 100,000 docs per minute can attained. During all of the test above, i only adjust the setting of client, which connect to the identical solr server and i empty the data directory of solr home before every test. By the way, i know the client code was very ugly occupied so many heap too, but, i wan`t permitted to promote them before i obtain a benchrank using solrj 3.5 as much as which the old version did using solrj 1.4. B.R murphy On Fri, Apr 6, 2012 at 5:54 AM, Erick Erickson erickerick...@gmail.comwrote: What's memory? Really, how are you measuring it? If it's virtual, you don't need to worry about it. Is this causing you a real problem or are you just nervous about the difference? Best Erick On Wed, Apr 4, 2012 at 11:23 PM, a sd liurx...@gmail.com wrote: hi,all. I have write a program which send data to solr using the update request handler, when i adopted server client library ( namely solrj ) with version 4.0 or 3.2, jvm`s heap size was up to 1.0 G about, but ,when i transfer the all of them to solr 3.5 ( both server and client libs), the size of heap was top to 3.0G ! There are the same server configuration and the identical program. What`s wrong with the new version of solrj 3.5 , i had looked the source code, there is no difference between solrj 3.2 and solrj 3.5 where my program may invoke. How can i do to decrease the memory cost by solrj 3.5? Any advice will be appreciated! murphy
