Just an observation... The OP is tweaking the heap size on post.jar, but wouldn't some tuning of the container that Solr is running in also be in order?
Michael Della Bitta ------------------------------------------------ Appinions, Inc. -- Where Influence Isn’t a Game. http://www.appinions.com On Tue, Jun 19, 2012 at 9:14 AM, François Schiettecatte <fschietteca...@gmail.com> wrote: > There is a lot of good information about that on the web, just google for > 'ubuntu performance monitor' > > Also the ubuntu website has a pretty good help section: > > https://help.ubuntu.com/ > > and a community wiki: > > https://help.ubuntu.com/community > > Cheers > > François > > On Jun 19, 2012, at 9:03 AM, Bruno Mannina wrote: > >> Linux Ubuntu :) since 2 months ! so I'm a new in this world :) >> >> Le 19/06/2012 15:01, François Schiettecatte a écrit : >>> Well that depends on the platform you are on, you did not mention that. >>> >>> If you are using linux, you could use atop ( http://www.atoptool.nl/ ), or >>> top, or iostat or stat, or all four. >>> >>> Cheers >>> >>> François >>> >>> On Jun 19, 2012, at 8:55 AM, Bruno Mannina wrote: >>> >>>> CPU is not used, just 50-60% sometimes during the process but How can I >>>> check IO HDD ? >>>> >>>> Le 19/06/2012 14:13, François Schiettecatte a écrit : >>>>> Just a suggestion, you might want to monitor CPU usage and disk I/O, >>>>> there might be a bottleneck. >>>>> >>>>> Cheers >>>>> >>>>> François >>>>> >>>>> On Jun 19, 2012, at 7:07 AM, Bruno Mannina wrote: >>>>> >>>>>> Actually -Xmx512m and no effect >>>>>> >>>>>> Concerning maxFieldLength, no problem it's commented >>>>>> >>>>>> Le 19/06/2012 13:02, Erick Erickson a écrit : >>>>>>> Then try -Xmx600M >>>>>>> next try -Xmx900M >>>>>>> >>>>>>> >>>>>>> etc. The idea is to bump things on separate runs. >>>>>>> >>>>>>> But be a little cautious here. Look in your solrconfig.xml file, you'll >>>>>>> see >>>>>>> a commented-out line >>>>>>> <maxFieldLength>10000</maxFieldLength> >>>>>>> >>>>>>> The default behavior for Solr/Lucene is to index the first 10,000 tokens >>>>>>> (not characters, think of tokens as words for not) in each >>>>>>> document and throw the rest on the floor. At the sizes you're talking >>>>>>> about, >>>>>>> that's probably not a problem, but do be aware of it. >>>>>>> >>>>>>> Best >>>>>>> Erick >>>>>>> >>>>>>> On Tue, Jun 19, 2012 at 5:44 AM, Bruno Mannina<bmann...@free.fr> >>>>>>> wrote: >>>>>>>> Like that? >>>>>>>> >>>>>>>> java -Xmx300m -jar post.jar myfile.xml >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Le 19/06/2012 11:11, Lance Norskog a écrit : >>>>>>>> >>>>>>>>> Ah! Java memory size is a java command line option: >>>>>>>>> >>>>>>>>> http://javahowto.blogspot.com/2006/06/6-common-errors-in-setting-java-heap.html >>>>>>>>> >>>>>>>>> You would try increasing the memory size in stages up to maybe 300m. >>>>>>>>> >>>>>>>>> On Tue, Jun 19, 2012 at 2:04 AM, Bruno Mannina<bmann...@free.fr> >>>>>>>>> wrote: >>>>>>>>>> Le 19/06/2012 10:51, Lance Norskog a écrit : >>>>>>>>>> >>>>>>>>>>> 675 doc/s is respectable for that server. You might move the memory >>>>>>>>>>> allocated to Java up and down- there is a balance between amount of >>>>>>>>>>> memory in Java v.s. the OS disk buffer. >>>>>>>>>> How can I do that ? is there an option during my command line or in a >>>>>>>>>> config >>>>>>>>>> file? >>>>>>>>>> sorry for this newbie question :( >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> And, of course, use the latest trunk. >>>>>>>>>> Solr 3.6 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Tue, Jun 19, 2012 at 12:10 AM, Bruno Mannina<bmann...@free.fr> >>>>>>>>>>> wrote: >>>>>>>>>>>> Correction: file size is 40 Mo !!! >>>>>>>>>>>> >>>>>>>>>>>> Le 19/06/2012 09:09, Bruno Mannina a écrit : >>>>>>>>>>>> >>>>>>>>>>>>> Dear All, >>>>>>>>>>>>> >>>>>>>>>>>>> I would like to know if the indexation speed is right. >>>>>>>>>>>>> >>>>>>>>>>>>> I have a 40Go file size with around 27 000 docs inside. >>>>>>>>>>>>> I index around 20 fields, >>>>>>>>>>>>> >>>>>>>>>>>>> My (old) test server is a DualCore 3.06GHz Intel Xeon with only >>>>>>>>>>>>> 1Go >>>>>>>>>>>>> Ram >>>>>>>>>>>>> >>>>>>>>>>>>> The file takes 40 seconds with the command line: >>>>>>>>>>>>> java -jar post.jar myfile.xml >>>>>>>>>>>>> >>>>>>>>>>>>> Could I increase this speed or reduce this time? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks a lot, >>>>>>>>>>>>> PS: Newbie user >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>> >>> >>> >> >
