I've tried to run different shards from different machine, and there is a slight improvement in the performance (about 3 mins faster for 1GB worth of data, from 22 mins to 19 mins).
Is this a normal scenario? Both of my machine are running on Intel i7 core. Regards, Edwin On 21 September 2015 at 16:24, Zheng Lin Edwin Yeo <edwinye...@gmail.com> wrote: > I'm not sure if that is because currently my machine is a normal PC and > not a server, but my CPU specification for each of the core is Intel(R) > Core(TM) i7-4910MQ CPU @ 2.90GHz. > > It should probably be better when the real server which has a much better > specification comes, and I should be able to do the indexing in a lesser > time using the knowledge that I've learnt here. > > > Regards, > Edwin > > > > On 21 September 2015 at 16:00, Toke Eskildsen <t...@statsbiblioteket.dk> > wrote: > >> On Mon, 2015-09-21 at 10:13 +0800, Zheng Lin Edwin Yeo wrote: >> > I didn't find any increase in indexing throughput by adding shards in >> the >> > same machine. >> > >> > However, I've managed to feed the index to Solr from more than one >> thread >> > at a time. It can take up to 3 threads without affecting the indexing >> > speed. Anything more than that, the CPU will hit 100%, and the indexing >> > speed in all the threads will be reduced. >> >> It is a bit surprising that the limit is 3 Threads on an 8 core machine, >> but I am happy to hear that your findings fit the overall theory. >> >> >> Thank you for the verification, >> Toke Eskildsen, State and University Library, Denmark >> >> >> >
