Re: Binding lucene instance/threads to a particular processor(or core)

ould suggest having adding the ability to set the > > > > ThreadPoolExecutor completely, with a getter/setter. This would allow > > > > this class to be useful beyond the use case of multiple indexes, > > > > becoming more generalizable to a number of use ca

Re: Binding lucene instance/threads to a particular processor(or core)

xternally (is null). > > > > > > -Glen > > > > > > 2008/4/22 Renaud Waldura <[EMAIL PROTECTED]>: > > > > > one solution is to set-up a ThreadPoolExecutor[2] with a fixed > > > > > number of threads and a limited queue siz

Re: Binding lucene instance/threads to a particular processor(or core)

(use a bound > > BlockingQueue[3]) > > > > > > Yes, this is precisely how the ConcurrentMultiSearcher works. > > > https://issues.apache.org/jira/browse/LUCENE-423 > > > > > > > > > > > > > > > -Original Messa

Re: Binding lucene instance/threads to a particular processor(or core)

On Tue, 2008-04-22 at 09:40 +0530, Anshum wrote: > Any other suggestions for handling a concurrency of over 7 search requests > per second for an index size of over 15Gigs containing over 13 million > records? Our index is 30GB+ with 9 million records and a machine handles an average search in abo

Re: Binding lucene instance/threads to a particular processor(or core)

ira/browse/LUCENE-423 > > > > > > > > > > -Original Message----- > > From: Glen Newton [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, April 22, 2008 5:40 AM > > To: java-user@lucene.apache.org > > Subject: Re: Binding lucene instanc

RE: Binding lucene instance/threads to a particular processor(or core)

LUCENE-423 > > > > > -Original Message- > From: Glen Newton [mailto:[EMAIL PROTECTED] > Sent: Tuesday, April 22, 2008 5:40 AM > To: java-user@lucene.apache.org > Subject: Re: Binding lucene instance/threads to a particular > processor(or > core) >

Re: Binding lucene instance/threads to a particular processor(or core)

Original Message- > From: Glen Newton [mailto:[EMAIL PROTECTED] > Sent: Tuesday, April 22, 2008 5:40 AM > To: java-user@lucene.apache.org > Subject: Re: Binding lucene instance/threads to a particular processor(or > core) > > > > Anshun, > > I think I am dealing

RE: Binding lucene instance/threads to a particular processor(or core)

: Glen Newton [mailto:[EMAIL PROTECTED] Sent: Tuesday, April 22, 2008 5:40 AM To: java-user@lucene.apache.org Subject: Re: Binding lucene instance/threads to a particular processor(or core) Anshun, I think I am dealing with an index of similar scale: 6.4 million records, 83 GB index (see [1] for mor

RE: Binding lucene instance/threads to a particular processor(or core)

@lucene.apache.org Subject: Re: Binding lucene instance/threads to a particular processor(or core) The paper seems pretty good but I am still wondering if there was a way to achieve this through the command line parameters. I'm just trying this to optimize the code, if this works, would let all know else

Re: Binding lucene instance/threads to a particular processor(or core)

Anshun, I think I am dealing with an index of similar scale: 6.4 million records, 83 GB index (see [1] for more info) I mistakenly thought from your original posting that you were interested in binding threads to processors for indexing, but it is sounding like you want to do this for searching.

Re: Binding lucene instance/threads to a particular processor(or core)

The paper seems pretty good but I am still wondering if there was a way to achieve this through the command line parameters. I'm just trying this to optimize the code, if this works, would let all know else would keep everyone informed :) Any other suggestions for handling a concurrency of over 7 s

Re: Binding lucene instance/threads to a particular processor(or core)

That paper from 1997 is pretty old, but mirrors our experiences in those days. Then, we used Solaris processor sets to really improve performance by binding one of our processes to a particular CPU while leaving the other CPUs to manage the thread intensive work. You can bind processes/LWPs to

Re: Binding lucene instance/threads to a particular processor(or core)

I realised that not everyone on this list might be able to access the IEEE paper I pointed-out, so I will include the abstract and some paragraphs from the paper which I have included below. Also of interest (and should be available to all): Fedorova et al. 2005. Performance of Multithreaded Chip

Re: Binding lucene instance/threads to a particular processor(or core)

And this discussion on bound threads may also shed light on things: http://coding.derkeiler.com/Archive/Java/comp.lang.java.programmer/2007-11/msg02801.html -Glen On 21/04/2008, Glen Newton <[EMAIL PROTECTED]> wrote: > BInding threads to processors - in many situations - improves > throughput by

Re: Binding lucene instance/threads to a particular processor(or core)

BInding threads to processors - in many situations - improves throughput by reducing memory overhead. When a thread is running on a core, its state is local; if it is timeshared-out and either 1) swapped back in on the same core, it is likely that there will be the core's L1 cache; or 2) onto anot

Re: Binding lucene instance/threads to a particular processor(or core)

This sounds odd. Why would restricting it to a single core improve performance? The point of using multiple cores (and multiple threads) is to improve performance isn't it? I'd leave thread scheduling decisions to the JVM. Plus, I don't think there is anything in Java to facilitate this (short of u