Re: Reusable Performance Tests

2014-06-23 Thread Michael McCandless
The luceneutil module (https://code.google.com/a/apache-extras.org/p/luceneutil/ ) has benchmarking code for indexing; it's what I use to generate Lucene's nightly performance graphs (http://people.apache.org/~mikemccand/lucenebench/indexing.html ). But it's somewhat involved to get it set up ...

Re: Reusable Performance Tests

2014-06-23 Thread Gaurav gupta
Srividhya, I am also looking something similar. I will try if I can find something. Thanks On Jun 20, 2014 12:50 PM, "Umashanker, Srividhya" < srividhya.umashan...@hp.com> wrote: > Are there any performance test suites available in lucene codebase which > can be reused by us to benchmark against

Re: Facet migration 4.6.1 to > 4.7.0

2014-06-23 Thread Nicola Buso
Hi, On Tue, 2014-06-17 at 17:51 +0300, Shai Erera wrote: > - we are extending FacetResultsHandler to change the order of > the facet > results (i.e. date facets ordered by date instead of count). > How can I > achieve this now? > > > Now everything is a Fa

Re: EarlyTerminatingSortingCollector help needed..

2014-06-23 Thread Ravikumar Govindarajan
> > This means that even though you have eg. 15 segments, if you requested > 50 documents, you will get the top 50 documents out of your > TopHitsCollector. Yes, we can get the top-50 docs finally. I am not denying that. I will probably re-phrase my question. Apologize if I am not clear How do

Re: Lucene Facets Module 4.8.1

2014-06-23 Thread Jigar Shah
Thanks very much for this valuable information. Good to know that, same indexFieldName can be used for multiple (similar in some cases) dimensions. For sure this will help me to design application better. Thanks once again. On Mon, Jun 23, 2014 at 7:00 PM, Shai Erera wrote: > Basically, it's

Re: Lucene Facets Module 4.8.1

2014-06-23 Thread Shai Erera
Basically, it's not very common to change the indexFieldName. You should do that in case you e.g. count facets in groups of dimensions, rather than counting all of them. So for example, if you have 20 dimensions, but you know you only count d1-d5, d6-d12 and d13-d20, then if you separate them to 3

Re: Lucene Facets Module 4.8.1

2014-06-23 Thread Jigar Shah
Thanks this worked for me :) Is there any advantage of indexing some facets as not providing any indexFieldName ? Thanks On Mon, Jun 23, 2014 at 12:55 PM, Shai Erera wrote: > There is no sample code for doing that but it's quite straightforward - if > you know you indexed some dimensions un

Re: frozen in PriorityQueue.downHeap for more than 25 minutes

2014-06-23 Thread Toke Eskildsen
On Mon, 2014-06-23 at 13:58 +0200, Jamie wrote: > How does one sort the results of a collector as opposed to the entire > result set? With only 50K as page size, this should not be necessary. But for the record, you do it by implementing a Collector that can potentially hold all documents in the

Re: frozen in PriorityQueue.downHeap for more than 25 minutes

2014-06-23 Thread Jamie
Toke On 2014/06/23, 2:08 PM, Toke Eskildsen wrote: On Mon, 2014-06-23 at 13:53 +0200, Jamie wrote: if (startIdx==0) { topDocs = indexSearcher.search(query,queryFilter,searchResult.getPageSize(), sort); } else { topDocs = indexSearcher.searchAfter(p.startScoreDoc, query, queryFilter, se

Re: frozen in PriorityQueue.downHeap for more than 25 minutes

2014-06-23 Thread Toke Eskildsen
On Mon, 2014-06-23 at 13:53 +0200, Jamie wrote: > if (startIdx==0) { >topDocs = > indexSearcher.search(query,queryFilter,searchResult.getPageSize(), sort); > } else { >topDocs = indexSearcher.searchAfter(p.startScoreDoc, query, > queryFilter, searchResult.getPageSize(),sort); > } > The pa

Re: EarlyTerminatingSortingCollector help needed..

2014-06-23 Thread Adrien Grand
On Sun, Jun 22, 2014 at 6:44 PM, Ravikumar Govindarajan wrote: > For a normal sorting-query, on a top-level searcher, I execute > > TopDocs docs = searcher.search(query, 50, sortField) > > Then I can issue reader.document() for final list of exactly 50 docs, which > gives me a global order across

Re: frozen in PriorityQueue.downHeap for more than 25 minutes

2014-06-23 Thread Jamie
Toke How does one sort the results of a collector as opposed to the entire result set? Do I need to implement my own sort algorithm or is there a way to do this with Lucene? If so, which API functions do I need to call? Thanks Jamie On 2014/06/23, 1:43 PM, Toke Eskildsen wrote: On Mon,

Re: frozen in PriorityQueue.downHeap for more than 25 minutes

2014-06-23 Thread Jamie
Toke Thanks for the tip. Sadly, we are only requesting a set page size worth of documents at a time. if (startIdx==0) { topDocs = indexSearcher.search(query,queryFilter,searchResult.getPageSize(), sort); } else { topDocs = indexSearcher.searchAfter(p.startScoreDoc, query, queryFilter, s

Re: frozen in PriorityQueue.downHeap for more than 25 minutes

2014-06-23 Thread Toke Eskildsen
On Mon, 2014-06-23 at 13:33 +0200, Jamie wrote: > While running a search over several million documents, the Yourkit > profiler reports a deadlock on the following method. Any ideas? > search worker <--- Frozen for at least 25m 37 sec > org.apache.lucene.util.PriorityQueue.downHeap() My guess is

frozen in PriorityQueue.downHeap for more than 25 minutes

2014-06-23 Thread Jamie
Hi While running a search over several million documents, the Yourkit profiler reports a deadlock on the following method. Any ideas? search worker <--- Frozen for at least 25m 37 sec org.apache.lucene.util.PriorityQueue.downHeap() org.apache.lucene.util.PriorityQueue.updateTop() org.apache.lu

Re: Lucene Facets Module 4.8.1

2014-06-23 Thread Shai Erera
There is no sample code for doing that but it's quite straightforward - if you know you indexed some dimensions under different indexFieldNames, initialize a FacetCounts per such field name, e.g.: FastTaxoFacetCounts defaultCounts = new FastTaxoFacetCounts(...); // for your regular facets FastTaxo