AW: Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Clemens Wyss DEV
Thanks for the input. In fact, we are using this very query as a Solr queryFilter (i.e. qf). Is there a specific (Lucene query) syntax to formulate (force the usage of) such a FieldValueFilter? -Ursprüngliche Nachricht- Von: Ahmet Arslan [mailto:iori...@yahoo.com.INVALID] Gesendet:

RE: Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Uwe Schindler
Hi, In Lucene you don't need to use a query parser for that, especially because range Queries is suboptimal and slow: There is already a very fast query/filter available. Ahmet Arslan already mentioned that, we had the same discussion a few weeks ago:

RE: Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Chris Hostetter
: In Lucene you don't need to use a query parser for that, especially : because range Queries is suboptimal and slow: There is already a very : fast query/filter available. Ahmet Arslan already mentioned that, we had : the same discussion a few weeks ago: :

Re: Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Jack Krupansky
Oops... I take that back! After I clicked Send I realized that this is the Lucene list - what I said is true for Solr queries, but that is because Solr added a hack to do things properly, but the Lucene query parser doesn't have that hack, so Erick is correct. -- Jack Krupansky On Wed, Jan 7,

A question on performance

2015-01-07 Thread rama44ster
Hi, I have a lucene index which has close to 480M documents. And I ran around 1000 queries against the index. Each query is a boolean query with 3 different tokens. That is the query has 3 operands which MUST occur. Executing such 3 token queries gives the following latency percentiles. 50 = 16

Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Clemens Wyss DEV
Say I wanted to find documents which have no content in field1 (or dosuments that have no field 'field1'), wouldn't that be the following query? -field1:[* TO *] Thanks for you help Clemens - To unsubscribe, e-mail:

RE: A question on performance

2015-01-07 Thread Toke Eskildsen
rama44ster [rama44s...@gmail.com] wrote: [3 MUST clauses] 50 = 16 ms 75 = 52 ms 90 = 121 ms 95 = 262 ms 99 = 76010 ms 99.9 = 76037 ms Is the latency expected to degrade when the number of docs is as high as 480M? Try plotting response times as a function of hit count. My guess is that

Re: A question on performance

2015-01-07 Thread Arvind Kalyan
Performance measurements must be made carefully. Have you performed any warmup? I recommend doing 10k calls just to let the dust settle including stuff like jit, before taking any kind if measurements. Also use mmapdirectory, if not already, to help with spikes in disk accesses. Also keep track

Re: Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Erick Erickson
Should be, but it's a bit confusing because the query syntax is not pure boolean, so there's no set to take away the docs with entries in field 1, you need the match all docs bit, i.e. *:* -field1:[* TO *] (That's asterisk:asterisk -field1:[* TO *] in case the silly list interprets the asterisks

AW: Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Clemens Wyss DEV
Thx! -Ursprüngliche Nachricht- Von: Erick Erickson [mailto:erickerick...@gmail.com] Gesendet: Mittwoch, 7. Januar 2015 17:31 An: java-user Betreff: Re: Looking for docs that have certain fields empty (an/or not set) Should be, but it's a bit confusing because the query syntax is not

Re: Looking for docs that have certain fields empty (an/or not set)

2015-01-07 Thread Ahmet Arslan
Hi Clemens, Since you are a lucene user, you might be interested in Uwe's response on a similar topic : http://find.searchhub.org/document/abb73b45a48cb89e Ahmet On Wednesday, January 7, 2015 6:30 PM, Erick Erickson erickerick...@gmail.com wrote: Should be, but it's a bit confusing because