Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-24 Thread T. Kuro Kurosaka
On 3/19/14 5:13 PM, Otis Gospodnetic wrote: Hi, Guessing it's surround query parser's support for within backed by span queries. Otis You mean this? http://wiki.apache.org/solr/SurroundQueryParser I guess this parser needs improvement in documentation area. It doesn't explain or have an

Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-24 Thread Otis Gospodnetic
I think SQP is getting axed, no? Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch Support * http://sematext.com/ On Mon, Mar 24, 2014 at 3:45 PM, T. Kuro Kurosaka k...@healthline.comwrote: On 3/19/14 5:13 PM, Otis Gospodnetic wrote: Hi, Guessing it's

Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-24 Thread Ahmet Arslan
Hi, There is no w/int syntax in surround.  /* Query language operators: OR, AND, NOT, W, N, (, ), ^, *, ?, and comma */ Ahmet On Monday, March 24, 2014 9:46 PM, T. Kuro Kurosaka k...@healthline.com wrote: On 3/19/14 5:13 PM, Otis Gospodnetic wrote: Hi, Guessing it's surround query parser's

Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-24 Thread Walter Underwood
That is similar to Verity VQL, but that used NEAR/10. --wunder On Mar 24, 2014, at 4:21 PM, Ahmet Arslan iori...@yahoo.com wrote: Hi, There is no w/int syntax in surround. /* Query language operators: OR, AND, NOT, W, N, (, ), ^, *, ?, and comma */ Ahmet On Monday, March 24,

Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-24 Thread Salman Akram
Basically we just created this syntax for the ease of users, otherwise on back end it uses W or N operators. On Tue, Mar 25, 2014 at 4:21 AM, Ahmet Arslan iori...@yahoo.com wrote: Hi, There is no w/int syntax in surround. /* Query language operators: OR, AND, NOT, W, N, (, ), ^, *, ?, and

Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-24 Thread Roman Chyla
perhaps useful, here is an open source implementation with near[digit] support, incl analysis of proximity tokens. When days become longer maybe itwill be packaged into a nice lib...:-) https://github.com/romanchyla/montysolr/blob/master/contrib/adsabs/grammars/ADS.g On 25 Mar 2014 00:14, Salman

Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-20 Thread Salman Akram
...@healthline.com wrote: In the thread Partial Counts in SOLR, Salman gave us this sample query: ((stock or share*) w/10 (sale or sell* or sold or bought or buy* or purchase* or repurchase*)) w/10 (executive or director) I'm not familiar with this w/10 notation. What does this mean, and what

Re: Partial Counts in SOLR

2014-03-19 Thread Salman Akram
Anyone? On Mon, Mar 17, 2014 at 12:03 PM, Salman Akram salman.ak...@northbaysolutions.net wrote: Below is one of the sample slow query that takes mins! ((stock or share*) w/10 (sale or sell* or sold or bought or buy* or purchase* or repurchase*)) w/10 (executive or director) If a filter

Re: Partial Counts in SOLR

2014-03-19 Thread Erick Erickson
Yes, that'll be slow. Wildcards are, at best, interesting and at worst resource consumptive. Especially when you're doing this kind of positioning information as well. Consider looking at the problem sideways. That is, what is your purpose in searching for, say, buy*? You want to find buy,

Re: Partial Counts in SOLR

2014-03-19 Thread Salman Akram
This was one example. Users can even add phrase searches with wildcards/proximity etc so can't really use stemming. Sharding is definitely something we are already looking into. On Wed, Mar 19, 2014 at 6:59 PM, Erick Erickson erickerick...@gmail.comwrote: Yes, that'll be slow. Wildcards are,

w/10 ? [was: Partial Counts in SOLR]

2014-03-19 Thread T. Kuro Kurosaka
In the thread Partial Counts in SOLR, Salman gave us this sample query: ((stock or share*) w/10 (sale or sell* or sold or bought or buy* or purchase* or repurchase*)) w/10 (executive or director) I'm not familiar with this w/10 notation. What does this mean, and what parser(s) supports

Re: w/10 ? [was: Partial Counts in SOLR]

2014-03-19 Thread Otis Gospodnetic
Hi, Guessing it's surround query parser's support for within backed by span queries. Otis Solr ElasticSearch Support http://sematext.com/ On Mar 19, 2014 4:44 PM, T. Kuro Kurosaka k...@healthline.com wrote: In the thread Partial Counts in SOLR, Salman gave us this sample query: ((stock

Re: Partial Counts in SOLR

2014-03-17 Thread Salman Akram
Below is one of the sample slow query that takes mins! ((stock or share*) w/10 (sale or sell* or sold or bought or buy* or purchase* or repurchase*)) w/10 (executive or director) If a filter is used it comes in fq but what can be done about plain keyword search? On Sun, Mar 16, 2014 at 4:37

Re: Partial Counts in SOLR

2014-03-15 Thread Erick Erickson
What are our complex queries? You say that your app will very rarely see the same query thus you aren't using caches... But, if you can move some of your clauses to fq clauses, then the filterCache might well be used to good effect. On Thu, Mar 13, 2014 at 7:22 AM, Salman Akram

Re: Partial Counts in SOLR

2014-03-13 Thread Salman Akram
Well some of the searches take minutes. Below are some stats about this particular index that I am talking about: Index size = 400GB (Using CommonGrams so without that the index is around 180GB) Position File = 280GB Total Docs = 170 million (just indexed for searching - for highlighting

Re: Partial Counts in SOLR

2014-03-13 Thread Dmitry Kan
1. What is your solr version? In 4.x family the proximity searches have been optimized among other query types. 2. Do you use the filter queries? What is the situation with the cache utilization ratios? Optimize (= i.e. bump up the respective cache sizes) if you have low hitratios and many

Re: Partial Counts in SOLR

2014-03-13 Thread Salman Akram
1- SOLR 4.6 2- We do but right now I am talking about plain keyword queries just sorted by date. Once this is better will start looking into caches which we already changed a little. 3- As I said the contents are not stored in this index. Some other metadata fields are but with normal queries its

Re: Partial Counts in SOLR

2014-03-12 Thread Dmitry Kan
As Hoss pointed out above, different projects have different requirements. Some want to sort by date of ingestion reverse, which means that having posting lists organized in a reverse order with the early termination is the way to go (no such feature in Solr directly). Some other projects want to

Re: Partial Counts in SOLR

2014-03-11 Thread Salman Akram
Its a long video and I will definitely go through it but it seems this is not possible with SOLR as it is? I just thought it would be quite a common issue; I mean generally for search engines its more important to show the first page results, rather than using timeAllowed which might not even

Re: Partial Counts in SOLR

2014-03-10 Thread Dmitry Kan
Salman, It looks like what you describe has been implemented at Twitter. Presentation from the recent Lucene / Solr Revolution conference in Dublin: http://www.youtube.com/watch?v=AguWva8P_DI On Sat, Mar 8, 2014 at 4:16 PM, Salman Akram salman.ak...@northbaysolutions.net wrote: The issue

Re: Partial Counts in SOLR

2014-03-08 Thread Salman Akram
The issue with timeallowed is you never know if it will return minimum amount of docs or not. I do want docs to be sorted based on date but it seems its not possible that solr starts searching from recent docs and stops after finding certain no. of docs...any other tweak? Thanks On Saturday,

Partial Counts in SOLR

2014-03-07 Thread Salman Akram
All, Is it possible to get partial counts in SOLR? The idea is to get the count but if its above a certain limit than just return that limit. Reason: In an index with millions of documents I don't want to know that a certain query matched 1 million docs (of course it will take time to calculate

Re: Partial Counts in SOLR

2014-03-07 Thread Gora Mohanty
On 7 March 2014 15:18, Salman Akram salman.ak...@northbaysolutions.net wrote: All, Is it possible to get partial counts in SOLR? The idea is to get the count but if its above a certain limit than just return that limit. Reason: In an index with millions of documents I don't want to know

Re: Partial Counts in SOLR

2014-03-07 Thread Dmitry Kan
, Is it possible to get partial counts in SOLR? The idea is to get the count but if its above a certain limit than just return that limit. Reason: In an index with millions of documents I don't want to know that a certain query matched 1 million docs (of course it will take time to calculate

Re: Partial Counts in SOLR

2014-03-07 Thread Salman Akram
not sure how it works in SOLR... On Fri, Mar 7, 2014 at 3:17 PM, Gora Mohanty g...@mimirtech.com wrote: On 7 March 2014 15:18, Salman Akram salman.ak...@northbaysolutions.net wrote: All, Is it possible to get partial counts in SOLR? The idea is to get the count but if its above a certain

Re: Partial Counts in SOLR

2014-03-07 Thread Chris Hostetter
: Reason: In an index with millions of documents I don't want to know that a : certain query matched 1 million docs (of course it will take time to : calculate that). Why don't just stop looking for more results lets say : after it finds 100 docs? Possible?? but if you care about sorting, ie: