Re: SolrCore.getSearcher() and postCommit()

2010-10-30 Thread Grant Ingersoll
On Oct 29, 2010, at 5:48 PM, Yonik Seeley wrote: On Fri, Oct 29, 2010 at 5:36 PM, Grant Ingersoll gsing...@apache.org wrote: Is it OK to call and increment a Searcher ref (i.e. SolrCore.getSearcher()) in a SolrEventListener.postCommit() hook as long as I decrement it when I am done? I

Re: Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-30 Thread Grant Ingersoll
On Oct 29, 2010, at 6:00 PM, Ron Mayer wrote: I have some documents with a bunch of attachments (images, thumbnails for them, audio clips, word docs, etc); and am currently dealing with them by just putting a path on a filesystem to them in solr; and then jumping through hoops of keeping

Re: Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-30 Thread Paul Libbrecht
I am quite interested by this story, including sample code. Back in Lucene 1.4 and 2.0 times, the reader vs string loading abilities was inconsistently handled and I switched to have one directory with thousands of files for our ActiveMath content storage. It works but fairly badly on smaller

solr 4.0 - pagination

2010-10-30 Thread Papp Richard
Hi all, I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I can't solve the pagination. Mainly there are two problems: - the query fields start rows doesn't work anymore - beside of the values, it always returns the data as the start would be 0 (start =

Re: solr 4.0 - pagination

2010-10-30 Thread Yonik Seeley
On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard ccode...@gmail.com wrote:  I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I can't solve the pagination. It's not implemented yet, but I'm working on that right now. -Yonik http://www.lucidimagination.com

RE: solr 4.0 - pagination

2010-10-30 Thread Papp Richard
Can you estimate please when it will be done? thanks, Rich -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Saturday, October 30, 2010 19:29 To: solr-user@lucene.apache.org Subject: Re: solr 4.0 - pagination On Sat, Oct 30, 2010 at

Re: solr 1.4 suggester component

2010-10-30 Thread Erick Erickson
Sorry it took so long to respond, I got wrapped up in work. I don't think you're going to be able to pull this off (but of course I've been wrong before). File based suggestions look like they do an awful lot of stuff for you, at some expense of customization (no surprise here). That said,

Basic Document Question

2010-10-30 Thread Eric Martin
HI everyone, I'm new which won't be hard to figure out after I ask this question: I use Drupal/Solr/Nutch http://svn.apache.org/viewvc/lucene/dev/trunk/solr/example/solr/conf/schema. xml?view=markup Solr specific: How do I re-index for specific content only? I am starting a legal

Re: Modelling Access Control

2010-10-30 Thread Erick Erickson
If that's in response to Lance's comment, the answer is that if you return autosuggest possibilities you effectively allow users to see data they shouldn't. Imagine you have a field of the real names of spies. You only want the persons way high up in the security chain to access these names and

Re: Highlighting for non-stored fields

2010-10-30 Thread Erick Erickson
Also, consider what you'd be reconstructing from if you could try it. The indexed data has been transformed by, say, stemming, casing, etc. So any attempt to reconstruct the fields for highlighting would necessarily show the transformed version, which would not be pleasing. Plus you could have

Re: Strange search

2010-10-30 Thread Erick Erickson
What does the admin page say is in your text field (see solr/admin, the schema browser link Also, see what returns if you add debugQuery=on to your URL, that often is useful... HTH Erick On Wed, Oct 27, 2010 at 3:53 AM, ramzesua michaelnaza...@gmail.com wrote: Can anyone give me working

Re: question about SolrCore

2010-10-30 Thread Erick Erickson
Why do you want to know? If there is a specific problem you're trying to solve, perhaps stating the problem itself will get you a better response. Best Erick On Thu, Oct 28, 2010 at 4:00 AM, Li Li fancye...@gmail.com wrote: is there anyone could help me? 2010/10/11 Li Li fancye...@gmail.com:

Re: solr stuck in writing to inexisting sockets

2010-10-30 Thread Erick Erickson
Are you saying that your Solr server is at times taking 5 minutes to complete? If so, I'd get to the bottom of that first off. My first guess would be you're either hitting memory issues and swapping horribly or..well, that would be my first guess. Best Erick On Thu, Oct 28, 2010 at 5:23 AM,

Re: No response from Solr on complex request (real issue explained)

2010-10-30 Thread Erick Erickson
This smells like a garbage collection issue. Here's an article from about a year ago by Mark Miller that might help: http://www.lucidimagination.com/blog/2009/09/19/java-garbage-collection-boot-camp-draft/

Re: Commit/Optimise question

2010-10-30 Thread Erick Erickson
What version of Solr are you using? About committing. I'd just let the solr defaults handle that. You configure this in the autocommit section of solrconfig.xml. I'm pretty sure this gets triggered even if you're using SolrJ. That said, it's probably wise to issue a commit after all your data

Re: Consulting in Solr tuning, stop words, dictionary, etc

2010-10-30 Thread Erick Erickson
Well, that all depends on what you want. Here's a list of Solr consultants. http://wiki.apache.org/solr/Support http://wiki.apache.org/solr/SupportHTH Erick On Thu, Oct 28, 2010 at 4:21 PM, Dennis Gearon gear...@sbcglobal.netwrote: Speaking of jobs on this list . . . . How much does a

Re: Reverse range search

2010-10-30 Thread Erick Erickson
I wonder if function queries might help you here (be aware that these aren't my strong point though). See: http://wiki.apache.org/solr/FunctionQuery http://wiki.apache.org/solr/FunctionQueryBest Erick On Thu, Oct 28, 2010 at 5:24 PM, kenf_nc ken.fos...@realestate.com wrote: Doing a range

Re: QueryElevation Component is so slow

2010-10-30 Thread Lance Norskog
Now you got me interested. Always a bad thing ;) Looking at the QueryElevationComponent, I don't know enough to decide if it has algorithms that don't scale. It does something odd with sorting. It has a concurrent access path for each query, which should not be a problem. It has not changed much

Re: Ensuring stable timestamp ordering

2010-10-30 Thread Erick Erickson
What are the actual values in your index? I'm wondering if they all get the same values somehow, perhaps due to the granularity of your dates? And (and I'm really grasping at straws here) your commit is causing enough delay to have time intervals be greater than your granularity. Unfortunately,

Re: Would it be nuts to store a bunch of large attachments (images, videos) in stored but-not-indexed fields

2010-10-30 Thread Lance Norskog
There is a binary field type for this problem. Trunk versions now do not have to base-64 encode but just store the bytes directly (I think). There is a quirk: Solr/Lucene field contents (the stored part) are stored in field order. So all of the fields are in order on the disk. So, when Lucene

Re: solr 4.0 - pagination

2010-10-30 Thread Erick Erickson
See: https://issues.apache.org/jira/browse/SOLR-2207 https://issues.apache.org/jira/browse/SOLR-2207You can track the progress here, but it hasn't been committed to trunk yet. Your options are: 1 get a recent trunk source tree and apply the patch and compile. 2 wait until it gets put in the trunk

Re: Ensuring stable timestamp ordering

2010-10-30 Thread Lance Norskog
Hi- NOW does not get re-run for each document. If you give a large upload batch, the same NOW is given to each document. It would be handy to have an auto-incrementing date field, so that each document would get a unique number and the timestamp would then be the unique ID of the document. On

Re: eDismax result differs from Dismax

2010-10-30 Thread Lance Norskog
Does the eDismax parser design guarantee that it will give the same results as the old dismax for a legal query? If not, are the differences minor? On Fri, Oct 29, 2010 at 6:40 AM, Yonik Seeley yo...@lucidimagination.com wrote: On Fri, Oct 29, 2010 at 9:30 AM, Ryan Walker

Re: QueryElevation Component is so slow

2010-10-30 Thread Chamnap Chhorn
Well, I use Solr 1.4. There are 30698 lines in my elevation file. I need only 20 results response back at a time. On Sun, Oct 31, 2010 at 9:12 AM, Lance Norskog goks...@gmail.com wrote: Now you got me interested. Always a bad thing ;) Looking at the QueryElevationComponent, I don't know

Re: QueryElevation Component is so slow

2010-10-30 Thread Lance Norskog
How many items for each query? On Sat, Oct 30, 2010 at 7:34 PM, Chamnap Chhorn chamnapchh...@gmail.com wrote: Well, I use Solr 1.4. There are 30698 lines in my elevation file. I need only 20 results response back at a time. On Sun, Oct 31, 2010 at 9:12 AM, Lance Norskog goks...@gmail.com

Re: QueryElevation Component is so slow

2010-10-30 Thread Chamnap Chhorn
20 On Sun, Oct 31, 2010 at 9:44 AM, Lance Norskog goks...@gmail.com wrote: How many items for each query? On Sat, Oct 30, 2010 at 7:34 PM, Chamnap Chhorn chamnapchh...@gmail.com wrote: Well, I use Solr 1.4. There are 30698 lines in my elevation file. I need only 20 results response