Re: Very basic questions: Indexing text

2010-06-29 Thread Ahmet Arslan
Could you give an example? E.g. lets say I have a field 'title' and a field 'fulltext' and my search term is 'solr'. What would be the right set of parameters to get back the whole title-field but only a sniplet of 50 words (or three sentences or whatever the unit) from the fulltext field.

Re: Very basic questions: Indexing text - working, but slow!

2010-06-29 Thread Peter Spam
Thanks for everyone's help - I have this working now, but sometimes the queries are incredibly slow!! For example, int name=QTime461360/int. Also, I had to bump up the min/max RAM size to 1GB/3.5GB for things to inject without throwing heap memory errors. However, my data set is very small!

Re: Very basic questions: Indexing text - working, but slow!

2010-06-29 Thread Peter Spam
To follow up, I've found that my queries are very fast (even with fq=), until I add hl=true. What can I do to speed up highlighting? Should I consider injecting a line at a time, rather than the entire file as a field? -Pete On Jun 29, 2010, at 11:07 AM, Peter Spam wrote: Thanks for

Re: Very basic questions: Indexing text - working, but slow!

2010-06-29 Thread Erick Erickson
What are you actual highlighting requirements? you could try things like maxAnalyzedChars, requireFieldMatch, etc http://wiki.apache.org/solr/HighlightingParameters has a good list, but you've probably already seen that page Best Erick On Tue, Jun 29, 2010 at 9:11 PM, Peter Spam

Re: Very basic questions: Indexing text - working, but slow!

2010-06-29 Thread Lance Norskog
To highlight a field, Solr needs some extra Lucene values. If these are not configured for the field in the schema, Solr has to re-analyze the field to highlight it. If you want faster highlighting, you have to add term vectors to the schema. Here is the grand map of such things:

Very basic questions: Indexing text

2010-06-28 Thread Peter Spam
Hi everyone, I'm looking for a way to index a bunch of (potentially large) text files. I would love to see results like Google, so I went through a few tutorials, but I've still got questions: 1) I can get my docs in the index, but when I search, it returns the entire document. I'd love to

Re: Very basic questions: Indexing text

2010-06-28 Thread Ahmet Arslan
1) I can get my docs in the index, but when I search, it returns the entire document.  I'd love to have it only return the line (or two) around the search term. Solr can generate Google-like snippets as you describe. http://wiki.apache.org/solr/HighlightingParameters 2) There are one or two

Re: Very basic questions: Indexing text

2010-06-28 Thread Peter Spam
Great, thanks for the pointers. Thanks, Peter On Jun 28, 2010, at 2:00 PM, Ahmet Arslan wrote: 1) I can get my docs in the index, but when I search, it returns the entire document. I'd love to have it only return the line (or two) around the search term. Solr can generate Google-like

Re: Very basic questions: Indexing text

2010-06-28 Thread Peter Spam
On Jun 28, 2010, at 2:00 PM, Ahmet Arslan wrote: 1) I can get my docs in the index, but when I search, it returns the entire document. I'd love to have it only return the line (or two) around the search term. Solr can generate Google-like snippets as you describe.

Re: Very basic questions: Indexing text

2010-06-28 Thread Erick Erickson
try adding hl.fl=text to specify your highlight field. I don't understand why you're only getting the ID field back though. Do note that the highlighting is after the docs, related by the ID. Try a (non highlighting) query of just * to verify that you're pointing at the index you think you are.

Re: Very basic questions: Indexing text

2010-06-28 Thread Michael Lackhoff
On 28.06.2010 23:00 Ahmet Arslan wrote: 1) I can get my docs in the index, but when I search, it returns the entire document. I'd love to have it only return the line (or two) around the search term. Solr can generate Google-like snippets as you describe.