Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-17 Thread Chris Hostetter
: Imagine there is a query like harry potter dvd-collection cheap or cheap : Harry Potter dvd-collection. : How can I customize, that, if there is something said about the category : cheap, Solr uses a facetting query on cat:cheap? To do so, I have to : alter the original query - how can I do

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-11 Thread MitchK
Hello Hossman, sorry for my late response. For this specific case, you are right. It makes more sense to do such work on the fly. However, I am only testing at the moment, what one can do with Solr and what not. Is the UpdateProcessor something that comes froms Lucene itself or from Solr?

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-11 Thread Erik Hatcher
On Jan 11, 2010, at 7:33 AM, MitchK wrote: Is the UpdateProcessor something that comes froms Lucene itself or from Solr? It's at the Solr level - http://lucene.apache.org/solr/api/org/apache/solr/update/processor/UpdateRequestProcessor.html Erik

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-11 Thread MitchK
Is there any schemata that explains which class is responsible for which level of processing my data to the index? My example was: I have categorized, whether something is cheap or expensive. Let's say I didn't do that on the fly, but with the help of the UpdateRequestProcessor. Imagine there

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-08 Thread MitchK
Okay, you're right. It really would be cleaner, if I do such stuff in the code which populates the document to Solr. Is there a way to prepare a document the described way with Lucene/Solr, before I analyze it? My use case is to categorize several documents in an automatic way, which includes

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-08 Thread Erick Erickson
Somewhere, you have to create the document XML you send to SOLR. Just add the calculated data to your new field there... HTH Erick On Fri, Jan 8, 2010 at 9:30 AM, MitchK mitc...@web.de wrote: Okay, you're right. It really would be cleaner, if I do such stuff in the code which populates the

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-08 Thread Chris Hostetter
: Is there a way to prepare a document the described way with Lucene/Solr, : before I analyze it? : My use case is to categorize several documents in an automatic way, which : includes that I have to create data from the given input doing some : information retrieval. As Ryan mentioned earlier:

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-07 Thread MitchK
Eric, you mean, everything is okay, but I do not see it? Internally for searching the analysis takes place and writes to the index in an inverted fashion, but the stored stuff is left alone. if I use an analyzer, Solr stores it's output two ways? One public output, which is similar to the

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-07 Thread Ryan McKinley
On Jan 7, 2010, at 10:50 AM, MitchK wrote: Eric, you mean, everything is okay, but I do not see it? Internally for searching the analysis takes place and writes to the index in an inverted fashion, but the stored stuff is left alone. if I use an analyzer, Solr stores it's output two

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-07 Thread MitchK
Thank you, Ryan. I will have a look on lucene's material and luke. I think I got it. :) Sometimes there will be the need, to response on the one hand the value and on the other hand the indexed version of the value. How can I fullfill such needs? Doing copyfield on indexed-only fields?

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-07 Thread Ryan McKinley
On Jan 7, 2010, at 12:11 PM, MitchK wrote: Thank you, Ryan. I will have a look on lucene's material and luke. I think I got it. :) Sometimes there will be the need, to response on the one hand the value and on the other hand the indexed version of the value. How can I fullfill such

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-07 Thread Erick Erickson
What is your use case for responding sometimes with the indexed value? Other than reconstructing a field that hasn't been stored, I can't think of one. I still think you're missing the point. Indexing and storing are orthogonal operations that have (almost) nothing to do with each other, for all

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-07 Thread MitchK
The difference between stored and indexed is clear now. You are right, if you are responsing only to normal users. Use case: You got a stored field The good, the bad and the ugly. And you got a really fantastic analyzer, which is doing some magic to this movie title. Let's say, the analyzer

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-07 Thread Erick Erickson
Well, I'd approach either of these use cases by simply performing my computations on the input and storing the result in another (non-indexed unless I wanted to search it) field. This wouldn't happen in the Analyzer, but in the code that populated the document fields. Which is a much cleaner

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-06 Thread Erick Erickson
Well, I have noticed that Solr isn't using ANY analyzer How do you know this? Because it's highly unlikely that SOLR is completely broken on that level. Erick On Wed, Jan 6, 2010 at 3:48 PM, MitchK mitc...@web.de wrote: I have tested a lot and all the time I thought I set wrong options

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-06 Thread Ryan McKinley
On Jan 6, 2010, at 3:48 PM, MitchK wrote: I have tested a lot and all the time I thought I set wrong options for my custom analyzer. Well, I have noticed that Solr isn't using ANY analyzer, filter or stemmer. It seems like it only stores the original input. The stored value is always

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-06 Thread MitchK
Hello Erick, thank you for answering. I can do whatever I want - Solr does nothing. For example: If I use the textgen-fieldtype which is predefined, nothing happens to the text. Even the stopFilter is not working - no stopword from stopword.txt was replaced. I think, that this only affects the

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-06 Thread MitchK
Hello Ryan, thank you for answering. In my schema.xml I am defining the field as indexed = true. The problem is: nothing, even the original predefined analyzers don't work anyway. Please, have a look on my response to Erick. Mitch P.S. Oh, I see what you mean. The field is indexed = true. My

Re: No Analyzer, tokenizer or stemmer works at Solr

2010-01-06 Thread Erik Hatcher
Mitch, Again, I think you're misunderstanding what analysis does. You must be expecting we think, though you've not provided exact duplication steps to be sure, that the value you get back from Solr is the analyzer processed output. It's not, it's exactly what you provide. Internally