Re: Problem searching Field.Keyword field
Are there any issues with having a bunch of boolean queries and than adding them to one big boolean queries (making them all required)? Or should I be looking at Query.combine()? Thanks, Luke - Original Message - From: Erik Hatcher [EMAIL PROTECTED] To: Lucene Users List lucene-user@jakarta.apache.org Sent: Tuesday, February 08, 2005 12:02 PM Subject: Re: Problem searching Field.Keyword field Kelvin - I respectfully disagree - could you elaborate on why this is not an appropriate use of Field.Keyword? If the category is How To, Field.Text would split this (depending on the Analyzer) into how and to. If the user is selecting a category from a drop-down, though, you shouldn't be using QueryParser on it, but instead aggregating a TermQuery(category, How To) into a BooleanQuery with the rest of it. The rest may be other API created clauses and likely a piece from QueryParser. Erik On Feb 8, 2005, at 11:28 AM, Kelvin Tan wrote: As I posted previously, Field.Keyword is appropriate in only certain situations. For your use-case, I believe Field.Text is more suitable. k On Tue, 8 Feb 2005 10:02:19 -0600, Mike Miller wrote: This may or may not be correct, but I am indexing it as a keyword because I provide a (required) radio button on the add screen for the user to determine which category the document should be assigned. Then in the search, provide a dropdown that can be used in the advanced search so that they can search only for a specific category of documents (like HowTo, Troubleshooting, etc). -Original Message- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 9:32 AM To: Lucene Users List Subject: RE: Problem searching Field.Keyword field Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
Are there any issues with having a bunch of boolean queries and than adding them to one big boolean queries (making them all required)? Or should I be looking at Query.combine()? Thanks, Luke - Original Message - From: Erik Hatcher [EMAIL PROTECTED] To: Lucene Users List lucene-user@jakarta.apache.org Sent: Tuesday, February 08, 2005 12:02 PM Subject: Re: Problem searching Field.Keyword field Kelvin - I respectfully disagree - could you elaborate on why this is not an appropriate use of Field.Keyword? If the category is How To, Field.Text would split this (depending on the Analyzer) into how and to. If the user is selecting a category from a drop-down, though, you shouldn't be using QueryParser on it, but instead aggregating a TermQuery(category, How To) into a BooleanQuery with the rest of it. The rest may be other API created clauses and likely a piece from QueryParser. Erik On Feb 8, 2005, at 11:28 AM, Kelvin Tan wrote: As I posted previously, Field.Keyword is appropriate in only certain situations. For your use-case, I believe Field.Text is more suitable. k On Tue, 8 Feb 2005 10:02:19 -0600, Mike Miller wrote: This may or may not be correct, but I am indexing it as a keyword because I provide a (required) radio button on the add screen for the user to determine which category the document should be assigned. Then in the search, provide a dropdown that can be used in the advanced search so that they can search only for a specific category of documents (like HowTo, Troubleshooting, etc). -Original Message- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 9:32 AM To: Lucene Users List Subject: RE: Problem searching Field.Keyword field Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
On Thursday 10 February 2005 18:44, Luke Shannon wrote: Are there any issues with having a bunch of boolean queries and than adding them to one big boolean queries (making them all required)? The 1.4.3 and earlier BooleanScorer has an out of bounds exception for More than 32 required/prohibited clauses in query. In the development version this restriction has gone. The limitation of the maximum clause count (default 1024, configurable) is still there. Regards, Paul Elschot - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
On Tue, 2005-02-08 at 12:19 -0500, Steven Rowe wrote:
Why is there no KeywordAnalyzer? That is, an analyzer which doesn't
mess with its input in any way, but just returns it as-is?
I realize that under most circumstances, it would probably be more code
to use it than just constructing a TermQuery, but having it would
regularize query handling, and simplify new users' experience. And for
the purposes of the PerFieldAnalyzerWrapper, it could be helpful.
It's fairly straightforward to write one. Here's the one I put together
for PerFieldAnalyzerWrapper situations:
package org.apache.lucene.analysis;
import java.io.Reader;
public class VerbatimAnalyzer extends Analyzer {
public VerbatimAnalyzer() {
super();
}
public TokenStream tokenStream(String fieldName, Reader reader) {
TokenStream result = new VerbatimTokenizer(reader);
return result;
}
/**
* This tokenizer assumes that the entire input is just one token.
*/
public static class VerbatimTokenizer extends CharTokenizer {
public VerbatimTokenizer(Reader reader) {
super(reader);
}
protected boolean isTokenChar(char c) {
return true;
}
}
}
--
Miles Barr [EMAIL PROTECTED]
Runtime Collective Ltd.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
The only caveat to your VerbatimAnalyzer is that it will still split
strings that are over 255 characters. CharTokenizer does that.
Granted, though, that keyword fields probably don't make much sense to
be that long.
As mentioned yesterday - I added the LIA KeywordAnalyzer into the
contrib area of Subversion. I had built one like you had also, but the
one I contributed reads the entire input stream into a StringBuffer
ensuring it does not get split like CharTokenizer would.
Erik
On Feb 9, 2005, at 4:40 AM, Miles Barr wrote:
On Tue, 2005-02-08 at 12:19 -0500, Steven Rowe wrote:
Why is there no KeywordAnalyzer? That is, an analyzer which doesn't
mess with its input in any way, but just returns it as-is?
I realize that under most circumstances, it would probably be more
code
to use it than just constructing a TermQuery, but having it would
regularize query handling, and simplify new users' experience. And
for
the purposes of the PerFieldAnalyzerWrapper, it could be helpful.
It's fairly straightforward to write one. Here's the one I put together
for PerFieldAnalyzerWrapper situations:
package org.apache.lucene.analysis;
import java.io.Reader;
public class VerbatimAnalyzer extends Analyzer {
public VerbatimAnalyzer() {
super();
}
public TokenStream tokenStream(String fieldName, Reader reader) {
TokenStream result = new VerbatimTokenizer(reader);
return result;
}
/**
* This tokenizer assumes that the entire input is just one token.
*/
public static class VerbatimTokenizer extends CharTokenizer {
public VerbatimTokenizer(Reader reader) {
super(reader);
}
protected boolean isTokenChar(char c) {
return true;
}
}
}
--
Miles Barr [EMAIL PROTECTED]
Runtime Collective Ltd.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
On Wed, 2005-02-09 at 06:56 -0500, Erik Hatcher wrote: The only caveat to your VerbatimAnalyzer is that it will still split strings that are over 255 characters. CharTokenizer does that. Granted, though, that keyword fields probably don't make much sense to be that long. As mentioned yesterday - I added the LIA KeywordAnalyzer into the contrib area of Subversion. I had built one like you had also, but the one I contributed reads the entire input stream into a StringBuffer ensuring it does not get split like CharTokenizer would. That's good to know. When indexing web sites I use the URL as the identifier and hence store it in a keyword field. While not common, it is possible for URLs to be longer than 255 characters. That could have led to some very awkward bugs to track down. I'll probably switch over to your KeywordAnalyzer. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
Javadocs for Field.Keyword says:
Constructs a Date-valued Field that is not tokenized and is indexed, and stored
in the index, for return with hits.
For most purposes dealing with Strings, use Field.Text, unless you have a date,
a GUID or some other string you don't want tokenized or processed in any way.
This basically means that Field.Keyword indexes the field as-is.
k
On Tue, 8 Feb 2005 07:54:57 -0600, Mike Miller wrote:
First let me say - Awesome tool! Almost too easy to be true, but
with that being said
Hi, I have read several articles and postings that indicate that
the Field.Keyword field should be searchable but it's not working
for me, until I change to Field.Text. Parts of the index and
search code are included below - mostly lifted from articles,etc,
including Erik Hatches article on java.net. I created a small
KnowledgeBase web application that contains a category field, which
I want to be searchable. Searching using a query string of
category:Doc* or
category:Documentation does not find a hit unless I change the code
to add the category to the index as a Field.Text instead of
Field.Keyword. The field value is out there: I have verified this
using the TermEnum to list the term values for field category and
Documentation is in the list of values.
The intention is to provide a 'Advanced Search' page that allows
the user to search specific fields, like category, title and author
instead of always using the 'all' field.
What am I doing wrong??? Thanks in advance.
Index code:
public boolean index(ArticleFormBean article) throws IOException {
IndexWriter writer = new IndexWriter(indexDir, new
StandardAnalyzer(), false);
Document doc = new Document();
doc.add(Field.UnStored(content, article.getContent()));
doc.add(Field.Text(title, article.getTitle()));
doc.add(Field.Text(author, article.getAuthor()));
doc.add(Field.UnIndexed(articleId,
String.valueOf(article.getArticleId(;
doc.add(Field.Keyword(createdDate, article.getCreateDate()));
doc.add(Field.Keyword(modDate, article.getModDate()));
doc.add(Field.Keyword(category, article.getCategory()));
// create an 'all' field
StringBuffer sb = new StringBuffer(4000);
sb.append(article.getTitle()).append(
).append(article.getAuthor()).append( );
sb.append(article.getContent()).append(
).append(article.getCategory());
doc.add(Field.UnStored(all, sb.toString()));
writer.addDocument(doc);
writer.optimize();
writer.close();
return false;
}
Search code:
File indexDir = new File(c:/dev/java/kb/index); Directory fsDir =
FSDirectory.getDirectory(indexDir, false); IndexSearcher is = new
IndexSearcher(fsDir); Query query = QueryParser.parse(q, all, new
StandardAnalyzer()); Hits hits = is.search(query);
Mike Miller
JDA Software Group, Inc.
7501 Ester's Blvd, Suite 100
Irving, Texas 75063
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: Problem searching Field.Keyword field
Hi Mike, If you use a different analyzer, say a custom one the didn't do anything to the original search query, then you could use the query parser to search on the keyword field. The standard analyzer does things like making everything lowercase, removing stop words etc. Since the value held in the keyword field didn't go through the same process during indexing it won't come up as a match. So basically you want to do: ... IndexSearcher is = new IndexSearcher(fsDir); ... Term t = new Term(keyword-field-name, keyword); Query query = new TermQuery(t); Hits hits = is.search(query); You will typically only do this when you're trying to retrieve a specific document, rather than doing a search. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
The problem is that QueryParser analyzes all pieces of a query expression regardless of whether you indexed them as a Field.Keyword or not. If you need to use QueryParser and still support keyword fields, you'll want to plug in an analyzer specific to that field using PerFieldAnalyzerWrapper. You'll see this demonstrated in the Lucene in Action source code. Here's a quick pointer to where we cover it in the book: http://www.lucenebook.com/search?query=KeywordAnalyzer On Feb 8, 2005, at 9:26 AM, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Problem searching Field.Keyword field
Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Problem searching Field.Keyword field
This may or may not be correct, but I am indexing it as a keyword because I provide a (required) radio button on the add screen for the user to determine which category the document should be assigned. Then in the search, provide a dropdown that can be used in the advanced search so that they can search only for a specific category of documents (like HowTo, Troubleshooting, etc). -Original Message- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 9:32 AM To: Lucene Users List Subject: RE: Problem searching Field.Keyword field Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Problem searching Field.Keyword field
As I posted previously, Field.Keyword is appropriate in only certain situations. For your use-case, I believe Field.Text is more suitable. k On Tue, 8 Feb 2005 10:02:19 -0600, Mike Miller wrote: This may or may not be correct, but I am indexing it as a keyword because I provide a (required) radio button on the add screen for the user to determine which category the document should be assigned. Then in the search, provide a dropdown that can be used in the advanced search so that they can search only for a specific category of documents (like HowTo, Troubleshooting, etc). -Original Message- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 9:32 AM To: Lucene Users List Subject: RE: Problem searching Field.Keyword field Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
Kelvin - I respectfully disagree - could you elaborate on why this is not an appropriate use of Field.Keyword? If the category is How To, Field.Text would split this (depending on the Analyzer) into how and to. If the user is selecting a category from a drop-down, though, you shouldn't be using QueryParser on it, but instead aggregating a TermQuery(category, How To) into a BooleanQuery with the rest of it. The rest may be other API created clauses and likely a piece from QueryParser. Erik On Feb 8, 2005, at 11:28 AM, Kelvin Tan wrote: As I posted previously, Field.Keyword is appropriate in only certain situations. For your use-case, I believe Field.Text is more suitable. k On Tue, 8 Feb 2005 10:02:19 -0600, Mike Miller wrote: This may or may not be correct, but I am indexing it as a keyword because I provide a (required) radio button on the add screen for the user to determine which category the document should be assigned. Then in the search, provide a dropdown that can be used in the advanced search so that they can search only for a specific category of documents (like HowTo, Troubleshooting, etc). -Original Message- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 9:32 AM To: Lucene Users List Subject: RE: Problem searching Field.Keyword field Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
Why is there no KeywordAnalyzer? That is, an analyzer which doesn't mess with its input in any way, but just returns it as-is? I realize that under most circumstances, it would probably be more code to use it than just constructing a TermQuery, but having it would regularize query handling, and simplify new users' experience. And for the purposes of the PerFieldAnalyzerWrapper, it could be helpful. Steve Erik Hatcher wrote: Kelvin - I respectfully disagree - could you elaborate on why this is not an appropriate use of Field.Keyword? If the category is How To, Field.Text would split this (depending on the Analyzer) into how and to. If the user is selecting a category from a drop-down, though, you shouldn't be using QueryParser on it, but instead aggregating a TermQuery(category, How To) into a BooleanQuery with the rest of it. The rest may be other API created clauses and likely a piece from QueryParser. Erik On Feb 8, 2005, at 11:28 AM, Kelvin Tan wrote: As I posted previously, Field.Keyword is appropriate in only certain situations. For your use-case, I believe Field.Text is more suitable. k On Tue, 8 Feb 2005 10:02:19 -0600, Mike Miller wrote: This may or may not be correct, but I am indexing it as a keyword because I provide a (required) radio button on the add screen for the user to determine which category the document should be assigned. Then in the search, provide a dropdown that can be used in the advanced search so that they can search only for a specific category of documents (like HowTo, Troubleshooting, etc). -Original Message- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 9:32 AM To: Lucene Users List Subject: RE: Problem searching Field.Keyword field Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
Erik, I was thinking about the case where category:document management category:document publishing and the user wants to search category:document and have both turn up. But that's obviously not the use-case in the situation of a drop-down, so you're right about this, Field.Keyword is correct here. Sorry for misleading you, Mike. k On Tue, 8 Feb 2005 12:02:15 -0500, Erik Hatcher wrote: Kelvin - I respectfully disagree - could you elaborate on why this is not an appropriate use of Field.Keyword? If the category is How To, Field.Text would split this (depending on the Analyzer) into how and to. If the user is selecting a category from a drop-down, though, you shouldn't be using QueryParser on it, but instead aggregating a TermQuery(category, How To) into a BooleanQuery with the rest of it. The rest may be other API created clauses and likely a piece from QueryParser. Erik On Feb 8, 2005, at 11:28 AM, Kelvin Tan wrote: As I posted previously, Field.Keyword is appropriate in only certain situations. For your use-case, I believe Field.Text is more suitable. k On Tue, 8 Feb 2005 10:02:19 -0600, Mike Miller wrote: This may or may not be correct, but I am indexing it as a keyword because I provide a (required) radio button on the add screen for the user to determine which category the document should be assigned. Then in the search, provide a dropdown that can be used in the advanced search so that they can search only for a specific category of documents (like HowTo, Troubleshooting, etc). -Original Message- From: Kelvin Tan [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 9:32 AM To: Lucene Users List Subject: RE: Problem searching Field.Keyword field Mike, is there a reason why you're indexing category as keyword not text? k On Tue, 8 Feb 2005 08:26:13 -0600, Mike Miller wrote: Thanks for the quick response. Sorry for my lack of understanding, but I am learning! Won't the query parser still handle this query? My limited understanding was that the search call provides the 'all' field as default field for query terms in the case where fields aren't specified. Using the current code, searches like author:Mike and title:Lucene work fine. -Original Message- From: Miles Barr [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 08, 2005 8:08 AM To: Lucene Users List Subject: Re: Problem searching Field.Keyword field You're using the query parser with the standard analyser. You should construct a term query manually instead. -- Miles Barr [EMAIL PROTECTED] Runtime Collective Ltd. -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e- mail: [EMAIL PROTECTED] -- -- - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e- mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- --- To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: lucene-user- [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem searching Field.Keyword field
On Feb 8, 2005, at 12:19 PM, Steven Rowe wrote: Why is there no KeywordAnalyzer? That is, an analyzer which doesn't mess with its input in any way, but just returns it as-is? I realize that under most circumstances, it would probably be more code to use it than just constructing a TermQuery, but having it would regularize query handling, and simplify new users' experience. And for the purposes of the PerFieldAnalyzerWrapper, it could be helpful. It's long been on my TODO list. I just adapted (changed the package names) the Lucene in Action KeywordAnalyzer and added it to the new contrib area: http://svn.apache.org/repos/asf/lucene/java/trunk/contrib/analyzers/ src/java/org/apache/lucene/analysis/KeywordAnalyzer.java In the next official release of Lucene, the contrib (formerly known as the Sandbox) components will be packaged along with the Lucene core. I'm still working on this packaging build process as I migrate the Sandbox over to contrib. Erik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
