Re: short search terms
Hi
You are searching with 3 characters but the items actually indexed has 4
characters. Use Luke and analyze the index.
If searching for ABC has to be matched with ABCD then you need to do a
wildcard search. Add * at the end of the search query (ABC*).
Regards
Aditya
www.findbestopensource.com
On Thu, Sep 27, 2012 at 3:05 AM, Edward W. Rouse wrote:
> I have an index and one of the items to search for is an identifier that
> will always be 3 characters, like ABC or XYZ. If I do a search for ABC I
> get
> no matches. If I add 1 more character so that ABC becomes ABCD and search
> for ABCD, it matches. I have been looking through the code (I inherited and
> the original coder is no longer with the company) to see if there is any
> place where he might have put a limitation in, but testing indicates that
> it
> is creating a query. Some code below:
>
> QueryParser parser = new QueryParser(Version.LUCENE_34,
> TaskRecord.BaseFields.PUBLIC_DEFAULT_FIELD.getName(), getAnalyzer());
> parser.setDefaultOperator(Operator.AND);
> Query query = parser.parse(qstring);
>
> qstring is the search text and getAnalyser returns a StandardAnalyzer.
>
> The Query is then used to search using the following code:
>
> public List search(Query query) throws IOException
> {
> IndexReader reader = null;
> try
> {
> reader = IndexReader.open(getRoot(), true);
> IndexSearcher searcher = new IndexSearcher(reader);
>
> // Do the search with an artificial limit of 32 results
> TopDocs hits = searcher.search(query, 32);
>
> // If the search actually has more hits, then run it again with
> correct max
> if(hits.totalHits > 32)
> {
> if(log.isDebugEnabled())
> {
> log.debug("Rerunning query with max size of " + hits.totalHits +
> "
> " + query);
> }
>
> hits = searcher.search(query, hits.totalHits);
> }
>
> // Create task ID list and return
> if(hits.totalHits < 1)
> {
> if(log.isDebugEnabled())
> log.debug("Query has no hits " + query);
>
> return Collections.emptyList();
> }
> else
> {
> if(log.isDebugEnabled())
> log.debug("Query has " + hits.totalHits + " hits " + query);
>
> List taskIds = new ArrayList(hits.totalHits);
> for(ScoreDoc doc: hits.scoreDocs)
> {
> taskIds.add(Long.valueOf(searcher.doc(doc.doc).get("task")));
> }
> return taskIds;
> }
> }
> finally
> {
> try
> {
> if(reader != null)
> reader.close();
> }
> catch(IOException e)
> {
> }
> }
> }
>
>
> > -Original Message-
> > From: Chris Hostetter [mailto:hossman_luc...@fucit.org]
> > Sent: Wednesday, September 26, 2012 5:18 PM
> > To: java-user@lucene.apache.org
> > Subject: Re: short search terms
> >
> >
> > : I have a key field that will only ever have a length of 3 characters.
> > I am
> > : using a StandardAnalyzer and a QueryParser to create the Query
> > : (parser.parse(string)), and an IndexReader and IndexSearcher to
> > execute the
> > : query (searcher(query)). I can't seem to find a setter to allow for a
> > 3
> > : character search string. There is one setMinWordLen, but it isn't
> > applicable
> >
> > there's a lot of missing information here ... what do you mean "allow
> > for
> > a 3 character search string" .. the query parser doesn't have anything
> > in
> > it that would prevent a 3 (or 3, or 1) character search string, so i
> > suspect that's not really the question you mean to ask.
> >
> > what is problem you are actaully seeing? do you have a query that
> > isn't
> > matching the docs you think it should? what query? what docs? what does
> > the code look like?
> >
> > can you explain more what this 3 character ifeld represents, and how
> > you
> > want to use it?
> >
> > https://people.apache.org/~hossman/#xyproblem
> > Your question appears to be an "XY Problem" ... that is: you are
> > dealing
> > with "X", you are assuming "Y" will help you, and you are asking about
> > "Y"
> > without giving more details about the "X" so that we can understand the
> > full issue. Perhaps the best solution doesn't involve "Y" at all?
> > See Also: http://www.perlmonks.org/index.pl?node_id=542341
> >
> > -Hoss
> >
> > -
> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
Re: Memory issues with Lucene deployment
On 25/09/2012 20:09, Uwe Schindler wrote: Hi, Without a full output of "free -h" we cannot say anything. But the total Linux memory use should always used by 100% on a good server otherwise it's useless (because full memory includes cache usage, too). I think, -Xmx may be too less for your Java deployment? We have no information about the size of your installation. If "top" reports too much residential memory for the Java VM process then there might be something wrong, and then we would need "pmap " of the java process. Uwe Hi , I was a bit misinformed, having read your excellant article http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html and a bit more analysis I dont think there is a memory issue, rather a cpu issue. The service talking to server is timing out getting connections, so maybe I have a problem with release IndexSearchers or something, anyway trying to get a stack dump done Paul - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
RE: short search terms
You misread what I wrote. When it's ABC in the index and I search for ABC I
get no match. But AFTER I change ABC to ABCD and search for ABCD, I get a
match.
> -Original Message-
> From: Aditya [mailto:findbestopensou...@gmail.com]
> Sent: Thursday, September 27, 2012 5:19 AM
> To: java-user@lucene.apache.org
> Subject: Re: short search terms
>
> Hi
>
> You are searching with 3 characters but the items actually indexed has
> 4
> characters. Use Luke and analyze the index.
>
> If searching for ABC has to be matched with ABCD then you need to do a
> wildcard search. Add * at the end of the search query (ABC*).
>
> Regards
> Aditya
> www.findbestopensource.com
>
>
> On Thu, Sep 27, 2012 at 3:05 AM, Edward W. Rouse
> wrote:
>
> > I have an index and one of the items to search for is an identifier
> that
> > will always be 3 characters, like ABC or XYZ. If I do a search for
> ABC I
> > get
> > no matches. If I add 1 more character so that ABC becomes ABCD and
> search
> > for ABCD, it matches. I have been looking through the code (I
> inherited and
> > the original coder is no longer with the company) to see if there is
> any
> > place where he might have put a limitation in, but testing indicates
> that
> > it
> > is creating a query. Some code below:
> >
> > QueryParser parser = new QueryParser(Version.LUCENE_34,
> > TaskRecord.BaseFields.PUBLIC_DEFAULT_FIELD.getName(), getAnalyzer());
> > parser.setDefaultOperator(Operator.AND);
> > Query query = parser.parse(qstring);
> >
> > qstring is the search text and getAnalyser returns a
> StandardAnalyzer.
> >
> > The Query is then used to search using the following code:
> >
> > public List search(Query query) throws IOException
> > {
> > IndexReader reader = null;
> > try
> > {
> > reader = IndexReader.open(getRoot(), true);
> > IndexSearcher searcher = new IndexSearcher(reader);
> >
> > // Do the search with an artificial limit of 32 results
> > TopDocs hits = searcher.search(query, 32);
> >
> > // If the search actually has more hits, then run it again with
> > correct max
> > if(hits.totalHits > 32)
> > {
> > if(log.isDebugEnabled())
> > {
> > log.debug("Rerunning query with max size of " +
> hits.totalHits +
> > "
> > " + query);
> > }
> >
> > hits = searcher.search(query, hits.totalHits);
> > }
> >
> > // Create task ID list and return
> > if(hits.totalHits < 1)
> > {
> > if(log.isDebugEnabled())
> > log.debug("Query has no hits " + query);
> >
> > return Collections.emptyList();
> > }
> > else
> > {
> > if(log.isDebugEnabled())
> > log.debug("Query has " + hits.totalHits + " hits " +
> query);
> >
> > List taskIds = new ArrayList(hits.totalHits);
> > for(ScoreDoc doc: hits.scoreDocs)
> > {
> >
> taskIds.add(Long.valueOf(searcher.doc(doc.doc).get("task")));
> > }
> > return taskIds;
> > }
> > }
> > finally
> > {
> > try
> > {
> > if(reader != null)
> > reader.close();
> > }
> > catch(IOException e)
> > {
> > }
> > }
> > }
> >
> >
> > > -Original Message-
> > > From: Chris Hostetter [mailto:hossman_luc...@fucit.org]
> > > Sent: Wednesday, September 26, 2012 5:18 PM
> > > To: java-user@lucene.apache.org
> > > Subject: Re: short search terms
> > >
> > >
> > > : I have a key field that will only ever have a length of 3
> characters.
> > > I am
> > > : using a StandardAnalyzer and a QueryParser to create the Query
> > > : (parser.parse(string)), and an IndexReader and IndexSearcher to
> > > execute the
> > > : query (searcher(query)). I can't seem to find a setter to allow
> for a
> > > 3
> > > : character search string. There is one setMinWordLen, but it isn't
> > > applicable
> > >
> > > there's a lot of missing information here ... what do you mean
> "allow
> > > for
> > > a 3 character search string" .. the query parser doesn't have
> anything
> > > in
> > > it that would prevent a 3 (or 3, or 1) character search string, so
> i
> > > suspect that's not really the question you mean to ask.
> > >
> > > what is problem you are actaully seeing? do you have a query that
> > > isn't
> > > matching the docs you think it should? what query? what docs? what
> does
> > > the code look like?
> > >
> > > can you explain more what this 3 character ifeld represents, and
> how
> > > you
> > > want to use it?
> > >
> > > https://people.apache.org/~hossman/#xyproblem
> > > Your question appears to be an "XY Problem" ... that is: you are
> > > dealing
> > > with "X", you are assuming "Y" will help you, and you are asking
> about
> > > "Y"
> > > without giving more details about the "X" so that we can understand
> the
> > > full issue. Perhaps the best solution doesn't involve "Y" at all?
> > > See Also: http://www.perlmonks.org/index.pl?node_id=542341
> >
