Ian, Turns out you were very close to the truth. The problem was in how I was ingesting the original file into memory before indexing.
Thanks, Mr. Ilya Zavorin Applied Research and Consulting CACI Advanced Knowledge Solutions Division 4831 Walden Lane, Lanham, MD 20706 ph: 1-301-306-2859 fx: 1-301-306-8201 izavo...@caci.com www.caci.com -----Original Message----- From: Ian Lea [mailto:ian....@gmail.com] Sent: Friday, July 13, 2012 1:06 PM To: java-user@lucene.apache.org Subject: Re: can't find queries when they are one per line in target file It's hard to tell from your description exactly what you are indexing and searching for, but I'd hazard a guess that the problem is related to your "content of entire target file" comment. Maybe you need to read the files line by line. -- Ian. On Fri, Jul 13, 2012 at 6:02 PM, Ilya Zavorin <izavo...@caci.com> wrote: > > Here are the details: > > I ran 2 tests: > > 1. Index only the first target file (the one where all the queries are in one > long line); Then loop over all queries and search for each using the code > block below. > 2. Index only the second target file (the one where all the queries are > listed one per line); Then loop over all queries and search for each using > the code block below. > > Query query = parser.parse(qStr); > TopDocs results = searcher.search(query, Integer.MAX_VALUE); > ScoreDoc[] hits = results.scoreDocs; > > 1st test: all queries found, i.e. the code block below returned a hit > for each query 2nd test: no queries found, i.e. the code block below > returned no hits for any query > > So the difference seems to be in the structure of the indexed target files. > > Here's the block where a target file gets added to the index: > > ... > Document doc = new Document(); > String oc = ...; // content of entire target file > doc.add(new Field("contents", > oc, > Field.Store.NO, > Field.Index.ANALYZED, > Field.TermVector.WITH_POSITIONS_OFFSETS)); > writer.addDocument(doc); > ... > > Let me know if more details are needed. > > Thanks, > > Ilya > > -----Original Message----- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Friday, July 13, 2012 12:44 PM > To: java-user@lucene.apache.org > Subject: RE: can't find queries when they are one per line in target > file > > What do you mean with "files"? Without a complete description what you are > doing we cannot answer your request. > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -----Original Message----- >> From: Ilya Zavorin [mailto:izavo...@caci.com] >> Sent: Friday, July 13, 2012 6:39 PM >> To: java-user@lucene.apache.org >> Subject: RE: can't find queries when they are one per line in target >> file >> >> >> >> But why then does it find all the querries in the 1st file? I use >> exactly > the same >> code. >> >> IZ >> >> >> -----Original Message----- >> From: Uwe Schindler [mailto:u...@thetaphi.de] >> Sent: Friday, July 13, 2012 12:32 PM >> To: java-user@lucene.apache.org >> Subject: RE: can't find queries when they are one per line in target >> file >> >> > String qStr = "Query1"; // or "Query2" or ... >> > QueryParser parser = ...; >> > IndexSearcher searcher = ...; >> > Query query = parser.parse(qStr); >> > TopDocs results = searcher.search(query, Integer.MAX_VALUE); >> > ScoreDoc[] >> hits >> > = results.scoreDocs; >> > >> > returned no hits for the 2nd test. >> >> Maybe because it runs out of memory? Passing Integer.MAX_VALUE is >> allocating >> 2 billion result slots... >> >> Uwe >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
