Re: Best Practices for getting Strings from a position range

2007-08-10 Thread Peter Keegan
ok, glad we're on the same page. I did some performance testing with span queries and, unfortunately, the results are discouraging for my intended use. When I added a simple SpanNearQuery to existing queries, the throughput decreased by a factor of 10+. I figured spans would be expensive, but not

[jira] Resolved: (LUCENE-976) MMapDirectory is missing newly added openInput method to FSDirectory

2007-08-10 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-976. --- Resolution: Fixed Thanks Peter! > MMapDirectory is missing newly added openInput met

Re: MMapDirectory is broken in 2.2

2007-08-10 Thread Michael McCandless
Awesome, thanks! I will commit shortly... Mike "Peter Keegan" <[EMAIL PROTECTED]> wrote: > Hi Mike, > > I tested the patch and it looks good. > > Thanks, > Peter > > On 8/10/07, Peter Keegan <[EMAIL PROTECTED]> wrote: > > > > Heck, no need to apologize - no harm done. For all the great work

Re: MMapDirectory is broken in 2.2

2007-08-10 Thread Peter Keegan
Hi Mike, I tested the patch and it looks good. Thanks, Peter On 8/10/07, Peter Keegan <[EMAIL PROTECTED]> wrote: > > Heck, no need to apologize - no harm done. For all the great work by you > and others, the least I can do is help find problems ;-) > > Peter > > On 8/10/07, Michael McCandless <[

[jira] Updated: (LUCENE-976) MMapDirectory is missing newly added openInput method to FSDirectory

2007-08-10 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-976: -- Attachment: LUCENE-976.patch Simple patch to fix this. I also added a unit test that l

[jira] Created: (LUCENE-976) MMapDirectory is missing newly added openInput method to FSDirectory

2007-08-10 Thread Michael McCandless (JIRA)
MMapDirectory is missing newly added openInput method to FSDirectory Key: LUCENE-976 URL: https://issues.apache.org/jira/browse/LUCENE-976 Project: Lucene - Java Issue Type

Re: MMapDirectory is broken in 2.2

2007-08-10 Thread Peter Keegan
Heck, no need to apologize - no harm done. For all the great work by you and others, the least I can do is help find problems ;-) Peter On 8/10/07, Michael McCandless <[EMAIL PROTECTED]> wrote: > > > OK, sorry about this :( I will fix. > > Mike > > "Peter Keegan" <[EMAIL PROTECTED]> wrote: > > >

Re: Fwd: Decouple Filter from BitSet: API change and xml query parser

2007-08-10 Thread markharw00d
Right, the only thing left is then how to get a Matcher from this iterator. I think the iterator *is* the equivalent of the Matcher as you've described it - a Scorer without the scores used once by a single thread to iterate across a set of doc ids. I suppose the Filter criterium is a

Re: MMapDirectory is broken in 2.2

2007-08-10 Thread Michael McCandless
OK, sorry about this :( I will fix. Mike "Peter Keegan" <[EMAIL PROTECTED]> wrote: > >it affects both 2.2 (released ) and trunk, right? > > I don't have a 2.2 index to test, but the code looks the same as the > trunk. > It would be tricky to write a unit test for this. I only noticed because >

Re: Fwd: Decouple Filter from BitSet: API change and xml query parser

2007-08-10 Thread Paul Elschot
On Friday 10 August 2007 19:21, mark harwood wrote: > >>I'll change the CachingWrapperFilter to use a BitSetFilter, > >>which data structure would you like to have cached for filtering > >>in the xml query parser? > > I think Filter/BitSetFilter might be the wrong choice of object to cache > in t

Re: MMapDirectory is broken in 2.2

2007-08-10 Thread Peter Keegan
>it affects both 2.2 (released ) and trunk, right? I don't have a 2.2 index to test, but the code looks the same as the trunk. It would be tricky to write a unit test for this. I only noticed because query performance went down the drain. Peter On 8/10/07, Michael McCandless <[EMAIL PROTECTED]>

Re: MMapDirectory is broken in 2.2

2007-08-10 Thread Michael McCandless
Whoa, I see -- this was caused by my commit for LUCENE-888, and, it affects both 2.2 (released ) and trunk, right? I think the fix is to add the corresponding constructor (openInput(String name, int bufferSize)) into MMapDirectory but have it ignore the bufferSize? I will open a Jira issue. Tha

Re: Best Practices for getting Strings from a position range

2007-08-10 Thread Grant Ingersoll
Sorry for the confusion. I thought you just wanted access to the term info per position.I think we will have to add something to the Spans like we talked about before. -Grant On Aug 10, 2007, at 11:03 AM, Peter Keegan wrote: Grant, I'm afraid I don't understand how to use this mapper

MMapDirectory is broken in 2.2

2007-08-10 Thread Peter Keegan
I have discovered a bug in 2.2 (trunk version) that prevents MMapDirectory from working. TermInfosReader has a new constructor that calls a new version of FSDirectory.openInput(String, int). This new openInput signature isn't implemented in MMapDirectory, so FSDirectory is used instead. Should I s

Re: Fwd: Decouple Filter from BitSet: API change and xml query parser

2007-08-10 Thread mark harwood
>>I'll change the CachingWrapperFilter to use a BitSetFilter, >>which data structure would you like to have cached for filtering in the xml >>query parser? I think Filter/BitSetFilter might be the wrong choice of object to cache in there. It is the *data that Filters create* which is most import

Re: Fwd: Decouple Filter from BitSet: API change and xml query parser

2007-08-10 Thread Paul Elschot
On Friday 10 August 2007 13:12, mark harwood wrote: > >>Could someone give me a clue as to why the test case TestRemoteCachingWrapperFilter fails with the patch applied? > > Regardless of the reasons for this particular test failure, this code is not safe in other ways which the test cases don

Re: Best Practices for getting Strings from a position range

2007-08-10 Thread Peter Keegan
Grant, I'm afraid I don't understand how to use this mapper in the context of a SpanQuery. It seems like I would have to modify SpanScorer to fetch payload data and provide a new method to access the payloads while iterating through the documents. If this can be accomplished without modifying Span

Re: Fwd: Decouple Filter from BitSet: API change and xml query parser

2007-08-10 Thread mark harwood
>>Could someone give me a clue as to why the test case >>TestRemoteCachingWrapperFilter fails with the patch applied? Regardless of the reasons for this particular test failure, this code is not safe in other ways which the test cases don't test for. To restate the issue: Matcher is not designe

Re: TREC Collection, NIST and Lucene

2007-08-10 Thread Tom White
> Furthermore, I think it would > encourage Lucene users/developers to think about relevance as much as > we think about speed. +1 However I think it would be much better to start by making informal approaches as you suggest - the open letter seems to me to be appropriate only as a last resort.

Fwd: Decouple Filter from BitSet: API change and xml query parser

2007-08-10 Thread Paul Elschot
Taking this to java-dev only. As I said at the jira issue, I'd like to have all test cases pass again, and I'm not happy with the current version of the patch to the xml query parser either. Some test cases currently fail maybe because they use RMI and the new version of Filter does serialize we