[jira] Commented: (LUCENE-1088) PriorityQueue 'wouldBeInserted' method

2007-12-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550843 ] Shai Erera commented on LUCENE-1088: If you're adding a wouldBeInserted method, I'd add a insertWithNoCheck that

[jira] Updated: (LUCENE-944) Remove deprecated methods in BooleanQuery

2007-12-11 Thread Michael Busch (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Busch updated LUCENE-944: - Fix Version/s: (was: 2.3) 3.0 > Remove deprecated methods in BooleanQuery

[jira] Resolved: (LUCENE-673) Exceptions when using Lucene over NFS

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-673. --- Resolution: Fixed > Exceptions when using Lucene over NFS > -

[jira] Updated: (LUCENE-1088) PriorityQueue 'wouldBeInserted' method

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1088: --- Attachment: LUCENE-1088.patch Attached patch. All tests pass. I plan to commit some

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550710 ] Michael McCandless commented on LUCENE-753: --- OK my results on Win XP now agree with Yonik's. On UNIX & OS

[jira] Updated: (LUCENE-1044) Behavior on hard power shutdown

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1044: --- Attachment: LUCENE-1044.take5.patch Initial patch attached: * Created new commit

[jira] Updated: (LUCENE-1087) MultiSearcher.explain returns incorrect score/explanation relating to docFreq

2007-12-11 Thread Hoss Man (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man updated LUCENE-1087: - Description: Creating 2 different indexes, searching each individually and print score details and com

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550701 ] Michael McCandless commented on LUCENE-753: --- Thanks! I'll re-run. {quote} Well, at least we've learned th

[jira] Updated: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated LUCENE-753: Attachment: FileReadTest.java OK, uploading latest version of the test that should fix ChannelTrans

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550687 ] Michael McCandless commented on LUCENE-753: --- Doh!! Woops :) I will rerun... > Use NIO positional read to

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550685 ] Yonik Seeley commented on LUCENE-753: - I'll try fixing the transferTo test before anyone re-runs any tests. > Us

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550682 ] Yonik Seeley commented on LUCENE-753: - Mike, it looks like you are running with a bufsize of 6.5MB! Apologies for

[jira] Assigned: (LUCENE-1088) PriorityQueue 'wouldBeInserted' method

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-1088: -- Assignee: Michael McCandless > PriorityQueue 'wouldBeInserted' method > --

[jira] Created: (LUCENE-1088) PriorityQueue 'wouldBeInserted' method

2007-12-11 Thread Peter Keegan (JIRA)
PriorityQueue 'wouldBeInserted' method -- Key: LUCENE-1088 URL: https://issues.apache.org/jira/browse/LUCENE-1088 Project: Lucene - Java Issue Type: New Feature Components: Other Repo

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550677 ] Michael McCandless commented on LUCENE-753: --- I also just ran a test with 4 threads, random access, on Linux

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550675 ] Michael McCandless commented on LUCENE-753: --- I ran Yonik's most recent FileReadTest.java on the platforms b

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Yonik Seeley
On Dec 11, 2007 1:21 PM, Timo Nentwig <[EMAIL PROTECTED]> wrote: > On Tuesday 11 December 2007 14:32:12 Shai Erera wrote: > > For (1) - I can't explain it but I've run into documents with 0.0f scores. > > For (2) - this is a simple logic - if the lowest score in the queue is 'x' > > and you want to

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Timo Nentwig
On Tuesday 11 December 2007 14:32:12 Shai Erera wrote: > For (1) - I can't explain it but I've run into documents with 0.0f scores. > For (2) - this is a simple logic - if the lowest score in the queue is 'x' > and you want to top docs only, then there's no point in attempting to > insert a documen

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Michael McCandless
Shai Erera wrote: Hi, I will open an issue and create the patch. One thing I'm not sure of is the wouldBeInserted method you mentioned - in what context should it be used? And ... lessThan shouldn't be public, it can stay protected. Sorry, this is a method Peter suggested (see below) in

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Shai Erera
Hi, I will open an issue and create the patch. One thing I'm not sure of is the wouldBeInserted method you mentioned - in what context should it be used? And ... lessThan shouldn't be public, it can stay protected. On 12/11/07, Michael McCandless <[EMAIL PROTECTED]> wrote: > > > I think we can't

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Michael McCandless
I think we can't make lessThan public since that would cause subclasses to fail to compile (ie this breaks backwards compatibility)? Adding "wouldBeInserted()" seems OK? Mike Peter Keegan wrote: See my similar request from last March: http://www.nabble.com/FieldSortedHitQueue-enhancement-

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Michael McCandless
I agree that even though we don't see gains on the queries tested, there are in theory cases where there could be a great many allocations that would be saved. I think we should do Shai's suggested option 1 (add the method and change TDC to call it), change heap to be protected not privat

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Peter Keegan
See my similar request from last March: http://www.nabble.com/FieldSortedHitQueue-enhancement-to9733550.html#a9733550 Peter On Dec 11, 2007 11:54 AM, Nadav Har'El <[EMAIL PROTECTED]> wrote: > On Mon, Dec 10, 2007, Shai Erera wrote about "Performance Improvement for > Search using PriorityQueue":

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Nadav Har'El
On Mon, Dec 10, 2007, Shai Erera wrote about "Performance Improvement for Search using PriorityQueue": > Hi > > Lucene's PQ implements two methods: put (assumes the PQ has room for the > object) and insert (checks whether the object can be inserted etc.). The > implementation of insert() requires

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Shai Erera
For (1) - I can't explain it but I've run into documents with 0.0f scores. For (2) - this is a simple logic - if the lowest score in the queue is 'x' and you want to top docs only, then there's no point in attempting to insert a document with score lower than 'x' (it will not be added). Maybe I did

Re: [jira] Commented: (LUCENE-1068) Invalid behavior of StandardTokenizerImpl

2007-12-11 Thread Shai Erera
Hi I attached two patch files (for "java" and "test"). Due to a problem in my checkout project in Eclipse, I don't have them under "src". I also added a test and modified two tests in TestStandardAnalyzer. On Dec 10, 2007 11:44 PM, Grant Ingersoll (JIRA) <[EMAIL PROTECTED]> wrote: > >[ > htt

[jira] Updated: (LUCENE-1068) Invalid behavior of StandardTokenizerImpl

2007-12-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-1068: --- Attachment: StandardTokenizer-test-4.patch StandardTokenizer-java-4.patch Code fies

Caching FuzzyQuery

2007-12-11 Thread Timo Nentwig
Hi! Actually FuzzyQuery.rewrite() is pretty expensive so why not introduce a caching decorator? A WeakHashMap with key==IndexReader and value==LRU of BooleanQueries. Timo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additio

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Timo Nentwig
On Monday 10 December 2007 09:15:12 Paul Elschot wrote: > The current TopDocCollector only allocates a ScoreDoc when the given > score causes a new ScoreDoc to be added into the queue, but it does I actually wrote my own HitCollector and now wonder about TopDocCollector: public void collect(int

[jira] Commented: (LUCENE-1081) Remove the "Experimental" warnings from search.function package

2007-12-11 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550419 ] Doron Cohen commented on LUCENE-1081: - {quote} I think we should resolve LUCENE-1085 first and move this to 2.4?

[jira] Updated: (LUCENE-1081) Remove the "Experimental" warnings from search.function package

2007-12-11 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen updated LUCENE-1081: Fix Version/s: (was: 2.3) 2.4 Affects Version/s: (was: 2.3)

[jira] Reopened: (LUCENE-944) Remove deprecated methods in BooleanQuery

2007-12-11 Thread Michael Busch (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Busch reopened LUCENE-944: -- Lucene Fields: [Patch Available] (was: [Patch Available, New]) You are right, Grant. I will r

[jira] Commented: (LUCENE-1081) Remove the "Experimental" warnings from search.function package

2007-12-11 Thread Michael Busch (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550411 ] Michael Busch commented on LUCENE-1081: --- I think we should resolve LUCENE-1085 first and move this to 2.4? >

Re: Performance Improvement for Search using PriorityQueue

2007-12-11 Thread Shai Erera
Hi Back from the experiments lab with more results. I've used two indexes (1 and 10 million documents) and ran over the two 2000 queries. Each run was executed 4 times and I paste here the average of the latest 3 (to eliminate any caching that is done by the OS and to mimic systems that are alread

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Brian Pinkerton (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550376 ] Brian Pinkerton commented on LUCENE-753: BTW, I think the performance win with Yonik's patch for some workloa

[jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput

2007-12-11 Thread Brian Pinkerton (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550351 ] Brian Pinkerton commented on LUCENE-753: Yeah, the file was full of zeroes. But I created the files w/o hole