Re: IndexReader.isCurrent in presence of many files

2007-05-11 Thread Doron Cohen
[EMAIL PROTECTED] wrote on 11/05/2007 20:07:11: > However, is there a way to portably stat a directory? That could lead > to a fast-path if no new files were added. Do you mean something like File.lastModified() on the index dir? -

Re: IndexReader.isCurrent in presence of many files

2007-05-11 Thread Yonik Seeley
On 5/11/07, Doron Cohen <[EMAIL PROTECTED]> wrote: However, isCurrent() may be called before every query. That's never going to be a high performance architecture. However, is there a way to portably stat a directory? That could lead to a fast-path if no new files were added. -Yonik

Re: IndexReader.isCurrent in presence of many files

2007-05-11 Thread Doron Cohen
Chris Hostetter <[EMAIL PROTECTED]> wrote on 11/05/2007 17:10:54: > > : If this really turns to be related to having many files in the index > : dir, could we maintain SEGMENTS_N files in a sub-directory..? > > I haven't done much experimenting / performance testing of File > operations in Java, b

Re: Token/Payload API

2007-05-11 Thread Yonik Seeley
On 5/11/07, Grant Ingersoll <[EMAIL PROTECTED]> wrote: On May 11, 2007, at 4:31 PM, Yonik Seeley wrote: > I hadn't kept up with the payload discussion/patch, and just got > around to looking at Token. > > public class Token implements Cloneable { > String termText;

Re: IndexReader.isCurrent in presence of many files

2007-05-11 Thread Chris Hostetter
: If this really turns to be related to having many files in the index : dir, could we maintain SEGMENTS_N files in a sub-directory..? I haven't done much experimenting / performance testing of File operations in Java, but just from looking at the java1.4.2 javadocs it seems like it *might* be po

[jira] Updated: (LUCENE-879) Document number integrity merge policy

2007-05-11 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-879: --- Attachment: LUNCENE-879.diff This new patch allows consumer to, based on a primary key, delete a docu

Re: Token/Payload API

2007-05-11 Thread Grant Ingersoll
On May 11, 2007, at 4:31 PM, Yonik Seeley wrote: I hadn't kept up with the payload discussion/patch, and just got around to looking at Token. public class Token implements Cloneable { String termText; // the text of the term int startOffset;

Re: IndexReader.isCurrent in presence of many files

2007-05-11 Thread Yonik Seeley
On 5/11/07, Doron Cohen <[EMAIL PROTECTED]> wrote: If this really turns to be related to having many files in the index dir, could we maintain SEGMENTS_N files in a sub-directory..? 1) There might be slight incompatibilities with tools that assume a lucene index is a bunch of files in the index

[jira] Commented: (LUCENE-523) FSDirectory.openFile(String) causes ClassCastException

2007-05-11 Thread Jukka Zitting (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495174 ] Jukka Zitting commented on LUCENE-523: -- We worked around the issue in Jackrabbit by using the new openInput meth

[jira] Commented: (LUCENE-523) FSDirectory.openFile(String) causes ClassCastException

2007-05-11 Thread Daniel Naber (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495163 ] Daniel Naber commented on LUCENE-523: - The issue at Jackrabbit is closed, so I guess this can be closed too? I'll

Token/Payload API

2007-05-11 Thread Yonik Seeley
I hadn't kept up with the payload discussion/patch, and just got around to looking at Token. public class Token implements Cloneable { String termText; // the text of the term int startOffset; // start in source text int endOffset

[jira] Commented: (LUCENE-879) Document number integrity merge policy

2007-05-11 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495145 ] Karl Wettin commented on LUCENE-879: Doron, thanks for the input. I have not had time to read and think everyth

[jira] Commented: (LUCENE-879) Document number integrity merge policy

2007-05-11 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495127 ] Doron Cohen commented on LUCENE-879: I skimmed through the patch and I understand that all terms and postings of

IndexReader.isCurrent in presence of many files

2007-05-11 Thread Doron Cohen
If this really turns to be related to having many files in the index dir, could we maintain SEGMENTS_N files in a sub-directory..? Doron -- Forward -- -- http://www.mail-archive.com/[EMAIL PROTECTED]/msg14398.html Chris Hostetter <[EMAIL PROTECTED]> wrote on 11/05/2007 11:02:50: > > : Are ther

[jira] Commented: (LUCENE-878) Concept Search

2007-05-11 Thread Hoss Man (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495110 ] Hoss Man commented on LUCENE-878: - http://lucene.apache.org/java/docs/mailinglists.html has all of the info on the

[jira] Commented: (LUCENE-878) Concept Search

2007-05-11 Thread Charles S Patridge (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495104 ] Charles S Patridge commented on LUCENE-878: --- Dear Grant, I'm new to this list - can you show me where

[jira] Commented: (LUCENE-879) Document number integrity merge policy

2007-05-11 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12495103 ] Karl Wettin commented on LUCENE-879: Forgot to tell about all the effects: 1. Replaces deleted documents with a

[jira] Created: (LUCENE-879) Document number integrity merge policy

2007-05-11 Thread Karl Wettin (JIRA)
Document number integrity merge policy -- Key: LUCENE-879 URL: https://issues.apache.org/jira/browse/LUCENE-879 Project: Lucene - Java Issue Type: Improvement Components: Store Affects Versio

[jira] Updated: (LUCENE-879) Document number integrity merge policy

2007-05-11 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-879: --- Attachment: LUNCENE-879.diff > Document number integrity merge policy > -

Re: [jira] Created: (LUCENE-854) Create merge policy that doesn't periodically inadvertently optimize

2007-05-11 Thread Michael McCandless
"Yonik Seeley" <[EMAIL PROTECTED]> wrote: > On 5/2/07, Michael McCandless <[EMAIL PROTECTED]> wrote: > > It would merge based on size (not # docs), would be free to merge > > adjacent segments (not just rightmost segments), and would merge N > > (configurable) at a time. > > Hopefully it will alw

Re: [jira] Created: (LUCENE-854) Create merge policy that doesn't periodically inadvertently optimize

2007-05-11 Thread Michael McCandless
"Yonik Seeley" <[EMAIL PROTECTED]> wrote: > On 5/3/07, Michael McCandless <[EMAIL PROTECTED]> wrote: > > I like your idea to keep "delete count per segment" in the segments > > file. This information is certainly useful to the merge policy > > because it should proportionally reducde a segments s

[jira] Closed: (LUCENE-878) Concept Search

2007-05-11 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll closed LUCENE-878. -- Resolution: Invalid Please ask these types of questions on the user mailing list before putting

[jira] Created: (LUCENE-878) Concept Search

2007-05-11 Thread Charles S Patridge (JIRA)
Concept Search -- Key: LUCENE-878 URL: https://issues.apache.org/jira/browse/LUCENE-878 Project: Lucene - Java Issue Type: New Feature Components: Build Environment: Unix - unsure of anything else R