[jira] Commented: (LUCENE-808) bufferDeleteTerm in IndexWriter might flush prematurely

2007-02-21 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12474924 ] Doron Cohen commented on LUCENE-808: [ moving discussion back to JIRA ] Ning Li wrote: > On 2/21/07, Doron Cohe

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Artem Vasiliev (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Vasiliev updated LUCENE-769: -- Description: It's a small addition to Lucene that significantly lowers memory consumption and

Re: Concurrent merge

2007-02-21 Thread Yonik Seeley
On 2/21/07, Doron Cohen <[EMAIL PROTECTED]> wrote: > > The downside is another complexity increase though. I think complexity can be divided in two: (1) more complex synchronization and data-manipulation/accounting (2) multi-threading. The multi-threading becoming part of and responsibility of

Re: Concurrent merge

2007-02-21 Thread Doron Cohen
> > The downside is another complexity increase though. I think complexity can be divided in two: (1) more complex synchronization and data-manipulation/accounting (2) multi-threading. The multi-threading becoming part of and responsibility of Lucne seems quite a change to me. Lucene's being sing

Re: [jira] Commented: (LUCENE-808) bufferDeleteTerm in IndexWriter might flush prematurely

2007-02-21 Thread Ning Li
On 2/21/07, Doron Cohen (JIRA) <[EMAIL PROTECTED]> wrote: Imagine the application and Lucene could talk, with the current implementation we could hear something like this: ... However, there could be multiple threads updating the same index. For example, thread 1 deletes the term "id:5" twice,

Re: Concurrent merge

2007-02-21 Thread robert engels
I think when you start discussing background threads you need to think server environment. It is fairly trivial there. I have pushed to move Lucene in that direction, rather than the multiple client accessing a shared resource via a network filesystem. No decent server product works this

Re: Concurrent merge

2007-02-21 Thread Yonik Seeley
On 2/21/07, Doron Cohen <[EMAIL PROTECTED]> wrote: Ning Li wrote: > There are three main challenges in enabling concurrent merge: > 1 a robust merge policy > 2 detect when merge lags document additions/deletions > 3 how to slow down document additions/deletions (and amortize > the cost

Re: Concurrent merge

2007-02-21 Thread Doron Cohen
Ning Li wrote: > There are three main challenges in enabling concurrent merge: > 1 a robust merge policy > 2 detect when merge lags document additions/deletions > 3 how to slow down document additions/deletions (and amortize > the cost) when merge falls behind I wonder what it means for

Re: Concurrent merge

2007-02-21 Thread robert engels
I agree, that is why I suggested using the queue. You will add new segments to be merged to a queue. The merge thread pulls segments form the queue when it is free to do so. The addDocument() creates a new segment and adds it to the queue. If the queue is full, the add to the queue will blo

Re: Concurrent merge

2007-02-21 Thread Nadav Har'El
On Tue, Feb 20, 2007, Ning Li wrote about "Concurrent merge": > I think it's possible for another version of IndexWriter to have > a concurrent merge thread so that disk segments could be merged > while documents are being added or deleted. > This would be beneficial not only because it will improv

Re: Concurrent merge

2007-02-21 Thread Nadav Har'El
On Tue, Feb 20, 2007, robert engels wrote about "Re: Concurrent merge": > What about a queue of segments to merge. The add document will add > segments to the queue, if the queue contains too many segments it > blocks. > > Another thread reads the segments from the queue and merges them. > >

Re: ANN: Luke 0.7 released

2007-02-21 Thread Antony Bowesman
Great Andrzej, that fixed it. Thanks. Antony Andrzej Bialecki wrote: Antony Bowesman wrote: With the luke.jar download, it throws an Exception java.lang.NoClassDefFoundError: org/apache/lucene/index/IndexGate Fixed - I uploaded an updated jar. Sorry for the problem. ---

Re: ANN: Luke 0.7 released

2007-02-21 Thread Andrzej Bialecki
Antony Bowesman wrote: With the luke.jar download, it throws an Exception java.lang.NoClassDefFoundError: org/apache/lucene/index/IndexGate Fixed - I uploaded an updated jar. Sorry for the problem. -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __

Re: ANN: Luke 0.7 released

2007-02-21 Thread Andrzej Bialecki
Antony Bowesman wrote: Hi Andrzej, Thanks for this - it's a great tool. With the luke.jar download, it throws an Exception java.lang.NoClassDefFoundError: org/apache/lucene/index/IndexGate at org.getopt.luke.Luke.getIndexFileNames(Unknown Source) at org.getopt.luke.Luke.showFil

Re: ANN: Luke 0.7 released

2007-02-21 Thread Antony Bowesman
Hi Andrzej, Thanks for this - it's a great tool. With the luke.jar download, it throws an Exception java.lang.NoClassDefFoundError: org/apache/lucene/index/IndexGate at org.getopt.luke.Luke.getIndexFileNames(Unknown Source) at org.getopt.luke.Luke.showFiles(Unknown Source)

Re: removal of JIRA attachments

2007-02-21 Thread Doug Cutting
Chris Hostetter wrote: ... if because of jira settings we are forced to choose between: 1) deletes are completely impossible 2) only members of "lucene-developers" jira group can delete 3) people can delete their own attachments ...i would choose #1 for safety (better to have a ton of c

Re: publishing lucene 2.0.0 contribs to ibiblio?

2007-02-21 Thread Joerg Hohwiller
Hi Erik, Hi everybody, > Anyone gonna tackle this? > > I personally have never used Maven or the POM stuff, and have never > published our releases to the repository. Could this step be automated > somehow? Volunteers? Should not be that hard to put 2 files somewhere (org/apache/lucene/lucene-h

ANN: Luke 0.7 released

2007-02-21 Thread Andrzej Bialecki
Hi all, I'm happy to announce that a new version of Luke - the Lucene Index Toolbox - is now available. As usually, you can get it from: http://www.getopt.org/luke Highlights of this release: * support for Lucene 2.1.0 release and earlier * pagination of search results * support for many

Re: removal of JIRA attachments

2007-02-21 Thread Chris Hostetter
: Yes. It is controlled by the "Delete Issues" permission: "Ability to : delete issues, comments and attachments." Lucene Java uses Jira's : Standard Permissions, and under those, the Contributor role has this : permission. We can switch Lucene Java to use a clone of Standard : Permissions call

Re: removal of JIRA attachments

2007-02-21 Thread Doug Cutting
Yonik Seeley wrote: I don't think most of us can see JIRA roles, so I don't know how they map to the groups like lucene-developer, but I think committers (or the lucene-developers JIRA group) still need removal privs (to remove spam, IP issues, *real* removal requests, etc). Sorry. For Lucene

[jira] Resolved: (LUCENE-809) No source or classes for the similarity contrib

2007-02-21 Thread Guy Loewy (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guy Loewy resolved LUCENE-809. -- Resolution: Invalid Classes were moved to the queries contrib jar > No source or classes for the simil

Re: removal of JIRA attachments

2007-02-21 Thread Yonik Seeley
On 2/21/07, Doug Cutting <[EMAIL PROTECTED]> wrote: Michael McCandless wrote: > Is this (ability to delete patches) something we can disable in Jira? Yes. It is controlled by the "Delete Issues" permission: "Ability to delete issues, comments and attachments." Lucene Java uses Jira's Standard

Re: removal of JIRA attachments

2007-02-21 Thread Doug Cutting
Michael McCandless wrote: Is this (ability to delete patches) something we can disable in Jira? Yes. It is controlled by the "Delete Issues" permission: "Ability to delete issues, comments and attachments." Lucene Java uses Jira's Standard Permissions, and under those, the Contributor role

Re: removal of JIRA attachments

2007-02-21 Thread Michael McCandless
On Wed, 21 Feb 2007 15:04:01 -0500, "Yonik Seeley" <[EMAIL PROTECTED]> said: > As a general rule, I don't think we should be removing attachments (of > code) on JIRA issues. +1 I've been hit by this in the past when I had wanted to diff the original patch against the new one to see details on wh

Re: removal of JIRA attachments

2007-02-21 Thread Doug Cutting
+1 We should not generally remove obsolete attachments in Jira. Doug Yonik Seeley wrote: As a general rule, I don't think we should be removing attachments (of code) on JIRA issues. - it represents a loss of code... it's not like subversion where you can go back in time. earlier patches can a

Re: [jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread karl wettin
21 feb 2007 kl. 20.55 skrev Yonik Seeley: I think some of the attachments that were deleted were uploaded by robert engels not just Artem Vasiliev (the person who requested deletion). Well I feel silly now. - To unsubscrib

Re: removal of JIRA attachments

2007-02-21 Thread Yonik Seeley
And another reason I just thought of... it provides better tracking of IP. Once something goes in JIRA, multiple people might start contributing ideas. If you remove all previous versions, there is no record of the original submission, or who contributed what to the final. -Yonik On 2/21/07, Yo

removal of JIRA attachments

2007-02-21 Thread Yonik Seeley
As a general rule, I don't think we should be removing attachments (of code) on JIRA issues. - it represents a loss of code... it's not like subversion where you can go back in time. earlier patches can also sometimes be simpler in places, or can take a different approach, and there can be value

[jira] Resolved: (LUCENE-793) Javadocs should explain possible causes for IOExceptions

2007-02-21 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-793. --- Resolution: Fixed Fix Version/s: 2.2 OK, I just committed this. I stuck with

[jira] Commented: (LUCENE-808) bufferDeleteTerm in IndexWriter might flush prematurely

2007-02-21 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12474808 ] Doron Cohen commented on LUCENE-808: Ning Li wrote: > The code correctly reflects its designed semantics: > numB

Re: [jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Yonik Seeley
On 2/21/07, Karl Wettin (JIRA) <[EMAIL PROTECTED]> wrote: [ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: IndexReaderUtils.java) I

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: QueryFilter.java) > [PATCH] Performance improvement for some cases of sorted se

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: DocCachingSorting.patch) > [PATCH] Performance improvement for some cases of so

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: IndexReaderUtils.java) > [PATCH] Performance improvement for some cases of sort

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: DocCachingSorting.patch) > [PATCH] Performance improvement for some cases of so

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: StoredFieldSorting.patch) > [PATCH] Performance improvement for some cases of s

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: selfContained.patch) > [PATCH] Performance improvement for some cases of sorted

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: selfContained.patch) > [PATCH] Performance improvement for some cases of sorted

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-769: --- Attachment: (was: selfContained.patch) > [PATCH] Performance improvement for some cases of sorted

[jira] Updated: (LUCENE-809) No source or classes for the similarity contrib

2007-02-21 Thread Guy Loewy (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guy Loewy updated LUCENE-809: - Fix Version/s: 2.1 > No source or classes for the similarity contrib > --

[jira] Created: (LUCENE-809) No source or classes for the similarity contrib

2007-02-21 Thread Guy Loewy (JIRA)
No source or classes for the similarity contrib --- Key: LUCENE-809 URL: https://issues.apache.org/jira/browse/LUCENE-809 Project: Lucene - Java Issue Type: Bug Affects Versions: 2.1

[jira] Updated: (LUCENE-769) [PATCH] Performance improvement for some cases of sorted search

2007-02-21 Thread Artem Vasiliev (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Vasiliev updated LUCENE-769: -- Description: The attachment №5 from 31 Jan is the last and the whole patch for this issue. Guy

Re: Concurrent merge

2007-02-21 Thread Yonik Seeley
On 2/21/07, Ning Li <[EMAIL PROTECTED]> wrote: I agree that the current blocking model works for some applications, especially if the indexes are batch built. But other applications, e.g. with online indexes, would greatly benefit from a non-blocking model. Most systems that merge data support b

Re: [jira] Created: (LUCENE-808) bufferDeleteTerm in IndexWriter might flush prematurely

2007-02-21 Thread Ning Li
The code correctly reflects its designed semantics: numBufferedDeleteTerms is a simple sum of terms passed to updateDocument or deleteDocuments. If the first of two successive calls to the same term should be considered no op if no docs were added in between, shouldn't the first also be considere

[jira] Assigned: (LUCENE-791) Update the Wiki

2007-02-21 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll reassigned LUCENE-791: -- Assignee: Grant Ingersoll > Update the Wiki > --- > > Key:

Re: Concurrent merge

2007-02-21 Thread robert engels
The point I was trying to make is that even if you use a background thread for merging, that at some point it may block, if the document additions out-pace the merging. On Feb 21, 2007, at 11:47 AM, Ning Li wrote: I agree that the current blocking model works for some applications, especial

[jira] Commented: (LUCENE-791) Update the Wiki

2007-02-21 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12474773 ] Doug Cutting commented on LUCENE-791: - To summarize: someone should file an issue in the Infrastructure Jira req

Re: Concurrent merge

2007-02-21 Thread Ning Li
I agree that the current blocking model works for some applications, especially if the indexes are batch built. But other applications, e.g. with online indexes, would greatly benefit from a non-blocking model. Most systems that merge data support background merges. As long as we keep it simple (

Re: Lucene 2.1, soon

2007-02-21 Thread Doug Cutting
Chris Hostetter wrote: : since we have different sets of committers for different sub-projects, : it probably makes more sense to have per-subproject KEYS files. That : way folks can add their own keys without altering our current permission : scheme. yeah, it would certianly be a "paradigm shi

Re: IndexWriter#deleteDocuments

2007-02-21 Thread Ning Li
On 2/20/07, karl wettin <[EMAIL PROTECTED]> wrote: Could the reader per segment be replaced by one single MultiReader created by the original indexDeleterFactory()? Or are the segments partially the RAMDirectory of the writer, partially the persistent index? All segments are disk segments. Howe

[jira] Updated: (LUCENE-759) Add n-gram tokenizers to contrib/analyzers

2007-02-21 Thread Otis Gospodnetic (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic updated LUCENE-759: Attachment: LUCENE-759.patch Here is the proper version. This one is essentially the Luce

[jira] Assigned: (LUCENE-808) bufferDeleteTerm in IndexWriter might flush prematurely

2007-02-21 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen reassigned LUCENE-808: -- Assignee: Doron Cohen > bufferDeleteTerm in IndexWriter might flush prematurely > -

[jira] Updated: (LUCENE-808) bufferDeleteTerm in IndexWriter might flush prematurely

2007-02-21 Thread Doron Cohen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen updated LUCENE-808: --- Attachment: successive_bufferDeleteTerm.patch > bufferDeleteTerm in IndexWriter might flush premature

[jira] Created: (LUCENE-808) bufferDeleteTerm in IndexWriter might flush prematurely

2007-02-21 Thread Doron Cohen (JIRA)
bufferDeleteTerm in IndexWriter might flush prematurely --- Key: LUCENE-808 URL: https://issues.apache.org/jira/browse/LUCENE-808 Project: Lucene - Java Issue Type: Bug Components