[
https://issues.apache.org/jira/browse/LUCENE-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless resolved LUCENE-870.
---
Resolution: Fixed
> add concurrent merge pol
on LUCENE-847.
> add concurrent merge policy
> ---
>
> Key: LUCENE-870
> URL: https://issues.apache.org/jira/browse/LUCENE-870
> Project: Lucene - Java
> Issue Type: New Feature
>
[
https://issues.apache.org/jira/browse/LUCENE-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless updated LUCENE-870:
--
Fix Version/s: 2.3
> add concurrent merge pol
o with 4 drives
in a RAID 0 array).
> add concurrent merge policy
> ---
>
> Key: LUCENE-870
> URL: https://issues.apache.org/jira/browse/LUCENE-870
> Project: Lucene - Java
> Issue Type: New Feat
lenging to parallelize. It's synchronized now, but you
probably would rather it not be? It's complicated by the fact that doc stores
are shared by multiple segments and so non-conflicting merges may stll share
doc stores.
> add concurrent merge policy
> --
47 is supposed to the
non-concurrent changes.
> add concurrent merge policy
> ---
>
> Key: LUCENE-870
> URL: https://issues.apache.org/jira/browse/LUCENE-870
> Project: Lucene - Java
>
is to
LUCENE-847, not LUCENE-848. Perhaps someone with JIRA "manage links"
permissions can delete the wrong one.
> add concurrent merge policy
> ---
>
> Key: LUCENE-870
> URL: https://issues.apache.org/jira/browse/LU
add concurrent merge policy
---
Key: LUCENE-870
URL: https://issues.apache.org/jira/browse/LUCENE-870
Project: Lucene - Java
Issue Type: New Feature
Components: Index
Reporter: Steven Parkes
FYI: Patch submitted in http://issues.apache.org/jira/browse/LUCENE-847.
Cheers,
Ning
"Here is a patch for concurrent merge as discussed in:
http://www.gossamer-threads.com/lists/lucene/java-dev/45651?search_string=concurrent%20merge;#45651
"I put it under this issue because it he
Tag - Search - Share
- Original Message
From: Ning Li <[EMAIL PROTECTED]>
To: java-dev@lucene.apache.org
Sent: Friday, March 2, 2007 5:44:55 PM
Subject: Re: Concurrent merge
Many good points! Thanks, guys!
When background merge is employed, document additions can
out-pace merging
Many good points! Thanks, guys!
When background merge is employed, document additions can
out-pace merging, no matter how many background merge threads
are used. Blocking has to happen at some point.
So, if we do anything, we make it simple. I agree with what
Robert and Yonik have proposed: docu
On 2/21/07, Doron Cohen <[EMAIL PROTECTED]> wrote:
> > The downside is another complexity increase though.
I think complexity can be divided in two:
(1) more complex synchronization and data-manipulation/accounting
(2) multi-threading.
The multi-threading becoming part of and responsibility of
Yonik Seeley wrote:
>
> > On 2/21/07, Doron Cohen <[EMAIL PROTECTED]> wrote:
> >> Ning Li wrote:
> >>
> >> > There are three main challenges in enabling concurrent merge:
> >> > 1 a robust merge policy
> >> > 2 detect wh
way.
On Feb 21, 2007, at 5:23 PM, Yonik Seeley wrote:
On 2/21/07, Doron Cohen <[EMAIL PROTECTED]> wrote:
Ning Li wrote:
> There are three main challenges in enabling concurrent merge:
> 1 a robust merge policy
> 2 detect when merge lags document additions/deletions
>
On 2/21/07, Doron Cohen <[EMAIL PROTECTED]> wrote:
Ning Li wrote:
> There are three main challenges in enabling concurrent merge:
> 1 a robust merge policy
> 2 detect when merge lags document additions/deletions
> 3 how to slow down document additions/deletions (and
Ning Li wrote:
> There are three main challenges in enabling concurrent merge:
> 1 a robust merge policy
> 2 detect when merge lags document additions/deletions
> 3 how to slow down document additions/deletions (and amortize
> the cost) when merge falls behind
I wonde
rote about "Concurrent merge":
I think it's possible for another version of IndexWriter to have
a concurrent merge thread so that disk segments could be merged
while documents are being added or deleted.
This would be beneficial not only because it will improve indexing
performance whe
On Tue, Feb 20, 2007, Ning Li wrote about "Concurrent merge":
> I think it's possible for another version of IndexWriter to have
> a concurrent merge thread so that disk segments could be merged
> while documents are being added or deleted.
> This would be benefici
On Tue, Feb 20, 2007, robert engels wrote about "Re: Concurrent merge":
> What about a queue of segments to merge. The add document will add
> segments to the queue, if the queue contains too many segments it
> blocks.
>
> Another thread reads the segments from
On 2/21/07, Ning Li <[EMAIL PROTECTED]> wrote:
I agree that the current blocking model works for some applications,
especially if the indexes are batch built.
But other applications, e.g. with online indexes, would greatly
benefit from a non-blocking model. Most systems that merge data
support b
The point I was trying to make is that even if you use a background
thread for merging, that at some point it may block, if the document
additions out-pace the merging.
On Feb 21, 2007, at 11:47 AM, Ning Li wrote:
I agree that the current blocking model works for some applications,
especial
I agree that the current blocking model works for some applications,
especially if the indexes are batch built.
But other applications, e.g. with online indexes, would greatly
benefit from a non-blocking model. Most systems that merge data
support background merges. As long as we keep it simple (
On 2/20/07, robert engels <[EMAIL PROTECTED]> wrote:
What about a queue of segments to merge. The add document will add
segments to the queue, if the queue contains too many segments it
blocks.
Another thread reads the segments from the queue and merges them.
This would effectively block adding
different than what happens now.
On Feb 20, 2007, at 1:22 PM, Ning Li wrote:
I think it's possible for another version of IndexWriter to have
a concurrent merge thread so that disk segments could be merged
while documents are being added or deleted.
This would be beneficial not only because it
I think it's possible for another version of IndexWriter to have
a concurrent merge thread so that disk segments could be merged
while documents are being added or deleted.
This would be beneficial not only because it will improve indexing
performance when there are enough system resources
25 matches
Mail list logo
