too :). At least the one I received.
But never mind that ... as long as we both agree the implementation should
change. I didn't mean to say anything bad about what you did .. I know the
limitations you had to work with.
> Parallel incremen
an upgraded version that works with 3.0 within IBM,
Shai.
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://issues.apache.org/jira/browse/LUCENE-1879
> Project: Lucene - Java
>
the intent, and was likely further confused
due to the fact that Michael B and I discussed it over tasty Belgian Beer in
Oakland. I'll open a discussion on list for incremental field updates.
> Parallel incremental indexing
> -
>
>
is interesting, and
deserves discussion, but in a separate issue/thread?
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://issues.apache.org/jira/browse/LUCENE-1879
> Project: Lucene - Ja
t more than a few thousand
changes in a minute or two and the background merger would be responsible for
keeping the total number of disjoint documents low.
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://i
will just improve the performance of that process?
This might require a bigger change to IW then I had anticipated, but perhaps
it's worth it.
What do you think?
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
&g
port multi-threaded parallel-indexing. If we
have single-threaded DocumentsWriters, then it should be easy to have a
ParallelDocumentsWriter?
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://issues.apa
e at
the right times. Vs the current approach that makes "faker" merge
policy/scheduler (I think?).
Some of this will require IW to open up some APIs -- eg making docID
assignment a separate method call. Likely many of these will just be
protected APIs w/in IW.
> Pa
art implementing today. If possible, I'd like to get this out in 3.1.
I'll try to break this issue down to as many issues as I can, to make the
contributions containable. We should just keep in mind for each such issue the
larger picture it solves.
I'd appreciate your comments.
the assignment of fields
into separate partitions?
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://issues.apache.org/jira/browse/LUCENE-1879
> Project: Lucene - Java
>
LUCENE-2026 right, I think we can implement parallel
indexing in this segment-oriented way nicely.
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://issues.apache.org/jira/browse/LUCENE-1879
>
he fields are then written
according to this partition. Eg if I map "body" to partition 1, and
"title" to partition 2, then I'd have two sets of postings files for
each segment.
Could something like this work?
> Parallel incremental indexing
> --
because it runs on top of Lucene's APIs (I wanted the code
to run with an unmodified Lucene jar).
Next I'll work on a patch that runs with current trunk.
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
>
arallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://issues.apache.org/jira/browse/LUCENE-1879
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
>
attach the tar + md5 here and send the signed software grant to you,
Grant?
> Parallel incremental indexing
> -
>
> Key: LUCENE-1879
> URL: https://issues.apache.org/jira/browse/LUCENE-1879
> Project: Lucene -
Parallel incremental indexing
-
Key: LUCENE-1879
URL: https://issues.apache.org/jira/browse/LUCENE-1879
Project: Lucene - Java
Issue Type: New Feature
Components: Index
Reporter: Michael
On Sun, Aug 30, 2009 at 6:08 AM, Yonik Seeley wrote:
> Cool stuff!
>
>
Thanks. It's actually really fun to work on! After I had the parallel
indexing working and didn't have to worry anymore about how to manage
parallel indexes the fun of implementing cool features on top of this
started. I hope y
Cool stuff!
We should also think about how to do single document field updates or
field adds since that is the most common usecase - not that it needs
to be implemented in the first version, but kept in mind so we don't
box ourselves in.
Doug mentioned some ideas he had in passing almost a year a
Michael Busch wrote:
Hi all,
I just added a wiki page for a new feature I'd like to add to
Lucene. Please take a look at the link. I will add more details and
diagrams to the page, but for now it should give a rough idea about
how to implement it:
http://wiki.apache.org/lucene-java/ParallelIncr
Hi all,
I just added a wiki page for a new feature I'd like to add to
Lucene. Please take a look at the link. I will add more details and
diagrams to the page, but for now it should give a rough idea about
how to implement it:
http://wiki.apache.org/lucene-java/ParallelIncrementalIndexing
Basic
20 matches
Mail list logo
