I applied the patch, and made code changes to use it. It did not make
any appreciable difference in performance over our current code
(delete using IndexReader and then update the documents using
IndexWriter - each document has a unique "key").
I attempted to evaluate the code on its own, b
Robert, it's better to put your comments in JIRA, where Ning Li is more likely
to see them.
As for performance, it looks like the biggest gain is when one has small
interleaving add/delete batches. It sounds like your app doesn't have that and
has fewer larger add/delete batches.
I do agree a
I don't like "mucking up" JIRA with "commentary:. I thought emails
were more approproate, and then update JIRA with more pertinent info.
Anyway, my test did exercise the small batches, in that in our
incremental updates we delete the documents with the unique term, and
then add the new (whi
RAMDirectory.close() should have a comment about not releasing any resources
Key: LUCENE-623
URL: http://issues.apache.org/jira/browse/LUCENE-623
Project: Lucene - Java
Type: Improvement
On Wed, Jul 05, 2006, Paul Elschot wrote about "Re: Flexible index format /
Payloads Cont'd":
> > Ok, then, I thought to myself - the normal queries and scorers only work
> > on the document level and don't use positions - but SpanQueries have
> positions
> > so I can create some sort of Proximit
[ http://issues.apache.org/jira/browse/LUCENE-623?page=all ]
Nadav Har'El updated LUCENE-623:
Attachment: ramdirectory.diff
I propose a trivial patch, which does two very simple things:
1. RAMDirectory.close(), instead of being a no-op, sets files=null.
On 7/6/06, robert engels <[EMAIL PROTECTED]> wrote:
I applied the patch, and made code changes to use it. It did not make
any appreciable difference in performance over our current code
Yes, I'm not sure the performance comparisons made in the patch
description are apples-to-apples. And hopefu
[
http://issues.apache.org/jira/browse/LUCENE-565?page=comments#action_12419580 ]
Ning Li commented on LUCENE-565:
For an overview of my changes, I'll repeat some of what I said in
my earlier e-mail (see
http://www.gossamer-threads.com/lists/lucene/java-dev
Hi Otis and Robert,
I added an overview of my changes in JIRA. Hope that helps.
> Anyway, my test did exercise the small batches, in that in our
> incremental updates we delete the documents with the unique term, and
> then add the new (which is what I assumed this was improving), and I
> saw o a
Hi Yonik,
> When one interleaves adds and deletes, it isn't the case that
> indexreaders and indexwriters need to be opened and closed each
> interleave.
I'm not sure I understand this. Could you elaborate?
I thought IndexWriter acquires the write lock and holds it until
it is done. This will pr
I guess we just chose a much simpler way to do this...
Even with you code changes, to see the modification made using the
IndexWriter, it must be closed, and a new IndexReader opened.
So a far simpler way is to get the collection of updates first, then
using opened indexreader,
for each doc
robert engels wrote on 07/06/2006 12:24 PM:
> I guess we just chose a much simpler way to do this...
>
> Even with you code changes, to see the modification made using the
> IndexWriter, it must be closed, and a new IndexReader opened.
>
> So a far simpler way is to get the collection of updates fi
> Even with you code changes, to see the modification made using the
> IndexWriter, it must be closed, and a new IndexReader opened.
That behaviour remains the same.
> So a far simpler way is to get the collection of updates first, then
> using opened indexreader,
> for each doc in collection
>
[ http://issues.apache.org/jira/browse/LUCENE-623?page=all ]
Hoss Man reassigned LUCENE-623:
---
Assign To: Hoss Man
> RAMDirectory.close() should have a comment about not releasing any resources
> -
I think that patch is for a different scenario, the one where you can't wait to
batch deletes and adds, and want/need to execute them more frequently and in
order they really are happening, without grouping them.
Otis
- Original Message
From: robert engels <[EMAIL PROTECTED]>
To: java-
I guess I don't see the difference...
You need the write lock to use the indexWriter, and you also need the
write lock to perform a deletion, so if you just get the write lock
you can perform the deletion and the add, then close the writer.
I have asked how this submission optimizes anythin
[
http://issues.apache.org/jira/browse/LUCENE-565?page=comments#action_12419605 ]
Otis Gospodnetic commented on LUCENE-565:
-
Thanks for all the information about coding changes, that makes it easier to
understand the diff.
Ideally this will become c
Interestingly enough, this patch triggers an NPE in one ofthe tests...
java.lang.NullPointerException at
org.apache.lucene.store.RAMDirectory$1.obtain(RAMDirectory.java:184) at
org.apache.lucene.store.Lock.obtain(Lock.java:51) at
org.apache.lucene.store.Lock$With.run(Lock.java:98) at
org.apache.l
> Yonik mentioned this in email. It does sound like a better place for
> this might be in a higher level class. IndexWriter would really not
> be just a writer/appender once delete functionality is added to it,
> even if it's the IndexReaders behind the scenes doing the work. So
> if you are goi
I think I finally see how this is supposed to optimize - basically
because it remember the terms, and then does the batch deletions.
We avoid all of this messiness by just making sure each document has
a primary key and we always remove/update by primary key and we can
keep the operations i
[ http://issues.apache.org/jira/browse/LUCENE-623?page=all ]
Hoss Man resolved LUCENE-623:
-
Resolution: Fixed
Patch commited.
Note that this patch caused an NPE in TestIndexWriterMerging.testLucene, but
after reviewing the test I'm of the opinion th
[
http://issues.apache.org/jira/browse/LUCENE-504?page=comments#action_12419624 ]
Doron Cohen commented on LUCENE-504:
I think it makes sense to separate here between efficiency and correctness.
The proposed fix above deals with efficiency, and maybe it
Robert,
Either you or I are missing something basic. I'm not sure which.
As I understand things, an IndexWriter and an IndexReader cannot both
have the write lock at the same time (they use the same write lock file
name). Only an IndexReader can delete and only an IndexWriter can add.
So to up
I think you still need to close the IndexWriter at some point, in
order to search the new documents. In effect all of the changes using
the "buffered" IndexWriter are meaningless until the IndexWriter is
closed and a new IndexReader opened.
Given that, it doesn't make much difference when y
The need to close the IndexWriter is no different with the patch for
deletes than it already is for adds. This is a separate issue that can
be managed asynchronously using the existing mechanism in the
applicaiton. The patch ensures the proper order of operations, so the
benefit remains. Applica
Hello,
I would like to know how to search for phrases that have wild cards in
them. For eg. I want to search for the phrase "Reduces Project Sucess
Risk". I want to add wildcard * to each of these terms, so that I can
search of phrases such as "Reduces Project Sucess Risks" Is this
possible?
Reg
: I would like to know how to search for phrases that have wild cards in
: them. For eg. I want to search for the phrase "Reduces Project Sucess
: Risk". I want to add wildcard * to each of these terms, so that I can
: search of phrases such as "Reduces Project Sucess Risks" Is this
more then li
I thought of a possible enhancement - before I go down the road, I am
looking for some input form the community?
Currently, the QueryFilter caches the bits base upon the IndexReader.
The problem with this is small incremental changes to the index
invalidate the cache.
What if instead the f
28 matches
Mail list logo
