Follow Up to the article from Friday
-Original Message-
From: Morus Walter [mailto:[EMAIL PROTECTED]
Sent: Monday, February 28, 2005 1:30 AM
To: Lucene Users List
Subject: Re: Search performance with one index vs. many indexes
Jochen Franke writes:
Topic: Search performance with large
Hi All,
Sorry about that please disregard that last email. I must not be fully
awake yet.
Sorry,
Kevin Runde
-Original Message-
From: Runde, Kevin [mailto:[EMAIL PROTECTED]
Sent: Monday, February 28, 2005 7:34 AM
To: Lucene Users List
Subject: RE: Search performance with one index vs
Jochen Franke writes:
Topic: Search performance with large numbers of indexes vs. one large index
My questions are:
- Is the size of the wordlist the problem?
- Would we be a lot faster, when we have a smaller number
of files per index?
sure.
Look:
Index lookup of a word is O(ln(n
Topic: Search performance with large numbers of indexes vs. one large index
Hello,
we are experiencing a performance problem when using large
numbers of indexes.
We have an application with about
6 Mio. Documents
one index of about 7 GB
probably 10 to 15 million different words in that
index
.
Best,
Sergiu
Michael
-Original Message-
From: David Townsend [mailto:[EMAIL PROTECTED]
Sent: Friday, February 18, 2005 11:50 AM
To: Lucene Users List
Subject: RE: Search Performance
IndexSearchers are thread safe, so you can use the same object on multiple
requests. If the index
What is single handedly the best way to improve search performance? I have
an index in the 2G range stored on the local file system of the searcher.
Under a load test of 5 simultaneous users my average search time is ~4700
ms. Under a load test of 10 simultaneous users my average search time
Performance
What is single handedly the best way to improve search performance? I have
an index in the 2G range stored on the local file system of the searcher.
Under a load test of 5 simultaneous users my average search time is ~4700
ms. Under a load test of 10 simultaneous users my average search time
I am creating new IndexSearchers... how do I cache my IndexSearcher...
Michael
-Original Message-
From: David Townsend [mailto:[EMAIL PROTECTED]
Sent: Friday, February 18, 2005 11:00 AM
To: Lucene Users List
Subject: RE: Search Performance
Are you creating new IndexSearchers
: RE: Search Performance
Are you creating new IndexSearchers or IndexReaders on each search? Caching
your IndexSearchers has a dramatic effect on speed.
David Townsend
-Original Message-
From: Michael Celona [mailto:[EMAIL PROTECTED]
Sent: 18 February 2005 15:55
To: Lucene Users List
February 2005 16:15
To: Lucene Users List
Subject: Re: Search Performance
Try a singleton pattern or an static field.
Stefan
Michael Celona wrote:
I am creating new IndexSearchers... how do I cache my IndexSearcher...
Michael
-Original Message-
From: David Townsend [mailto:[EMAIL PROTECTED
My index is changing in real time constantly... in this case I guess this
will not work for me any suggestions...
Michael
-Original Message-
From: David Townsend [mailto:[EMAIL PROTECTED]
Sent: Friday, February 18, 2005 11:50 AM
To: Lucene Users List
Subject: RE: Search Performance
]
MC Sent: Friday, February 18, 2005 11:50 AM
MC To: Lucene Users List
MC Subject: RE: Search Performance
MC IndexSearchers are thread safe, so you can use the same object on multiple
MC requests. If the index is static and not constantly updating, just keep one
MC IndexSearcher for the life
:
public static long gc()
{
long bef = mem();
System.gc();
sleep( 100);
System.runFinalization();
sleep( 100);
System.gc();
long aft= mem();
return aft-bef;
}
Michael Celona wrote:
What is single handedly the best way to improve search
://www.opensymphony.com/oscache/
Michael Celona wrote:
What is single handedly the best way to improve search performance? I have
an index in the 2G range stored on the local file system of the searcher.
Under a load test of 5 simultaneous users my average search time is ~4700
ms. Under a load
I am using the highlighter... does this matter
-Original Message-
From: David Spencer [mailto:[EMAIL PROTECTED]
Sent: Friday, February 18, 2005 2:05 PM
To: Lucene Users List
Subject: Re: Search Performance
Are you using the highlighter or doing anything non-trivial in
displaying
I should have mentioned, the reason for not doing this the obvious,
simple way (just close the Searcher and reopen it if a new version is
available) is because some threads could be in the middle of iterating
through the search Hits. If you close the Searcher they get a Bad
file descriptor
Thanks... I am seeing this problem right now Has anyone implemented a
better solution...?
Michael
-Original Message-
From: Chris Lamprecht [mailto:[EMAIL PROTECTED]
Sent: Friday, February 18, 2005 4:14 PM
To: Lucene Users List
Subject: Re: Search Performance
I should have mentioned
Or you could just open a new IndexSearcher, forget the old one, and
have GC collect it when everyone is done with it.
Otis
--- Chris Lamprecht [EMAIL PROTECTED] wrote:
I should have mentioned, the reason for not doing this the obvious,
simple way (just close the Searcher and reopen it if a
Just tried that... works like a charm... thanks...
Michael
-Original Message-
From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]
Sent: Friday, February 18, 2005 4:42 PM
To: Lucene Users List; Chris Lamprecht
Subject: Re: Search Performance
Or you could just open a new IndexSearcher
Users List; Chris Lamprecht
Subject: Re: Search Performance
Or you could just open a new IndexSearcher, forget the old one, and
have GC collect it when everyone is done with it.
Otis
--- Chris Lamprecht [EMAIL PROTECTED] wrote:
I should have mentioned, the reason for not doing this the obvious
Wouldn't this leave open file handles? I had a problem where there
were lots of open file handles for deleted index files, because the
old searchers were not being closed.
On Fri, 18 Feb 2005 13:41:37 -0800 (PST), Otis Gospodnetic
[EMAIL PROTECTED] wrote:
Or you could just open a new
Yes, until it's cleaned up, and as soon as the last client is done with
Hits, the originating IndexSearcher is ready for cleanup if nobody else
is holding references to it. You can close it explicityly, as you are
doing, too, no harm.
Otis
--- Chris Lamprecht [EMAIL PROTECTED] wrote:
Wouldn't
22 matches
Mail list logo
