John Wang wrote:
In the Lucene code, I don't see where the reader speicified when
creating a field is closed. That holds on to the file.
I am looking at DocumentWriter.invertDocument()
It is closed in a finally clause on line 170, when the TokenStream is
closed.
Doug
I have also seen this problem.
In the Lucene code, I don't see where the reader speicified when
creating a field is closed. That holds on to the file.
I am looking at DocumentWriter.invertDocument()
Thanks
-John
On Mon, 22 Nov 2004 16:21:35 -0600, Chris Lamprecht
[EMAIL PROTECTED] wrote:
A
If you are on linux the number of file handles for a session is much lower than
that for the whole machine. ulimit -n will tell you. There are instructions
on the web for changing this setting, it involves the /etc/security/limits.conf
and setting the values for nofile.
(bulkadm is my user)
I'm sorry, I wasn't involved in the original conversation but maybe I
can jump in with some info that will help.
The number of files depends on the merge factor, number of segments, and
number of indexed fields in your index. It also depends on whether you
are using compound files or not (this
A useful resource for increasing the number of file handles on various
operating systems is the Volano Report:
http://www.volano.com/report/
I had requested help on an issue we have been facing with the Too many
open files Exception garbling the search indexes and crashing the
search on the
-
From: Dmitry Serebrennikov [mailto:[EMAIL PROTECTED]
Sent: Sunday, October 03, 2004 5:08 AM
To: Lucene Users List
Subject: Re: Too many Open Files + lucene 1.4.1 + Linux O/s
Karthik N S wrote:
Hi Luceners,
Apologies.
Other day was Trying to Search using the Luceneweb version
Karthik N S wrote:
Hi Luceners,
Apologies.
Other day was Trying to Search using the Luceneweb version
with Lucene1-4-1.zip and O/s = Linux, J2SDK version 1.4.2_03-b02
With Roughly around 500 Documents (715116 kb ) Indexed using
Lucene1.4-final.jar and
I suspect it has to do with this change:
--- jakarta-lucene/src/java/org/apache/lucene/index/SegmentMerger.java 2004/08/08
13:03:59 1.12
+++ jakarta-lucene/src/java/org/apache/lucene/index/SegmentMerger.java 2004/08/11
17:37:52 1.13
I wouldn't know where to start to reproduce the
Patrick,
For your second problem, are you seeing a behavior
similar to the one discussed in the following thread?
http://www.mail-archive.com/[EMAIL PROTECTED]/msg08952.html
If yes, you can see the solution there.
Thanks,
George
--- Patrick Kates [EMAIL PROTECTED] wrote:
I am having two
Thanks Doug! Reducing the MergeFactor to 10 reduced the number of files
in the index dramatically.
-Reece
-Original Message-
From: Doug Cutting [mailto:[EMAIL PROTECTED]
Sent: Tuesday, October 07, 2003 8:20 PM
To: Lucene Users List
Subject: Re: Too Many Open Files
Wilton, Reece wrote
To: Lucene Users List
Subject: Re: Too Many Open Files
Wilton, Reece wrote:
The index directory that Lucene created has 2,322 files in it. When I
try to open it I get the dreaded Too Many Open Files problem:
java.io.FileNotFoundException: C:\Index\_1lvq.f107 (Too many open
files)
The index
try to check your codes if you are openning a file, close it after using.
--buics
Wilton, Reece [EMAIL PROTECTED] wrote:
Hi,
The index directory that Lucene created has 2,322 files in it. When I
try to open it I get the dreaded Too Many Open Files problem:
java.io.FileNotFoundException:
Wilton, Reece wrote:
The index directory that Lucene created has 2,322 files in it. When I
try to open it I get the dreaded Too Many Open Files problem:
java.io.FileNotFoundException: C:\Index\_1lvq.f107 (Too many open
files)
The index has about 50,000 docs in it. It was created with a merge
-Original Message-
From: Hang Li [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, July 23, 2002 5:22 PM
To: Lucene Users List
Subject: Re: Too many open files?
I did close searcher after each done. Maybe I should try the
CachedSearcher someone posted before ... Is there a final
Thanks for your quick reponse, I still want to know why we ran out of
file descriptors.
--Yup. Cache and reuse your Searcher as much as possible.
--Scott
-Original Message-
From: Hang Li [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, July 23, 2002 9:59 AM
To: Lucene Users List
Li [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, July 23, 2002 10:10 AM
To: Lucene Users List
Subject: Re: Too many open files?
Thanks for your quick reponse, I still want to know why we ran out of
file descriptors.
--Yup. Cache and reuse your Searcher as much as possible.
--Scott
on the OS holding the files handles.
Either way, the only real option is to avoid thrashing the searchers...
Scott
-Original Message-
From: Hang Li [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, July 23, 2002 10:10 AM
To: Lucene Users List
Subject: Re: Too many open files?
Thanks
To: 'Lucene Users List'
Subject: RE: Too many open files?
Are you closing the searcher after each when done?
No: Waiting for the garbage collector is not a good idea.
Yes: It could be a timeout on the OS holding the files handles.
Either way, the only real option is to avoid
just close your Searcher after finishing work
- Original Message -
From: Hang Li [EMAIL PROTECTED]
To: Lucene Users List [EMAIL PROTECTED]
Sent: Tuesday, July 23, 2002 5:59 PM
Subject: Too many open files?
I have seen a lot postings about this topic. Any final thoughts?
We did a
I cached searcher now. As the results, multiple threads try to use the same
searcher. It seems it is much SLOWER than each thread has its own
searcher. Are there any synchronized methods/blocks in Lucene causing this
performance problem?
Scott Ganyo wrote:
Yup. Cache and reuse your Searcher
On Tuesday, 9. April 2002 14:08, you wrote:
root wrote:
Doesn't Lucene releases the filehandles??
because I get too many open files in system after running lucene a
while!
Are you closing the readers and writers after you've finished using
them?
cheers,
Chris
Yes I close the
how many open files you think can be used at your process??
Not sure. It varies with usage pattern. I will check it out in any case.
cat /proc/sys/fs/file-max
cat: /proc/sys/fs/file-max: No such file or directory
echo 5 /proc/sys/fs/file-max
Unfortunately, I cannot use this kind of
root wrote:
Doesn't Lucene releases the filehandles??
because I get too many open files in system after running lucene a while!
Are you closing the readers and writers after you've finished using them?
cheers,
Chris
--
To unsubscribe, e-mail: mailto:[EMAIL PROTECTED]
For additional
it's not a matter of releasing the handles, it needs to keep them open,
this tricked me as well I thought it kept the file handles of the
source XML files open, but if you look at the code it actually reads the
contents of
the files from an HTTP request, the file handles are consumed by the files
On Tuesday, 9. April 2002 14:08, you wrote:
root wrote:
Doesn't Lucene releases the filehandles??
because I get too many open files in system after running lucene a
while!
Are you closing the readers and writers after you've finished using them?
cheers,
Chris
Yes I close the
Subject: Re: too many open files in system
On Tuesday, 9. April 2002 14:08, you wrote:
root wrote:
Doesn't Lucene releases the filehandles??
because I get too many open files in system after running lucene a
while!
Are you closing the readers and writers after you've finished using them
that might be the case I'm indexing 200 000 files each one has about 30 XML
fields each one
has a set of attributes .. could that be it ?
-Original Message-
From: Karl Øie [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 09, 2002 7:03 PM
To: Lucene Users List
Subject: Re: too many open
: Monday, March 18, 2002 10:55 AM
Subject: Re: Too many open files exception...
: Nikhil:
:
: I don't have an answer to your question, but I can tell you that you're
: not likely to get one unless you supply a sample of code that generates
: this error. You haven't really given troubleshooters
28 matches
Mail list logo
