I'm posting this as follow up information to my
initial question posted yesterday. I have pasted that
original email below.
Apparently the indexer is crashing because of "too
many open files". I have posted the entry from the
logs for this below. So my question becomes, is this a
bug in the indexer or is this (the number of allowed
open files) something I have to configure at the OS or
JVM level? I should add that I get several of these
errors in the logs before the indexer crashes.
Any Ideas?
Here's the entry in the logs:
2002-08-27 11:33:56 DEBUG (2002-08-27) 11:33.56:184
[sitemap.] (/amber-dan/samples/search/create)
HttpProcessor[8180][4]/ServerPagesGenerator:
ProcessingException in generate()
org.apache.cocoon.ProcessingException: IOException
in index(): java.io.FileNotFoundException:
/var/cache/tomcat4/localhost/amber-dan/cocoon-files/index/_24.f116
(Too many open files)
at
org.apache.cocoon.components.search.SimpleLuceneCocoonIndexerImpl.index(SimpleLuceneCocoonIndexerImpl.java:261)
at
org.apache.cocoon.www.samples.search.create_index_xsp.createIndex(/var/cache/tomcat4/localhost/amber-dan/cocoon-files/org/apache/cocoon/www/samples/search/create_index_xsp.java:100)
at
org.apache.cocoon.www.samples.search.create_index_xsp.generate(/var/cache/tomcat4/localhost/amber-dan/cocoon-files/org/apache/cocoon/www/samples/search/create_index_xsp.java:184)
at
org.apache.cocoon.generation.ServerPagesGenerator.generate(ServerPagesGenerator.java:263)
at
org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.processXMLPipeline(AbstractProcessingPipeline.java:495)
----original email---
Hello,
I have something like this for my xml documents:
docs/
dir1/
dir2/
dir3/
dir4/
dir5/
I can index each individual dir in docs separately and
a lucene index will be built properly.
However, when I try to index all together (either by
aggregation, or by crawling from docs/, it get the
following exception at random times. It never fails on
the same file:
2002-08-26 16:04:35
StandardWrapper[/amber-dan.bak:default]: Loading
container servlet default
2002-08-26 16:04:35 default: init
2002-08-26 16:04:35
StandardWrapper[/amber-dan.bak:invoker]: Loading
container servlet invoker
2002-08-26 16:04:35 invoker: init
2002-08-26 16:04:35 jsp: init
2002-08-26 16:06:10 StandardWrapperValve[Cocoon2]:
Servlet.service() for servlet Cocoon2 threw exception
java.lang.IllegalStateException
at
org.apache.catalina.connector.ResponseFacade.reset(ResponseFacade.java:243)
at
org.apache.cocoon.servlet.CocoonServlet.service(CocoonServlet.java:1071)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:243)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:190)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2343)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:170)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:170)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:468)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
at
org.apache.catalina.connector.http.HttpProcessor.process(HttpProcessor.java:1012)
at
org.apache.catalina.connector.http.HttpProcessor.run(HttpProcessor.java:1107)
at java.lang.Thread.run(Thread.java:479)
I'm pretty sure my technique for doing the indexing is
fine. Everythign worked fine before when I had fewer
files in each of the directories (about 5 each). I
started getting the above error when I added more
files to a couple of the directories (now i have about
25 in at least two of the children of dir/. I don't
know if this problem stems from the fact that there
are more files and cocoon is timing out or something.
What does an IllegalStateException mean in general? Is
there a decent way to run the supplied cocoon indexer
on these files from the command line to This is
really puzzling me... each dir will index fine on its
own. The problems appear when I try to index them all.
(I need one big index).
Thanks for any suggestions on how I can track this
problem down.
__________________________________________________
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes
http://finance.yahoo.com
---------------------------------------------------------------------
Please check that your question has not already been answered in the
FAQ before posting. <http://xml.apache.org/cocoon/faq/index.html>
To unsubscribe, e-mail: <[EMAIL PROTECTED]>
For additional commands, e-mail: <[EMAIL PROTECTED]>
