On quick glance it looks to me that the working file repository is trying to save a file but can't because jetty is timing out. Maybe the file is particularly large?
Maybe someone on list can advise on how to change the timeout size for jetty? Chris On Fri, 09 Mar 2012 08:40:52 -0700 Jack Vant <[email protected]> wrote: > We've been running version 1.2 since early January with no problems > until late last week when captures suddenly started failing with this: > > 2012-03-01 13:00:09 ERROR (AbstractFaultChainInitiatorObserver:101) - > Error occurred during error handling, gi > ve up! > > I'll paste the rest of the message below. Just wondering what this > means. I have since stopped matterhorn, cleaned out the cache > directory and restarted the admin and worker servers, so the latest > capture now fails with a text extraction error which is another issue. > Here's the rest of the error message. If some java expert could > 'splain AbstractFaultChainInitiatorObserver:101, I would sure > appreciate it. > > Here's the rest of the error: > > org.apache.cxf.interceptor.Fault: org.mortbay.jetty.EofException: > timeout at > org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:155) > at > org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:121) > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:133) at > org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:82) at > org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) at > org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37) > at > org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:1 > 06) at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:243) > at > org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:110) > at > org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:98) > at > org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:423) > at > org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:139) > at > org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:142) > at > org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179) > at > org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:103) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at > org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159) > at > org.apache.felix.http.base.internal.handler.ServletHandler.doHandle(ServletHandler.java:91) > at > org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:77) > at > org.apache.felix.http.base.internal.dispatch.ServletPipeline.handle(ServletPipeline.java:42) > at > org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.j > ava:49) at > org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:33) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :343) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityIn > terceptor.java:109) at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurity > Interceptor.java:83) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFil > ter.java:97) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.j > ava:100) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAut > henticationFilter.java:78) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter( > RememberMeAuthenticationFilter.java:119) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(Securi > tyContextHolderAwareRequestFilter.java:54) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFil > ter.java:35) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.oauth.provider.OAuthProviderProcessingFilter.doFilter(OAuthProviderPro > cessingFilter.java:173) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.authentication.www.DigestAuthenticationFilter.doFilter(DigestAuthe > nticationFilter.java:319) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(Defaul > tLoginPageGeneratingFilter.java:90) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(Abs > tractAuthenticationProcessingFilter.java:188) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextP > ersistenceFilter.java:79) at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java > :355) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:149) > at > org.opencastproject.kernel.security.SecurityFilter.doFilter(SecurityFilter.java:115) > at > org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:80) > at > org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:68) > at > org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.j > ava:47) at > org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:33) > at > org.opencastproject.kernel.security.OrganizationFilter.doFilter(OrganizationFilter.java:94) > at > org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:80) > at > org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:68) > at > org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.j > ava:47) at > org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:33) > at > org.opencastproject.kernel.rest.JsonpFilter.doFilter(JsonpFilter.java:106) > at > org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:80) > at > org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:68) > at > org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.j > ava:47) at > org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:33) > at > org.apache.felix.http.base.internal.dispatch.FilterPipeline.dispatch(FilterPipeline.java:48) > at > org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:39) > at > org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:55) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > at org.mortbay.jetty.Server.handle(Server.java:326) at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:536) > at > org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:930) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:834) at > org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at > org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405) at > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) > at > org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) > Caused by: java.lang.RuntimeException: > org.mortbay.jetty.EofException: timeout at > org.opencastproject.workingfilerepository.impl.WorkingFileRepositoryImpl.putInCollection(WorkingFil > eRepositoryImpl.java:444) at > org.opencastproject.workingfilerepository.impl.WorkingFileRepositoryRestEndpoint.restPutInCollectio > n(WorkingFileRepositoryRestEndpoint.java:141) at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) at > org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173) > at > org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89) > ... > 84 more Caused by: org.mortbay.jetty.EofException: timeout at > org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:1164) > at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:1114) at > org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:97 > 7) at > org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:887) > at java.io.InputStream.read(InputStream.java:85) at > org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1025) at > org.apache.commons.io.IOUtils.copy(IOUtils.java:999) at > org.opencastproject.workingfilerepository.impl.WorkingFileRepositoryImpl.putInCollection(WorkingFil > eRepositoryImpl.java:442) ... 91 more 2012-03-01 13:00:24 INFO > (SchedulerServiceSolrIndex:681) - Solr query: dc_spatial:mpcf201 > 2012-03-01 13:00:40 INFO (SchedulerServiceSolrIndex:681) - Solr > query: dc_spatial:mpcf106 2012-03-01 13:00:48 ERROR > (WorkflowOperationWorker:156) - Workflow operation 'prepare-av' > failed org.opencastproject.workflow.api.WorkflowOperationException: > Muxing video track > http://feather1.boisestate.edu > :8080/files/mediapackage/5458/track-2/windows.mpg > and audio track http://feather1.boisestate.edu:8080/files/me > diapackage/5458/track-3/audio.mp2 failed at > org.opencastproject.workflow.handler.PrepareAVWorkflowOperationHandler.mux(PrepareAVWorkflowOperati > onHandler.java:263) at > org.opencastproject.workflow.handler.PrepareAVWorkflowOperationHandler.start(PrepareAVWorkflowOpera > tionHandler.java:130) at > org.opencastproject.workflow.impl.WorkflowOperationWorker.start(WorkflowOperationWorker.java:216) > at > org.opencastproject.workflow.impl.WorkflowOperationWorker.execute(WorkflowOperationWorker.java:138) > at > org.opencastproject.workflow.impl.WorkflowServiceImpl.runWorkflowOperation(WorkflowServiceImpl.java > :704) > at > org.opencastproject.workflow.impl.WorkflowServiceImpl.process(WorkflowServiceImpl.java:1412) > at > org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:1717) > at > org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:1690) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) 2012-03-01 13:00:53 INFO > (ZipWorkflowOperationHandler:158) - Archiving mediapackage > 8c477517-df4b-4f8f-a9a5-3 53963bf4a8d in workflow Workflow {5458} > > > -- Christopher Brooks, BSc, MSc ARIES Laboratory, University of Saskatchewan Web: http://www.cs.usask.ca/~cab938 Phone: 1.306.966.1442 Mail: Advanced Research in Intelligent Educational Systems Laboratory Department of Computer Science University of Saskatchewan 176 Thorvaldson Building 110 Science Place Saskatoon, SK S7N 5C9 _______________________________________________ Matterhorn-users mailing list [email protected] http://lists.opencastproject.org/mailman/listinfo/matterhorn-users
