KOS wrote:
Hi Freeman
See my configuration below.
The heap size is always about 100 - 150 MB, also when raising errors.
I see, in addition to the stack trace I posted below, this error:
WARNING: EXCEPTION
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
at
org.mortbay.jetty.nio.SelectChannelConnector$1.acceptChannel(SelectChannelConnector.java:75)
at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:475)
at
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:166)
at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:515)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
cxf-bc configuration:
<cxfbc:consumer targetService="ts" targetEndoint="tep" wsdl="localWsdlFile"
/>
<cxfbc:provider service="serviceAsInWsdl" endoint="epAsInWsdl"
wsdl="remoteWsdl" />
I have two service units. So one for consumer, one for provider.
Is there another way to tune the cxf-component? Or raising/lowering
threadpool size to prevent the TOO MANY OPEN FILES error? Because sometimes,
after having these errors, I even have to remove the data folder to be able
to restart SM...
Hi,
You can configure a cxf bus with jetty thread pool using spring file to
control the threads, let's say the file name is jettyThreadPool.xml,
then set the busCfg="jettyThreadPool,xml" for your cxf bc consumer endpoint.
The jettyThreadPool.xml should be
<beans xmlns="http://www.springframework.org/schema/beans";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns:sec="http://cxf.apache.org/configuration/security";
xmlns:http="http://cxf.apache.org/transports/http/configuration";
xmlns:httpj="http://cxf.apache.org/transports/http-jetty/configuration";
xmlns:jaxws="http://java.sun.com/xml/ns/jaxws";
xsi:schemaLocation="
http://cxf.apache.org/configuration/security
http://cxf.apache.org/schemas/configuration/security.xsd
http://cxf.apache.org/transports/http/configuration
http://cxf.apache.org/schemas/configuration/http-conf.xsd
http://cxf.apache.org/transports/http-jetty/configuration
http://cxf.apache.org/schemas/configuration/http-jetty.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd";>
<http:destination
name="{your_port_in_wsdl_namespace}port_name_in_wsdl.http-destination">
</http:destination>
<httpj:engine-factory bus="cxf">
<httpj:engine port="9000(the_port_your_cxf_bc_consumer_listening)">
<httpj:threadingParameters minThreads="1(or what you want)"
maxThreads="3(or what you want)" />
<httpj:connector>
<bean class="org.mortbay.jetty.nio.SelectChannelConnector">
<property name = "port" value="9000" />
<property name="threadPool">
<bean class="org.mortbay.thread.BoundedThreadPool"/>
</property>
</bean>
</httpj:connector>
</httpj:engine>
</httpj:engine-factory>
<!-- We need a bean named "cxf" -->
<bean id="cxf" class="org.apache.cxf.bus.CXFBusImpl"/>
</beans>
Freeman
Thanks a lot
Freeman Fang wrote:
Hi,
When you see the error throw, is the memory reach the heap max size?
Would you please append your configuration for cxf bc consumer/provider
also?
Freeman
KOS wrote:
Hi
(Please see my setting below)
I tried to find out how many messages ServiceMix can handle. After
sending
more than about 1400 requests (in parallel with 100 connections) from an
external soap client to an external Web Service (through the ESB), SM
raises
errors, for about every second request (see stacktrace below). If I send
the
requests from only one client (one connection), it takes longer (about
2000
- 3000 requests) until SM raises errors.
The strange thing is: It makes no difference if I send 1000 requests,
wait a
few seconds, and send another 1000, compared to sending 2000 without
break.
It always fails after about 1400 requests. The only thing that helps is
restarting SM. Then, I can send another 1400 messages without fails. It
seems that SM does not "recover" from the high load.
My question:
1. Is ServiceMix (the cxf-bc) designed to raise errors when too many
requests are sent? Can't they be queued (before cxf-consumer, this one
seems
to raise the error)?
2. How can I tune it (the cxf-bc, whatever would help)?
3. Raising Java VM memory settings helps, SM can then handle more
requests.
But I have still the problem that SM raises a lot of errors when I once
put
too much load. It seems "not to recover"... How can I solve this (without
restarting SM)?
4. Even worse, when I keep sending requests, SM throws this error (see
below: error-message and stacktrace for second error). Then, it closes
the
ports and the service is not available anymore. I have to delete the data
folder and restart SM.
Thanks
My setting:
- ServiceMix 3.3.1 SNAPSHOT
- A WebService-Bridge (cxf-bc consumer and cxf-bc provider). I send
soap-requests from an external client to an external WebService through
the
ESB.
- 2 Intel quad-core @ 2.33 GHz, >> 4 GB RAM
- Linux version 2.6.18
- OpenJDK Server VM build 1.6.0_0-b12
- java -server -Xms128M -Xmx512M (higher settings help, but the problem
of
"not recovering" stays)
Stacktrace for exception:
org.apache.servicemix.cxfbc.interceptors.JbiFault: Fault occured
at
org.apache.servicemix.cxfbc.CxfBcConsumer$JbiPostInvokerInterceptor.handleMessage(CxfBcConsumer.java:653)
at
org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:220)
at
org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:78)
at
org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:278)
at
org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:252)
at
org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:70)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:722)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
at
org.mortbay.jetty.handler.HandlerList.handle(HandlerList.java:49)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:324)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:842)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
at
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
at
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
error-message and stacktrace for second error:
<loc-message>Target component servicemix-cxf-bc for service unit
ontology-change-su is not installed</loc-message>
at
org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:125)
at
org.apache.servicemix.jbi.framework.ManagementSupport.failure(ManagementSupport.java:111)
at
org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.start(ServiceAssemblyLifeCycle.java:146)
at
org.apache.servicemix.jbi.framework.ServiceAssemblyLifeCycle.restore(ServiceAssemblyLifeCycle.java:330)
at
org.apache.servicemix.jbi.framework.DeploymentService.start(DeploymentService.java:100)
at
org.apache.servicemix.jbi.container.JBIContainer.start(JBIContainer.java:670)
at
org.apache.servicemix.jbi.container.SpringJBIContainer.afterPropertiesSet(SpringJBIContainer.java:115)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1368)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1334)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
at java.security.AccessController.doPrivileged(Native Method)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:221)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:429)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:729)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:381)
at
org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:161)
at
org.apache.xbean.spring.context.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:51)
at org.apache.servicemix.Main.main(Main.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.codehaus.classworlds.Launcher.launchStandard(Launcher.java:353)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:264)
at
org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
