Hello,
in a Carbon instance which still runs Carbon 4.0.5 and Axis2
1.6.1-wso2v7, we perform simplistic load tests by repeatedly calling a
SOAP service method which accesses a small number of methods of a data
service, and returns a response which contains parts of the
DataService's return values.
The SOAP service is secured with UsernameToken security - this might be
important as the problem seems to occur only when the service is secured
and not when calling the same services without security.
After a load test which creates 1000 requests from 10 different threads,
we cannot login to the management console any more. The start of the
exception stack trace is (full stack at end of message):
TID: [0] [AS] [2014-01-15 11:23:19,835] ERROR
{org.wso2.carbon.utils.UserUtils} - Failed pasring config file
/opt/prelado/core/repository/conf/user-mgt.xml. {org.wso2.carbon.uit
ls.UserUtils}
java.lang.NullPointerException
at
org.apache.axiom.om.impl.dom.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(DocumentBuilderFactoryImpl.java:93)
at
org.wso2.carbon.utils.UserUtils.hasMultipleUserStores(UserUtils.java:24)
at
org.wso2.carbon.ui.CarbonUILoginUtil.handleLogin(CarbonUILoginUtil.java:336)
This happens 100% of the time, so I looked into the code. The offending
code section is:
return isDOOMRequired()
? new DocumentBuilderImpl(this)
: originalDocumentBuilderFactory.newDocumentBuilder();
Searching brought me to this issue:
https://issues.apache.org/jira/browse/AXIS2-1570
which explains nicely what the concurrency problems are. Following that,
rampart seems to have been fixed to not call
DocumentBuilderFactoryImpl.setDOOMRequired.
But my exception indicates that somebody is indeed calling
DocumentBuilderFactoryImpl.setDOOMRequired (or some other method which
messes with the system property), as I start the JVM with the following
property definition in wso2server.sh:
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl
\
Does all of this ring a bell with someone?
Is there something we can do short-term to solve this problem?
Thanks a lot
Kambiz
Kambiz Darabi
--
m-creations gmbh
Acker 2
55116 Mainz
Germany
W: http://www.m-creations.com
E: [email protected]
T: +49 6131 6224417
F: +49 6131 6224418
--
Registered Office: Mainz, HRB Mainz 7382
Managing Directors: Frank Pacholak, Kambiz Darabi
Full exception stack:
TID: [0] [AS] [2014-01-15 11:23:19,835] ERROR
{org.wso2.carbon.utils.UserUtils} - Failed pasring config file
/opt/prelado/core/repository/conf/user-mgt.xml. {org.wso2.carbon.uti
ls.UserUtils}
java.lang.NullPointerException
at
org.apache.axiom.om.impl.dom.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(DocumentBuilderFactoryImpl.java:93)
at
org.wso2.carbon.utils.UserUtils.hasMultipleUserStores(UserUtils.java:24)
at
org.wso2.carbon.ui.CarbonUILoginUtil.handleLogin(CarbonUILoginUtil.java:336)
at
org.wso2.carbon.ui.CarbonSecuredHttpContext.handleSecurity(CarbonSecuredHttpContext.java:238)
at
org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:86)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:67)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at
org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at
org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:172)
at
org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
at
org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
