Hi Willem, Thanks a lot for your quick reply!
> How did you get the all this threads blocked? Since there are concurrent requests coming in > For your stack trace , it looks like there are some concurrent > issues in the CacheMap. Yes, I want to check whether someone else also run into the similar issue and whether this is an issue of CXF. > I think you need add some lock in your proxy object creating code, to > create the proxy one by one for each thread. This will have bad impact on performance, I hope that should not be necessary. CXF should allow multi-threaded clients to work well. Will one service object per thread resolve this problem? Thanks again, Jian On Thu, Oct 30, 2008 at 12:07 AM, Willem Jiang <[EMAIL PROTECTED]> wrote: > Hi, > How did you get the all this threads blocked? > Did you try to use one client to send one request to your server? > > For your stack trace , it looks like there are some concurrent issues in > the CacheMap. > > I think you need add some lock in your proxy object creating code, to > create the proxy one by one for each thread. > > Willem > > jian wu wrote: >> Hi, >> >> I'm running a WSDL2JAVA generated CXF WS Client in Tomcat 6.0.10 to >> call external WS >> for the incoming request. >> >> I'm using one Serivce Object to create one new port object for each WS >> Call per thread, >> it seems that, eventually all the http threads are blocked at: >> >> org.apache.cxf.jaxb.JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530) >> >> I'm using CXF 2.1.1, the Tomcat Server is running on Linux with IBM >> 64bit JVM. The >> following is the thread dump snippet: >> >> 3LKMONOBJECT >> org/apache/cxf/common/util/[EMAIL PROTECTED]/00000080D941B848: >> Flat locked by "http-7101-2" (0x000000001129F300), entry count 1 >> 3LKWAITERQ Waiting to enter: >> 3LKWAITER "http-7101-1" (0x0000000010870F00) >> 3LKWAITER "http-7101-3" (0x00000000112A5C00) >> 3LKWAITER "http-7101-4" (0x00000000128C4300) >> 3LKWAITER "http-7101-5" (0x00000000137B9D00) >> 3LKWAITER "http-7101-6" (0x00000000137CAF00) >> 3LKWAITER "http-7101-7" (0x00000000137DF000) >> 3LKWAITER "http-7101-8" (0x00000000137EF200) >> 3LKWAITER "http-7101-9" (0x0000000013800300) >> 3LKWAITER "http-7101-10" (0x000000001335A600) >> (... ...) >> 3LKWAITER "http-7101-60" (0x0000000014797300) >> >> 3XMTHREADINFO "http-7101-2" (TID:0x000000001129F300, >> sys_thread_t:0x0000000010862110, state:CW, native >> ID:0x00000000000057CA) prio=5 >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/TypeInfoSetImpl.getElementInfo(TypeInfoSetImpl.java:275(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:102) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeTypeInfoSetImpl.getElementInfo(RuntimeTypeInfoSetImpl.java:59) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.addGenericElement(ReferencePropertyInfoImpl.java:196) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.calcTypes(ReferencePropertyInfoImpl.java:158(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.getElements(ReferencePropertyInfoImpl.java:112) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.getElements(RuntimeReferencePropertyInfoImpl.java:67) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ReferencePropertyInfoImpl.ref(ReferencePropertyInfoImpl.java:103) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:71) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeReferencePropertyInfoImpl.ref(RuntimeReferencePropertyInfoImpl.java:51) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RegistryInfoImpl.<init>(RegistryInfoImpl.java:99) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.addRegistry(ModelBuilder.java:357(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:218(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/TypeRefImpl.calcRef(TypeRefImpl.java:92) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/TypeRefImpl.getTarget(TypeRefImpl.java:69) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:58) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeTypeRefImpl.getTarget(RuntimeTypeRefImpl.java:51) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:74) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ElementPropertyInfoImpl$1.get(ElementPropertyInfoImpl.java:77) >> 4XESTACKTRACE at >> java/util/AbstractList$Itr.next(AbstractList.java:437(Compiled Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:255(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:98) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getClassInfo(ModelBuilder.java:209) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:93) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/RuntimeModelBuilder.getClassInfo(RuntimeModelBuilder.java:79) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:315(Compiled >> Code)) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/model/impl/ModelBuilder.getTypeInfo(ModelBuilder.java:330) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/runtime/JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:432) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/runtime/JAXBContextImpl.<init>(JAXBContextImpl.java:286) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:139) >> 4XESTACKTRACE at >> com/sun/xml/bind/v2/ContextFactory.createContext(ContextFactory.java:117) >> 4XESTACKTRACE at >> sun/reflect/NativeMethodAccessorImpl.invoke0(Native Method) >> 4XESTACKTRACE at >> sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64(Compiled >> Code)) >> 4XESTACKTRACE at >> sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43(Compiled >> Code)) >> 4XESTACKTRACE at >> java/lang/reflect/Method.invoke(Method.java:615(Compiled Code)) >> 4XESTACKTRACE at >> javax/xml/bind/ContextFinder.newInstance(ContextFinder.java:211) >> 4XESTACKTRACE at >> javax/xml/bind/ContextFinder.find(ContextFinder.java:372) >> 4XESTACKTRACE at >> javax/xml/bind/JAXBContext.newInstance(JAXBContext.java:574) >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:532) >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234) >> 4XESTACKTRACE at javax/xml/ws/Service.getPort(Service.java:40) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91) >> >> 3XMTHREADINFO "http-7101-3" (TID:0x00000000112A5C00, >> sys_thread_t:0x00000000108625E0, state:B, native >> ID:0x00000000000057CB) prio=5 >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530) >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234) >> 4XESTACKTRACE at javax/xml/ws/Service.getPort(Service.java:40) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91) >> >> (... ...) >> >> "http-7101-30" (TID:0x00000000131A6400, >> sys_thread_t:0x000000001442D4A0, state:B, native >> ID:0x0000000000004808) prio=5 >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled >> Code)) >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234) >> 4XESTACKTRACE at javax/xml/ws/Service.getPort(Service.java:40) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91) >> >> (... ...) >> >> "http-7101-60" (TID:0x0000000014797300, >> sys_thread_t:0x0000000014782D40, state:B, native >> ID:0x0000000000004C25) prio=5 >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.createJAXBContext(JAXBDataBinding.java:530(Compiled >> Code)) >> 4XESTACKTRACE at >> org/apache/cxf/jaxb/JAXBDataBinding.initialize(JAXBDataBinding.java:287) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:313) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:394) >> 4XESTACKTRACE at >> org/apache/cxf/service/factory/ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:181) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/support/JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:163) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/AbstractWSDLBasedEndpointFactory.createEndpoint(AbstractWSDLBasedEndpointFactory.java:79) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientFactoryBean.create(ClientFactoryBean.java:51) >> 4XESTACKTRACE at >> org/apache/cxf/frontend/ClientProxyFactoryBean.create(ClientProxyFactoryBean.java:97) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java:93) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.createPort(ServiceImpl.java:369) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:247) >> 4XESTACKTRACE at >> org/apache/cxf/jaxws/ServiceImpl.getPort(ServiceImpl.java:234) >> 4XESTACKTRACE at javax/xml/ws/Service.getPort(Service.java:40) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/adminedqueue/AdminEDQueueService.getAdminEDQueueServiceHttpPort(AdminEDQueueService.java:65) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.getAdminEDQueueServicePort(AdminEDQueueServiceClient.java:87) >> 4XESTACKTRACE at >> com/netflix/watchnow/wsclient/AdminEDQueueServiceClient.prefetchEDQueueForNewBox(AdminEDQueueServiceClient.java:91) >> >> >> Is this a bug in CXF? Or I have to create new Service Object per Thread? >> >> Any suggestion or tip is really appreciated. >> >> Thanks a lot! >> >> Jian >> > >
