Jitendra, It's hard to identify why is your process taking such a longtime to respond. Your logs are not indicating the real problem. It will be helpful if you can share the bpel of your process and if you can't, make a simulated replica of your process.
Can you share the server.xml from tomcat and ode-axis2.properties file. You may be interested in looking into my response on this thread in configuring ode and tomcat. http://markmail.org/message/sfbb6otrbp5prcce?q=Error+when+connecting+ODE+with+bitronix regards, sathwik On Tue, Mar 25, 2014 at 12:24 PM, Kharche, Jitendra < [email protected]> wrote: > Please help. I am getting this error everyday. > > Regards, > Jitendra > > -----Original Message----- > From: Kharche, Jitendra [mailto:[email protected]] > Sent: Wednesday, March 19, 2014 3:31 PM > To: [email protected] > Subject: RE: SocketTimeoutException in ODE > > Hi Sathwik, > > Thanks. > But I cannot find parameter *ode-axi2.threads.pool.size* in > ode-axis2.properties file (attached). Do we need to explicitly configure it? > This file is default except the database (External - PostgreSql) and > transaction factory as GeronimoFactory. > > >>Probable causes could be your database is slow on committing > >>transactions, > The database is dedicated for ODE. I do not think this could be a problem. > > >> or ODE is being loaded with requests and ODE scheduler is taking time > >> to process them > The load on ODE is very minimal. Only one process runs at a time. All the > 11 processes are scheduled at an interval of 5 min, 4 times a day. And each > process takes around 2-3 mins for completion. So from processing > perspective there is no load on ODE. > > >> You might have to play around this parameter by increasing it to higher > values if jobs are piling up on the ODE scheduler. > With this much of minimal load, I don't think the parameter > *ode-axi2.threads.pool.size*setting will make any difference. > > Regards, > Jitendra > > -----Original Message----- > From: Sathwik B P [mailto:[email protected]] > Sent: Wednesday, March 19, 2014 3:11 PM > To: [email protected] > Subject: Re: SocketTimeoutException in ODE > > Hi Jitendra, > > Your facade process is taking more than 2mins to respond. For the log > snippet it's not clear where is it taking more time. > > I can only speculate, > Probable causes could be your database is slow on committing transactions, > or ODE is being loaded with requests and ODE scheduler is taking time to > process them. > > What is the value of the parameter *ode-axi2.threads.pool.size* in your > ode-axis2.properties file? > You might have to play around this parameter by increasing it to higher > values if jobs are piling up on the ODE scheduler. > > regards, > sathwik > > > On Tue, Mar 18, 2014 at 4:15 PM, Kharche, Jitendra < > [email protected]> wrote: > > > Hi Sathwik, > > > > Sorry if am not clearly able to explain. To simplify, I have 2 > > processes - 1. A Façade Process, 2. My Actual/Target Process. > > > > Actual Process has following broad steps: > > 1. Receive start request > > 2. Reply to caller to indicate process started successfully 3. Invoke > > an application adapter - async call 4. Receive feedback from > > application adapter 5. further steps > > > > Façade Process has following steps: > > 1. Receive start request > > 2. Invoke Actual Process > > 3. Reply to caller > > > > > > I am pasting the exception I see in ode logs when I see the > > SocketTimoutException on client. > > > > 2014-03-12 01:37:00,711 ERROR [ODEService] Timeout or execution error > > when waiting for response to MEX {MyRoleMex#hqejbhcnphr942iow8mfqj > > [Client hqejbhcnphr942iow8mfqi] calling > > {http://a/b/processes/ProcessesFacade}ProcessesFacadeService.exportFee > > dback(...)} > > java.util.concurrent.TimeoutException: Message exchange > > > org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture@75f385cetimedout(120000 > ms) when waiting for a response! > > java.util.concurrent.TimeoutException: Message exchange > > > org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture@75f385cetimedout(120000 > ms) when waiting for a response! > > at > > > org.apache.ode.bpel.engine.MyRoleMessageExchangeImpl$ResponseFuture.get(MyRoleMessageExchangeImpl.java:245) > > at > > > org.apache.ode.axis2.ODEService.onAxisMessageExchange(ODEService.java:162) > > at > > > org.apache.ode.axis2.hooks.ODEMessageReceiver.invokeBusinessLogic(ODEMessageReceiver.java:69) > > at > > > org.apache.ode.axis2.hooks.ODEMessageReceiver.invokeBusinessLogic(ODEMessageReceiver.java:52) > > at > > > org.apache.axis2.receivers.AbstractMessageReceiver.receive(AbstractMessageReceiver.java:114) > > at > org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:173) > > at > > > org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:173) > > at > > org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:144) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:647) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:728) > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305) > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > > at > > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > > at > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) > > at > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) > > at > > > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:611) > > at > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > > at > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) > > at > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) > > at > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > > at > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) > > at > > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041) > > at > > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603) > > at > > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:724) > > 2014-03-12 01:37:00,789 ERROR [INVOKE] Failure during invoke: No > > response received for invoke (mexId=hqejbhcnphr942iow8mfqn), forcing > > it into a failed state. > > 2014-03-12 01:37:00,789 INFO [BpelRuntimeContextImpl] ActivityRecovery: > > Registering activity 9, failure reason: No response received for > > invoke (mexId=hqejbhcnphr942iow8mfqn), forcing it into a failed state. > > on channel > > 18 > > > > Regards, > > Jitendra > > > > -----Original Message----- > > From: Sathwik B P [mailto:[email protected]] > > Sent: Tuesday, March 18, 2014 3:40 PM > > To: [email protected] > > Subject: Re: SocketTimeoutException in ODE > > > > Can you attach the ODE logs. > > > > I don't think the process model was clear to me after your latest mail. > > You said that there 2 recevie (bpel activites) in each process and > > then in the latest mail you specify that there is receive-reply (bpel > activities). > > > > > > On Tue, Mar 18, 2014 at 2:14 AM, Kharche, Jitendra < > > [email protected]> wrote: > > > > > Hi Sathwik, > > > > > > Thanks for quick reply. > > > > > > The façade process is request-response. When it receives a request, > > > it invokes the target process. Each target process replies as soon > > > as it receives a request (before starting its work) - so there is no > > > blocking in the target process for the façade process. As soon as > > > the façade receives the reply from the target process it replies > > > back to the client - no extra processing in the façade process. > > > > > > >> client side raises the exception since it has not recived the > > > >> response > > > within the stiputaled time. > > > This is obvious. However I am looking for the reasons for this. As I > > > mentioned this happens rarely - not always and also randomly for any > > > target process - not for any specific target process. > > > > > > Hope I am clear. > > > > > > Regards, > > > Jitendra > > > > > > -----Original Message----- > > > From: Sathwik B P [mailto:[email protected]] > > > Sent: Tuesday, March 18, 2014 2:28 PM > > > To: [email protected] > > > Subject: Re: SocketTimeoutException in ODE > > > > > > Hi Jitendra, > > > > > > Is your facade process exposed as request-response or one-way mep? > > > > > > In case of request-response mep and if the facade process is a time > > > consuming process, then you are bound to see read time out exceptions. > > > > > > Process execution within ODE is asyncronous, and hence the process > > > continues to completion. But your transport channel on the client > > > side raises the exception since it has not recived the response > > > within the stiputaled time. > > > > > > regards, > > > sathwik > > > > > > > > > On Tue, Mar 18, 2014 at 1:27 AM, Kharche, Jitendra < > > > [email protected]> wrote: > > > > > > > Hi, > > > > > > > > I am using ODE 1.3.6 on Tomcat 7.0.47 with 11 processes deployed > > > > into > > it. > > > > On top of these I have a process that acts as a façade so that > > > > clients have a single interface to call. The interface of the > > > > façade process is a consolidation of interfaces provided by all > processes. > > > > This façade process then invokes the actual process. > > > > > > > > Each process is a long running using explicit correlation. > > > > Each process has 2 receive activities - first receive starts the > > > > process and second receive after receiving a completion reply from > > > > the application adapter. > > > > There is a scheduler to execute processes in scheduled manner. The > > > > scheduler invokes the façade process which then invokes the target > > > process. > > > > > > > > Randomly (once in 2-3 days for any random process), I get > > > > SocketTimeoutException in client (the scheduler) as follows during > > > > invocation of the process. > > > > Sometimes it is also observed that, even though the client got the > > > > exception the process started and completed successfully. > > > > > > > > What could be the cause? > > > > > > > > Thanks in advance. > > > > > > > > Exception in client: > > > > > > > > > > > > javax.xml.ws.WebServiceException: Could not send Message. > > > > > > > > at > > > > org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java > > > > :1 > > > > 45 > > > > ) > > > > > > > > at com.sun.proxy.$Proxy104.startEmpSync3D(Unknown > > > > Source) > > > > > > > > at > > > > com.geo.ml.scheduler.jobs.EmpSync3DJob.executeProcess(EmpSync3DJob > > > > .j > > > > av > > > > a:115) > > > > > > > > at > > > > com.geo.ml.scheduler.jobs.EmpSync3DJob.execute(EmpSync3DJob.java:4 > > > > 9) > > > > > > > > at > > > > org.quartz.core.JobRunShell.run(JobRunShell.java:213) > > > > > > > > at > > > > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool. > > > > ja > > > > va:557) > > > > > > > > Caused by: java.net.SocketTimeoutException: SocketTimeoutException > > > > invoking [Façade Process URL]: Read timed out > > > > > > > > at > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > > > Method) > > > > > > > > at > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstr > > > > uc > > > > to > > > > rAccessorImpl.java:57) > > > > > > > > at > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Delegati > > > > ng > > > > Co > > > > nstructorAccessorImpl.java:45) > > > > > > > > at > > > > java.lang.reflect.Constructor.newInstance(Constructor.java:526) > > > > > > > > at > > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapE > > > > xc > > > > ep > > > > tion(HTTPConduit.java:1467) > > > > > > > > at > > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.clos > > > > e( > > > > HT > > > > TPConduit.java:1452) > > > > > > > > at > > > > org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java: > > > > 56 > > > > ) > > > > > > > > at > > > > org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:6 > > > > 59 > > > > ) > > > > > > > > at > > > > org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderE > > > > nd > > > > in > > > > gInterceptor.handleMessage(MessageSenderInterceptor.java:62) > > > > > > > > at > > > > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterc > > > > ep > > > > to > > > > rChain.java:262) > > > > > > > > at > > > > org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530) > > > > > > > > at > > > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:463) > > > > > > > > at > > > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:366) > > > > > > > > at > > > > org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:319) > > > > > > > > at > > > > org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96 > > > > ) > > > > > > > > at > > > > org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java > > > > :1 > > > > 34 > > > > ) > > > > > > > > ... 5 more > > > > > > > > Caused by: java.net.SocketTimeoutException: Read timed out > > > > > > > > at java.net.SocketInputStream.socketRead0(Native > > > > Method) > > > > > > > > at > > > > java.net.SocketInputStream.read(SocketInputStream.java:152) > > > > > > > > at > > > > java.net.SocketInputStream.read(SocketInputStream.java:122) > > > > > > > > at > > > > sun.security.ssl.InputRecord.readFully(InputRecord.java:442) > > > > > > > > at > > > > sun.security.ssl.InputRecord.read(InputRecord.java:480) > > > > > > > > at > > > > sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) > > > > > > > > at > > > > sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:8 > > > > 84 > > > > ) > > > > > > > > at > > > > sun.security.ssl.AppInputStream.read(AppInputStream.java:102) > > > > > > > > at > > > > java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > > > > > > > > at > > > > java.io.BufferedInputStream.read1(BufferedInputStream.java:275) > > > > > > > > at > > > > java.io.BufferedInputStream.read(BufferedInputStream.java:334) > > > > > > > > at > > > > sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687) > > > > > > > > at > > > > sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) > > > > > > > > at > > > > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURL > > > > Co > > > > nn > > > > ection.java:1323) > > > > > > > > at > > > > java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java: > > > > 46 > > > > 8) > > > > > > > > at > > > > sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode( > > > > Ht > > > > tp > > > > sURLConnectionImpl.java:338) > > > > > > > > at > > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.hand > > > > le > > > > Re > > > > sponseInternal(HTTPConduit.java:1604) > > > > > > > > at > > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.hand > > > > le > > > > Re > > > > sponse(HTTPConduit.java:1530) > > > > > > > > at > > > > org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.clos > > > > e( > > > > HT > > > > TPConduit.java:1438) > > > > > > > > ... 15 more > > > > > > > > > > > > Regards, > > > > Jitendra > > > > > > > > > > > > > >
