> Were you able to re-create the problem without passing -1 to accept? No I was not able to make accept fail without passing -1. Added a printf there to print any failure in accept and it didn't. However, ran into the segfault issue when accept actually failed, which I had to do manually.
I did test with 1000 back-2-back mtom requests in 5 terminals, and also 1000 repeated calls to an echo client that sends 1000 back-2-back requests, and in 4 terminals. None of these cases failed. There were occasional hangs but still it didn't result in any failure on either client or server side. However, I didn't have any way to test multiple requests made to a single port from multiple machines. That may be a different scenario. Regards, Senaka > > Samisa... > > Senaka Fernando wrote: >>> Senaka Fernando wrote: >>> >>>> Hi Samisa, >>>> >>>> I fixed two bugs regarding stream handling inside >>>> simple_http_svr_conn.c. >>>> This fixes an issue of this nature. >>>> >>>> Say, we have two server threads that try to serve requests offered to >>>> the >>>> axis2 server. Now, according to what is intended, one must serve and >>>> the >>>> other must wait until it can serve. The previous implementation caused >>>> this to fail with a segfault. But, with the fix, it will rather not >>>> crash >>>> but, loop until it gets a chance to serve the request. >>>> >>>> >>> Did you test before the fix and did it seg fault for you? Can you >>> please >>> explain the scenario under which it seg faulted? If it is a problem, I >>> too should be able to re-create it. >>> >>> And how did you solve the problem? By adding a sleep? >>> >> >> I made the accept(2) call fail. By passing a -1 to it. Once that failed, >> there were two places that segfaulted which I already fixed, few minutes >> ago. No, I didn't add a sleep(). I simply prevented the segfault by >> adding >> necessary is_null checks before calling methods. Then, the server just >> kept on spawning threads and freeing them as it can't serve the request. >> This is the expected behaviour in http_svr_thread.c I believe. >> >> In a real world scenario, a situation similar to a -1 passed to >> accept(2) >> will be temporal. So a respawned thread down the line should server the >> request without any problem. >> >> Passing -1 to accept was done by, >> >> Index: util/src/network_handler.c >> =================================================================== >> --- util/src/network_handler.c (revision 634578) >> +++ util/src/network_handler.c (working copy) >> @@ -221,7 +221,7 @@ >> AXIS2_ENV_CHECK(env, AXIS2_CRITICAL_FAILURE); >> >> cli_len = sizeof(cli_addr); >> - cli_socket = accept(svr_socket, (struct sockaddr *) &cli_addr, >> &cli_len); >> + cli_socket = accept(-1, (struct sockaddr *) &cli_addr, &cli_len); >> if (cli_socket < 0) >> AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, >> "[Axis2][network_handler] Socket accept \ >> --- >> >> I'm not sure whether this is what really lead to the core dump, or was >> it >> something else. >> >> This was all that I could try on based on the less informative >> valgrind.log that was sent. :(... >> >> Also i did test 1000 back-to-back requests and they all were served >> without any issues. >> >> Regards, >> Senaka >> >> >>> Samisa... >>> >>> >>>> Could this solve the core dump issue? >>>> >>>> Regards, >>>> Senaka >>>> >>>> >>>> >>>>> Raghavendra SM wrote: >>>>> >>>>> >>>>>> As expected, when I put a delay of about 50 milliseconds between the >>>>>> requests there is no dump. >>>>>> >>>>>> But we would still like to know the following, >>>>>> + Isn't Axis capable of handling such huge bombardment of requests >>>>>> (of >>>>>> the order of 1000's) one after another without any delay between the >>>>>> requests? Has anyone tried such client before? >>>>>> >>>>>> >>>>>> >>>>> Yes it is supposed to. We have done performance testing that proved >>>>> that >>>>> Axis2/C can serve more than 300 rps. >>>>> >>>>> >>>>> >>>>>> + Can we make out anything out of the valgrind report or the axis >>>>>> logs >>>>>> that were attached? >>>>>> >>>>>> >>>>>> >>>>> Unfortunately not. When valgrind runs, it makes the system slow, that >>>>> prevents the system crashing in situations like that you are >>>>> experiencing. To solve this kind of a problem, it is a must that you >>>>> provide us with some sample code that can be used to reproduce the >>>>> problem. Without having a way to re-produce the situation, there is >>>>> no >>>>> way to fix this problem - it could well be a problem in Axis2/C as >>>>> well >>>>> as it could also be in the code that you use. >>>>> >>>>> Samisa... >>>>> >>>>> >>>>> >>>>> >>>>>> Regards, >>>>>> ~raghav >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>> From: Senaka Fernando [mailto:[EMAIL PROTECTED] >>>>>> Sent: Thursday, March 06, 2008 4:22 PM >>>>>> To: Apache AXIS C User List >>>>>> Subject: RE: Core dumps in Axis libraries >>>>>> >>>>>> Hi Raghav, >>>>>> >>>>>> Can you please try this with --enable-guthilla=yes, and send the >>>>>> dump.log, >>>>>> and also the appropriate log_files generated by the server/client. >>>>>> >>>>>> There also can be issues with your "condition variable" being used >>>>>> in >>>>>> multiple threads. >>>>>> >>>>>> It also seems that if you pause between your requests you might get >>>>>> this >>>>>> solved. According to what you say, when you increase the log_level, >>>>>> obviously you waste some time there which might be the solution >>>>>> rather. >>>>>> You can use the AXIS2_SLEEP() and AXIS2_USLEEP() inside Axis2/C to >>>>>> pause >>>>>> between requests. You can get an idea on how to use these functions >>>>>> by >>>>>> taking a look at the Axis2/C user_guide/client samples. >>>>>> >>>>>> Regards, >>>>>> Senaka >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> As suggested, I tried using axis2c-1.3.0 both with and without >>>>>>> --enable-guthilla=yes. But I still see different dumps >>>>>>> consistently. >>>>>>> Please find the attached file with back-traces. >>>>>>> >>>>>>> Are you using the same service client instance from multiple >>>>>>> threads? >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>> We have a axis thread pool with default number of threads and a >>>>>>>>>> set >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>> of our own threads(about 10, lets call them "database threads"). >>>>>>> + On receiving a HTTP soap request, axis thread invokes the >>>>>>> >>>>>>> >>>>>>> >>>>>> appropriate >>>>>> >>>>>> >>>>>> >>>>>>> function that is bound to our logic. >>>>>>> + Now, the axis thread requests a "database thread" to perform the >>>>>>> appropriate query and waits on a condition variable. >>>>>>> + our "db thread" performs the necessary query and signals on that >>>>>>> condition variable so that axis thread shall take the control over. >>>>>>> + axis thread will create the om output and dispatch as usual. >>>>>>> >>>>>>> My doubts: >>>>>>> + does this design has any apparent drawback when there is a flurry >>>>>>> of >>>>>>> back-to-back soap requests? >>>>>>> + Is there a need to increase the axis thread pool size? If yes, >>>>>>> how >>>>>>> >>>>>>> >>>>>>> >>>>>> to >>>>>> >>>>>> >>>>>> >>>>>>> do it? >>>>>>> + As told before, the dump is seen during a particular scenario >>>>>>> only. >>>>>>> i.e: while trying to add a list of already existing users to my >>>>>>> database. For each user there is a SOAP request generated and the >>>>>>> requests are sequential, i.e the second request is sent only after >>>>>>> the >>>>>>> response for previous request is received. >>>>>>> + if I increase the log level of my own module that uses axis2c, >>>>>>> there >>>>>>> is no dump. On increasing the log level, my module writes quite a >>>>>>> bit >>>>>>> >>>>>>> >>>>>>> >>>>>> of >>>>>> >>>>>> >>>>>> >>>>>>> information to my log file. >>>>>>> >>>>>>> Your help is appreciated. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> ~raghav >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Samisa Abeysinghe [mailto:[EMAIL PROTECTED] >>>>>>> Sent: Wednesday, March 05, 2008 7:25 PM >>>>>>> To: Apache AXIS C User List >>>>>>> Subject: Re: Core dumps in Axis libraries >>>>>>> >>>>>>> Raghavendra SM wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Thanks Manjula, >>>>>>>> >>>>>>>> But thats not a viable option for us. We would still want to >>>>>>>> continue >>>>>>>> using axis2C-1.0 for some more time. >>>>>>>> >>>>>>>> Are these dumps well known? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> No. >>>>>>> >>>>>>> Are you using the same service client instance from multiple >>>>>>> threads? >>>>>>> >>>>>>> Samisa... >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>>> >>>>>>> >>>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> -- >>>>> Samisa Abeysinghe >>>>> Software Architect; WSO2 Inc. >>>>> >>>>> http://www.wso2.com/ - "Oxygenating the Web Service Platform." >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>>> >>>>> >>>>> >>>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>>> For additional commands, e-mail: [EMAIL PROTECTED] >>>> >>>> >>>> >>>> >>>> >>> -- >>> Samisa Abeysinghe >>> Software Architect; WSO2 Inc. >>> >>> http://www.wso2.com/ - "Oxygenating the Web Service Platform." >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>> For additional commands, e-mail: [EMAIL PROTECTED] >>> >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >> >> > > > -- > Samisa Abeysinghe > Software Architect; WSO2 Inc. > > http://www.wso2.com/ - "Oxygenating the Web Service Platform." > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
