Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
jean-frederic clere wrote: Bojan Smojver wrote: Bojan Smojver wrote: Unfortunately, the problem is still there... Let me run gdb on the thing again and then I'll send you the backtrace. Bojan Slighty different problem this time, but along the lines of the previous one: DARN, That is the same kind of error, caused by the line: JkMount /login/j_security_check ajp13 The code only supports /context/* - I will try to fix this one - Just downloaded and complied the latest CVS snapshot of mod_jk (it still has version 1.2.0 on it). I tried to replicate the problem and it seems to be gone (hurray!!!). Do I take it this issue has been addressed? Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
jean-frederic clere wrote: Bojan Smojver wrote: Bojan Smojver wrote: Unfortunately, the problem is still there... Let me run gdb on the thing again and then I'll send you the backtrace. Bojan Slighty different problem this time, but along the lines of the previous one: DARN, That is the same kind of error, caused by the line: JkMount /login/j_security_check ajp13 The code only supports /context/* - I will try to fix this one - Was the above problem in any way related to this post? Re: Patch: Mod_jk remove query string as factor in MATCH_TYPE_EXACT And just one other detail that bothers me... How come the problem only occurs when I gracefully restart Apache twice or more in a row. Wouldn't this kind error occur the first or second time around (ie. if the mount isn't supported, it would create problems all the time). Did I miss something here? Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Bojan Smojver wrote: Just a little trivial problem with the new source: it doesn't build because file common/jk_uri_worker_map.c has an unintentional comment end (the * after apache) in line 312: * I have fixed jk_mount_context() in apache*/mod_jk.c so we should The comment actually ends in line 314. Oops, sorry I added the comment after testing - My bad ! - Bojan Bojan Smojver wrote: jean-frederic clere wrote: Well I have fixed the code to prevent using the faulting JkMount syntax. I am happy my stupidity is good for something. At least the code became more idiot-proof ;-) Thanks heaps, Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Bojan Smojver wrote: Bojan Smojver wrote: Unfortunately, the problem is still there... Let me run gdb on the thing again and then I'll send you the backtrace. Bojan Slighty different problem this time, but along the lines of the previous one: DARN, That is the same kind of error, caused by the line: JkMount /login/j_security_check ajp13 The code only supports /context/* - I will try to fix this one - -- Program received signal SIGSEGV, Segmentation fault. 0x4021d26f in strncmp (s1=0x4f444145 Address 0x4f444145 out of bounds, s2=0x82d2c9c /, n=5852238) at ../sysdeps/generic/strncmp.c:42 42 ../sysdeps/generic/strncmp.c: No such file or directory. (gdb) where #0 0x4021d26f in strncmp (s1=0x4f444145 Address 0x4f444145 out of bounds, s2=0x82d2c9c /, n=5852238) at ../sysdeps/generic/strncmp.c:42 #1 0x8076ed1 in map_uri_to_worker (uw_map=0x82ef390, uri=0x82d2c9c /, l=0x82ef200) at jk_uri_worker_map.c:423 #2 0x8072a5c in jk_translate (r=0x82d155c) at mod_jk.c:1179 #3 0x8146b41 in run_method (r=0x82d155c, offset=0, run_all=0) at http_config.c:369 #4 0x8146b8a in ap_translate_name (r=0x82d155c) at http_config.c:381 #5 0x815535c in process_request_internal (r=0x82d155c) at http_request.c:1198 #6 0x815572a in ap_process_request (r=0x82d155c) at http_request.c:1323 #7 0x814f378 in child_main (child_num_arg=0) at http_main.c:4299 #8 0x814f593 in make_child (s=0x827c3e4, slot=0, now=996898126) at http_main.c:4467 #9 0x814f608 in startup_children (number_to_start=1) at http_main.c:4494 #10 0x814fc79 in standalone_main (argc=1, argv=0xb884) at http_main.c:4854 #11 0x81501e0 in main (argc=1, argv=0xb884) at http_main.c:5124 #12 0x401b9b5c in __libc_start_main (main=0x814fed4 main, argc=1, ubp_av=0xb884, init=0x806f500 _init, fini=0x81b7d10 _fini, rtld_fini=0x4000d634 _dl_fini, stack_end=0xb87c) at ../sysdeps/generic/libc-start.c:129 -- Hope this helps. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
jean-frederic clere wrote: Bojan Smojver wrote: jean-frederic clere wrote: Bojan Smojver wrote: Don't want to be a pain in the back side... but (there is always at least one of those :-), is anyone checking this one out? I had a peek into the code and that NULL pointer is a variable called 'uwr-context'. Don't know much about mod_jk, but this one gets initialised in the same file, function uri_worker_map_add. If that thing gets blown away (or isn't redone) when Apache gracefully restarts... I will have a look... It seems uwr-context is not initialized... Do you have something like: +++ JkMount toto titi +++ In your httpd.conf (uri not starting with /)? I think this leaves uwr wrongly initialized... Thanks. Bojan As the matter of fact I do. Here is the config: ### # Apache JK Configuration File# ### IfModule mod_jk.c JkWorkersFile /etc/tomcat/workers.properties JkLogFile /var/apache/logs/mod_jk.log JkLogLevelinfo JkMount *.jsp ajp13 JkMount /*.jsp ajp13 JkMount *.vmajp13 JkMount /*.vm ajp13 JkMount /servlet/* ajp13 JkMount /login/j_security_check ajp13 /IfModule Does that mean that entries like *.jsp and *.vm would segfault Apache? Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Bojan Smojver wrote: jean-frederic clere wrote: Bojan Smojver wrote: Don't want to be a pain in the back side... but (there is always at least one of those :-), is anyone checking this one out? I had a peek into the code and that NULL pointer is a variable called 'uwr-context'. Don't know much about mod_jk, but this one gets initialised in the same file, function uri_worker_map_add. If that thing gets blown away (or isn't redone) when Apache gracefully restarts... I will have a look... It seems uwr-context is not initialized... Do you have something like: +++ JkMount toto titi +++ In your httpd.conf (uri not starting with /)? I think this leaves uwr wrongly initialized... Thanks. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Bojan Smojver wrote: jean-frederic clere wrote: Bojan Smojver wrote: jean-frederic clere wrote: Bojan Smojver wrote: Don't want to be a pain in the back side... but (there is always at least one of those :-), is anyone checking this one out? I had a peek into the code and that NULL pointer is a variable called 'uwr-context'. Don't know much about mod_jk, but this one gets initialised in the same file, function uri_worker_map_add. If that thing gets blown away (or isn't redone) when Apache gracefully restarts... I will have a look... It seems uwr-context is not initialized... Do you have something like: +++ JkMount toto titi +++ In your httpd.conf (uri not starting with /)? I think this leaves uwr wrongly initialized... Thanks. Bojan As the matter of fact I do. Here is the config: ### # Apache JK Configuration File# ### IfModule mod_jk.c JkWorkersFile /etc/tomcat/workers.properties JkLogFile /var/apache/logs/mod_jk.log JkLogLevelinfo JkMount *.jsp ajp13 JkMount /*.jsp ajp13 JkMount *.vmajp13 JkMount /*.vm ajp13 JkMount /servlet/* ajp13 JkMount /login/j_security_check ajp13 /IfModule Does that mean that entries like *.jsp and *.vm would segfault Apache? Yep, they could: I have the following: +++ IfModule mod_jk.c JkWorkersFile /usr/local/apache/conf/workers.properties JkLogFile logs/mod_jk.log JkLogLeveldebug JkMount toto titi JkMount /examples/* examples JkMount /servlet-tests/* servlet-tests JkMount /jsp-tests/* jsp-tests #JkMount toto titi /IfModule +++ And I have added a trace in map_uri_to_worker that shows: +++ jk_uri_worker_map_t::map_uri_to_worker, 0 jk_uri_worker_map_t::map_uri_to_worker,080d08b8 10 jk_uri_worker_map_t::map_uri_to_worker,080d0900 15 jk_uri_worker_map_t::map_uri_to_worker,080d0958 11 +++ I will try to fix the problem. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Just a little trivial problem with the new source: it doesn't build because file common/jk_uri_worker_map.c has an unintentional comment end (the * after apache) in line 312: * I have fixed jk_mount_context() in apache*/mod_jk.c so we should The comment actually ends in line 314. Bojan Bojan Smojver wrote: jean-frederic clere wrote: Well I have fixed the code to prevent using the faulting JkMount syntax. I am happy my stupidity is good for something. At least the code became more idiot-proof ;-) Thanks heaps, Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Unfortunately, the problem is still there... Let me run gdb on the thing again and then I'll send you the backtrace. Bojan jean-frederic clere wrote: Clere Jean-Frederic FSC EP LP COM 5 wrote: Bojan Smojver wrote: jean-frederic clere wrote: Bojan Smojver wrote: jean-frederic clere wrote: Bojan Smojver wrote: Don't want to be a pain in the back side... but (there is always at least one of those :-), is anyone checking this one out? I had a peek into the code and that NULL pointer is a variable called 'uwr-context'. Don't know much about mod_jk, but this one gets initialised in the same file, function uri_worker_map_add. If that thing gets blown away (or isn't redone) when Apache gracefully restarts... I will have a look... It seems uwr-context is not initialized... Do you have something like: +++ JkMount toto titi +++ In your httpd.conf (uri not starting with /)? I think this leaves uwr wrongly initialized... Thanks. Bojan As the matter of fact I do. Here is the config: ### # Apache JK Configuration File# ### IfModule mod_jk.c JkWorkersFile /etc/tomcat/workers.properties JkLogFile /var/apache/logs/mod_jk.log JkLogLevelinfo JkMount *.jsp ajp13 JkMount /*.jsp ajp13 JkMount *.vmajp13 JkMount /*.vm ajp13 JkMount /servlet/* ajp13 JkMount /login/j_security_check ajp13 /IfModule Does that mean that entries like *.jsp and *.vm would segfault Apache? Yep, they could: I have the following: +++ IfModule mod_jk.c JkWorkersFile /usr/local/apache/conf/workers.properties JkLogFile logs/mod_jk.log JkLogLeveldebug JkMount toto titi JkMount /examples/* examples JkMount /servlet-tests/* servlet-tests JkMount /jsp-tests/* jsp-tests #JkMount toto titi /IfModule +++ And I have added a trace in map_uri_to_worker that shows: +++ jk_uri_worker_map_t::map_uri_to_worker, 0 jk_uri_worker_map_t::map_uri_to_worker,080d08b8 10 jk_uri_worker_map_t::map_uri_to_worker,080d0900 15 jk_uri_worker_map_t::map_uri_to_worker,080d0958 11 +++ I will try to fix the problem. Well I have fixed the code to prevent using the faulting JkMount syntax. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Bojan Smojver wrote: Unfortunately, the problem is still there... Let me run gdb on the thing again and then I'll send you the backtrace. Bojan Slighty different problem this time, but along the lines of the previous one: -- Program received signal SIGSEGV, Segmentation fault. 0x4021d26f in strncmp (s1=0x4f444145 Address 0x4f444145 out of bounds, s2=0x82d2c9c /, n=5852238) at ../sysdeps/generic/strncmp.c:42 42 ../sysdeps/generic/strncmp.c: No such file or directory. (gdb) where #0 0x4021d26f in strncmp (s1=0x4f444145 Address 0x4f444145 out of bounds, s2=0x82d2c9c /, n=5852238) at ../sysdeps/generic/strncmp.c:42 #1 0x8076ed1 in map_uri_to_worker (uw_map=0x82ef390, uri=0x82d2c9c /, l=0x82ef200) at jk_uri_worker_map.c:423 #2 0x8072a5c in jk_translate (r=0x82d155c) at mod_jk.c:1179 #3 0x8146b41 in run_method (r=0x82d155c, offset=0, run_all=0) at http_config.c:369 #4 0x8146b8a in ap_translate_name (r=0x82d155c) at http_config.c:381 #5 0x815535c in process_request_internal (r=0x82d155c) at http_request.c:1198 #6 0x815572a in ap_process_request (r=0x82d155c) at http_request.c:1323 #7 0x814f378 in child_main (child_num_arg=0) at http_main.c:4299 #8 0x814f593 in make_child (s=0x827c3e4, slot=0, now=996898126) at http_main.c:4467 #9 0x814f608 in startup_children (number_to_start=1) at http_main.c:4494 #10 0x814fc79 in standalone_main (argc=1, argv=0xb884) at http_main.c:4854 #11 0x81501e0 in main (argc=1, argv=0xb884) at http_main.c:5124 #12 0x401b9b5c in __libc_start_main (main=0x814fed4 main, argc=1, ubp_av=0xb884, init=0x806f500 _init, fini=0x81b7d10 _fini, rtld_fini=0x4000d634 _dl_fini, stack_end=0xb87c) at ../sysdeps/generic/libc-start.c:129 -- Hope this helps. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Don't want to be a pain in the back side... but (there is always at least one of those :-), is anyone checking this one out? I had a peek into the code and that NULL pointer is a variable called 'uwr-context'. Don't know much about mod_jk, but this one gets initialised in the same file, function uri_worker_map_add. If that thing gets blown away (or isn't redone) when Apache gracefully restarts... Bojan Bojan Smojver wrote: GOMEZ Henri wrote: Just realised that the core I have was compiled without -g option. VERY useful. I also wanted to replicate the core dump with the httpd that was compiled with -g, but that didn't work. Now I'm starting to think that the core file I had was a result of a different crash (I was experimenting with a lot of stuff and Apache would sometimes spit the dummy). So, how do I get to you something you can use? I've seen the Apache debugging document (http://dev.apache.org/debugging.html) and forceful core dumping techniques, but I couldn't find truss for Linux. Or maybe I should use something else? What are your suggestions? Ask to Apache specialist in new-http list, may be JF Clere could forward your question to them ? Here is the backtrace from gdb when the segfault happens (I'm starting to enjoy this debugging of Apache :-) - Program received signal SIGSEGV, Segmentation fault. 0x4021d26f in strncmp (s1=0x0, s2=0x82d2bb4 /whatwedo/, n=137294736) at ../sysdeps/generic/strncmp.c:42 42 ../sysdeps/generic/strncmp.c: No such file or directory. (gdb) where #0 0x4021d26f in strncmp (s1=0x0, s2=0x82d2bb4 /whatwedo/, n=137294736) at ../sysdeps/generic/strncmp.c:42 #1 0x8076e91 in map_uri_to_worker (uw_map=0x82ef290, uri=0x82d2bb4 /whatwedo/, l=0x82ef100) at jk_uri_worker_map.c:412 #2 0x8072a4c in jk_translate (r=0x82d145c) at mod_jk.c:1177 #3 0x8146a99 in run_method (r=0x82d145c, offset=0, run_all=0) at http_config.c:369 #4 0x8146ae2 in ap_translate_name (r=0x82d145c) at http_config.c:381 #5 0x81552b4 in process_request_internal (r=0x82d145c) at http_request.c:1198 #6 0x8155682 in ap_process_request (r=0x82d145c) at http_request.c:1323 #7 0x814f2d0 in child_main (child_num_arg=0) at http_main.c:4299 #8 0x814f4eb in make_child (s=0x827c2e4, slot=0, now=996554230) at http_main.c:4467 #9 0x814f560 in startup_children (number_to_start=1) at http_main.c:4494 #10 0x814fbd1 in standalone_main (argc=1, argv=0xb854) at http_main.c:4854 #11 0x8150138 in main (argc=1, argv=0xb854) at http_main.c:5124 #12 0x401b9b5c in __libc_start_main (main=0x814fe2c main, argc=1, ubp_av=0xb854, init=0x806f500 _init, fini=0x81b7c70 _fini, rtld_fini=0x4000d634 _dl_fini, stack_end=0xb84c) at ../sysdeps/generic/libc-start.c:129 - Looks like strncmp received a NULL pointer at #0... Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Just realised that the core I have was compiled without -g option. VERY useful. I also wanted to replicate the core dump with the httpd that was compiled with -g, but that didn't work. Now I'm starting to think that the core file I had was a result of a different crash (I was experimenting with a lot of stuff and Apache would sometimes spit the dummy). So, how do I get to you something you can use? I've seen the Apache debugging document (http://dev.apache.org/debugging.html) and forceful core dumping techniques, but I couldn't find truss for Linux. Or maybe I should use something else? What are your suggestions? Ask to Apache specialist in new-http list, may be JF Clere could forward your question to them ? OK, after talking to some Apache people I have the core and the strace to go with it (kind of a package deal ;-) Since the core is rather big, I'll tar/bzip it and send it to your e-mail address rather then the mailing list (don't want to go on the hate list just yet). Is that OK with you? Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Just realised that the core I have was compiled without -g option. VERY useful. I also wanted to replicate the core dump with the httpd that was compiled with -g, but that didn't work. Now I'm starting to think that the core file I had was a result of a different crash (I was experimenting with a lot of stuff and Apache would sometimes spit the dummy). So, how do I get to you something you can use? I've seen the Apache debugging document (http://dev.apache.org/debugging.html) and forceful core dumping techniques, but I couldn't find truss for Linux. Or maybe I should use something else? What are your suggestions? Ask to Apache specialist in new-http list, may be JF Clere could forward your question to them ? Here is the backtrace from gdb when the segfault happens (I'm starting to enjoy this debugging of Apache :-) - Program received signal SIGSEGV, Segmentation fault. 0x4021d26f in strncmp (s1=0x0, s2=0x82d2bb4 /whatwedo/, n=137294736) at ../sysdeps/generic/strncmp.c:42 42 ../sysdeps/generic/strncmp.c: No such file or directory. (gdb) where #0 0x4021d26f in strncmp (s1=0x0, s2=0x82d2bb4 /whatwedo/, n=137294736) at ../sysdeps/generic/strncmp.c:42 #1 0x8076e91 in map_uri_to_worker (uw_map=0x82ef290, uri=0x82d2bb4 /whatwedo/, l=0x82ef100) at jk_uri_worker_map.c:412 #2 0x8072a4c in jk_translate (r=0x82d145c) at mod_jk.c:1177 #3 0x8146a99 in run_method (r=0x82d145c, offset=0, run_all=0) at http_config.c:369 #4 0x8146ae2 in ap_translate_name (r=0x82d145c) at http_config.c:381 #5 0x81552b4 in process_request_internal (r=0x82d145c) at http_request.c:1198 #6 0x8155682 in ap_process_request (r=0x82d145c) at http_request.c:1323 #7 0x814f2d0 in child_main (child_num_arg=0) at http_main.c:4299 #8 0x814f4eb in make_child (s=0x827c2e4, slot=0, now=996554230) at http_main.c:4467 #9 0x814f560 in startup_children (number_to_start=1) at http_main.c:4494 #10 0x814fbd1 in standalone_main (argc=1, argv=0xb854) at http_main.c:4854 #11 0x8150138 in main (argc=1, argv=0xb854) at http_main.c:5124 #12 0x401b9b5c in __libc_start_main (main=0x814fe2c main, argc=1, ubp_av=0xb854, init=0x806f500 _init, fini=0x81b7c70 _fini, rtld_fini=0x4000d634 _dl_fini, stack_end=0xb84c) at ../sysdeps/generic/libc-start.c:129 - Looks like strncmp received a NULL pointer at #0... Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Just realised that the core I have was compiled without -g option. VERY useful. I also wanted to replicate the core dump with the httpd that was compiled with -g, but that didn't work. What did not work? -It does not core any more? Now I'm starting to think that the core file I had was a result of a different crash (I was experimenting with a lot of stuff and Apache would sometimes spit the dummy). So, how do I get to you something you can use? I've seen the Apache debugging document (http://dev.apache.org/debugging.html) and forceful core dumping techniques, but I couldn't find truss for Linux. Or maybe I should use something else? The strace you have used is truss for Linux What are your suggestions? Try to get a core (otherwise it is hard to help). Remove dynamic linked modules until it stops coring. BTW I do not have the httpd.conf file, could send it to me? Ask to Apache specialist in new-http list, may be JF Clere could forward your question to them ? Not yet ;-)
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Ask to Apache specialist in new-http list, may be JF Clere could forward your question to them ? I'll subscribe to the list and try to find out how it's done. Then (hopefully) I'll be able to send over something you can work with. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
jean-frederic clere wrote: GOMEZ Henri wrote: Just realised that the core I have was compiled without -g option. VERY useful. I also wanted to replicate the core dump with the httpd that was compiled with -g, but that didn't work. What did not work? -It does not core any more? The core dump was the result of some other Apache run (probably when I attempted to use kgcc rather then gcc to compile), not problems with mod_jk. So, that wouldn't help much anyway. It doesn't core dump. There is just segmentation fault of the child processes after second and thereafter graceful restart. Now I'm starting to think that the core file I had was a result of a different crash (I was experimenting with a lot of stuff and Apache would sometimes spit the dummy). So, how do I get to you something you can use? I've seen the Apache debugging document (http://dev.apache.org/debugging.html) and forceful core dumping techniques, but I couldn't find truss for Linux. Or maybe I should use something else? The strace you have used is truss for Linux Thanks. I saw the undocumented -S option for truss explained on Apache debugging page, which 'freezes' the core so it can be forcefully dumped. That's what I was after. What are your suggestions? Try to get a core (otherwise it is hard to help). I'm sending an e-mail over to new-httpd. Maybe they can help with this. Remove dynamic linked modules until it stops coring. This is statically linked Apache. BTW I do not have the httpd.conf file, could send it to me? I've attached it. Don't worry about the SSL section as it's wrong (ie. one configuration for all virtual hosts). I'm not using SSL to segfault the server (at least it works fine with the TC 3.3 B1 mod_jk :-) Bojan apache-conf.tar.bz2
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Good information, and could send us a strace ? This may sound strange to all you Apache experts, but how do I do that? How do I run strace on a detached process? strace httpd -f (follow forks) Thanks for the tip. The log file (bziped) is attached. Bojan strace.log.bz2
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Bojan Smojver wrote:
GOMEZ Henri wrote:
Good information, and could send us a strace ?
This may sound strange to all you Apache experts, but how do I do that?
How do I run strace on a detached process?
strace httpd -f (follow forks)
Thanks for the tip. The log file (bziped) is attached.
Bojan
Name: strace.log.bz2
strace.log.bz2Type: unspecified type (application/octet-stream)
Encoding: base64
+++
15329 ... rt_sigaction resumed {0x814df6c, [], SA_INTERRUPT|0x400}, 8) =
015329 getsockname(8, {sin_family=AF_INET, sin_port=htons(80),
sin_addr=inet_addr(10.0.0.20)}}, [16]) = 0
15329 setsockopt(8, IPPROTO_TCP1, [1], 4) = 0
15329 brk(0x8347000)= 0x8347000
15329 brk(0x834a000)= 0x834a000
15329 read(8, GET /about.html HTTP/1.0\r\nIf-Mod..., 4096) = 436
15329 rt_sigaction(SIGUSR1, {SIG_IGN}, {SIG_IGN}, 8) = 0
15329 time(NULL)= 995959824
15329 gettimeofday({995959824, 346428}, NULL) = 0
15329 --- SIGSEGV (Segmentation fault) ---
15329 chdir(/etc/httpd) = 0
15329 rt_sigaction(SIGSEGV, {SIG_DFL}, {SIG_DFL}, 8) = 0
15329 getpid() = 15329
15329 kill(15329, SIGSEGV) = 0
15329 sigreturn() = ? (mask now [])
15329 --- SIGSEGV (Segmentation fault) ---
+++
That the core?
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
jean-frederic clere wrote:
Bojan Smojver wrote:
GOMEZ Henri wrote:
Good information, and could send us a strace ?
This may sound strange to all you Apache experts, but how do I do that?
How do I run strace on a detached process?
strace httpd -f (follow forks)
Thanks for the tip. The log file (bziped) is attached.
Bojan
Name: strace.log.bz2
strace.log.bz2Type: unspecified type (application/octet-stream)
Encoding: base64
+++
15329 ... rt_sigaction resumed {0x814df6c, [], SA_INTERRUPT|0x400}, 8) =
015329 getsockname(8, {sin_family=AF_INET, sin_port=htons(80),
sin_addr=inet_addr(10.0.0.20)}}, [16]) = 0
15329 setsockopt(8, IPPROTO_TCP1, [1], 4) = 0
15329 brk(0x8347000)= 0x8347000
15329 brk(0x834a000)= 0x834a000
15329 read(8, GET /about.html HTTP/1.0\r\nIf-Mod..., 4096) = 436
15329 rt_sigaction(SIGUSR1, {SIG_IGN}, {SIG_IGN}, 8) = 0
15329 time(NULL)= 995959824
15329 gettimeofday({995959824, 346428}, NULL) = 0
15329 --- SIGSEGV (Segmentation fault) ---
15329 chdir(/etc/httpd) = 0
15329 rt_sigaction(SIGSEGV, {SIG_DFL}, {SIG_DFL}, 8) = 0
15329 getpid() = 15329
15329 kill(15329, SIGSEGV) = 0
15329 sigreturn() = ? (mask now [])
15329 --- SIGSEGV (Segmentation fault) ---
+++
That the core?
Don't quite follow (my C debugging skills are so rusty you can call them
non-existent)...
Bojan
RE: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Good information, and could send us a strace ? This may sound strange to all you Apache experts, but how do I do that? How do I run strace on a detached process? strace httpd -f (follow forks) Thanks for the tip. The log file (bziped) is attached. I didn't locate the core !
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Good information, and could send us a strace ? This may sound strange to all you Apache experts, but how do I do that? How do I run strace on a detached process? strace httpd -f (follow forks) Thanks for the tip. The log file (bziped) is attached. I didn't locate the core ! Just realised that the core I have was compiled without -g option. VERY useful. I also wanted to replicate the core dump with the httpd that was compiled with -g, but that didn't work. Now I'm starting to think that the core file I had was a result of a different crash (I was experimenting with a lot of stuff and Apache would sometimes spit the dummy). So, how do I get to you something you can use? I've seen the Apache debugging document (http://dev.apache.org/debugging.html) and forceful core dumping techniques, but I couldn't find truss for Linux. Or maybe I should use something else? What are your suggestions? Bojan
RE: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Just realised that the core I have was compiled without -g option. VERY useful. I also wanted to replicate the core dump with the httpd that was compiled with -g, but that didn't work. Now I'm starting to think that the core file I had was a result of a different crash (I was experimenting with a lot of stuff and Apache would sometimes spit the dummy). So, how do I get to you something you can use? I've seen the Apache debugging document (http://dev.apache.org/debugging.html) and forceful core dumping techniques, but I couldn't find truss for Linux. Or maybe I should use something else? What are your suggestions? Ask to Apache specialist in new-http list, may be JF Clere could forward your question to them ?
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Now I have some new info here. After recompiling everything clean, I've taken out PHP altogether. So it's not part of the equation. It is sufficient to compile mod_jk in and use static pages only (ie. no JSP's/VM at all). If you gracefully restart the server a number of times (the number varies), it will eventually segfault. Good information, and could send us a strace ? Just occurred to me that I have the dumped core of Apache when it died. Would that help you? Bojan
RE: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Good information, and could send us a strace ? Just occurred to me that I have the dumped core of Apache when it died. Would that help you? May be send a gdb where output ?)
RE: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Good information, and could send us a strace ? This may sound strange to all you Apache experts, but how do I do that? How do I run strace on a detached process? strace httpd -f (follow forks)
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: I still have to distribute this DSO version of Apache and mod_jk to some slower machines to see if that makes any difference at all. The machine that didn't produce any errors is a 1 GHz Athlon, so timing issues could have been resolved by brute force, who knows... Thanks to tell us what happened... Just installed the DSO version on a Pentium 166. Well, it segfaults when I load PHP pages on this machine but it doesn't on normal pages after a graceful restart. Which leads me to believe... that I haven't a clue why is all this happening! Back to square one (ie. make distclean etc.) Shall keep you posted. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Just did a DSO version and couldn't replicate the problem (BTW, I've recompiled/statically linked Apache and mod_jk again since then and the problem was still there). So, maybe it has to do with statically linking Apache after all... Strange problem. What's the difference between static and dynamic. We may have missed something at mod_jk init time ? Now I have some new info here. After recompiling everything clean, I've taken out PHP altogether. So it's not part of the equation. It is sufficient to compile mod_jk in and use static pages only (ie. no JSP's/VM at all). If you gracefully restart the server a number of times (the number varies), it will eventually segfault. The only absolutely safe solution is to do stop and then start. Then it works OK. So, I guess looking into the init code would be the proper thing to do. If you give me a few pointers, I can help. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Bojan Smojver wrote: As for DSO, I'll have to work on that, so probably some time tomorrow (Sydney time). Just did a DSO version and couldn't replicate the problem (BTW, I've recompiled/statically linked Apache and mod_jk again since then and the problem was still there). So, maybe it has to do with statically linking Apache after all... I still have to distribute this DSO version of Apache and mod_jk to some slower machines to see if that makes any difference at all. The machine that didn't produce any errors is a 1 GHz Athlon, so timing issues could have been resolved by brute force, who knows... Is there a recommendation as to the order of loading mod_jk for both statically linked version and the DSO version? Maybe that's important. And just one other observation, I'm also using php4. When statically linked Apache doesn't contain php4 the problem is more difficult to replicate (ie. it only occurs occasionally) which sort of pointed me towards timing issues. But then again, I really don't understand the internals of Apache to say what could be causing this. All suggestions, fixes, resolutions etc. welcome ;-) Bojan
RE: Problem with mod_jk 1.2.0 (latest CVS snapshot)
Just did a DSO version and couldn't replicate the problem (BTW, I've recompiled/statically linked Apache and mod_jk again since then and the problem was still there). So, maybe it has to do with statically linking Apache after all... Strange problem. What's the difference between static and dynamic. We may have missed something at mod_jk init time ? I still have to distribute this DSO version of Apache and mod_jk to some slower machines to see if that makes any difference at all. The machine that didn't produce any errors is a 1 GHz Athlon, so timing issues could have been resolved by brute force, who knows... Thanks to tell us what happened... Is there a recommendation as to the order of loading mod_jk for both statically linked version and the DSO version? Maybe that's important. I load mod_jk after alias and before rewrite and never get any problems. Not I also use php4 LoadModule mmap_static_module lib/apache/mod_mmap_static.so LoadModule vhost_alias_module lib/apache/mod_vhost_alias.so LoadModule env_module lib/apache/mod_env.so LoadModule define_module lib/apache/mod_define.so LoadModule config_log_module lib/apache/mod_log_config.so LoadModule agent_log_module lib/apache/mod_log_agent.so LoadModule referer_log_module lib/apache/mod_log_referer.so LoadModule mime_magic_module lib/apache/mod_mime_magic.so LoadModule mime_modulelib/apache/mod_mime.so LoadModule negotiation_module lib/apache/mod_negotiation.so LoadModule status_module lib/apache/mod_status.so LoadModule info_modulelib/apache/mod_info.so LoadModule includes_modulelib/apache/mod_include.so LoadModule autoindex_module lib/apache/mod_autoindex.so LoadModule dir_module lib/apache/mod_dir.so LoadModule cgi_module lib/apache/mod_cgi.so LoadModule asis_modulelib/apache/mod_asis.so LoadModule imap_modulelib/apache/mod_imap.so LoadModule action_module lib/apache/mod_actions.so LoadModule speling_module lib/apache/mod_speling.so LoadModule userdir_module lib/apache/mod_userdir.so LoadModule alias_module lib/apache/mod_alias.so LoadModule jk_module lib/apache/mod_jk.so LoadModule auth_ldap_module lib/apache/auth_ldap.so LoadModule rewrite_module lib/apache/mod_rewrite.so LoadModule access_module lib/apache/mod_access.so LoadModule auth_modulelib/apache/mod_auth.so LoadModule anon_auth_module lib/apache/mod_auth_anon.so LoadModule dbm_auth_modulelib/apache/mod_auth_dbm.so LoadModule db_auth_module lib/apache/mod_auth_db.so LoadModule digest_module lib/apache/mod_digest.so LoadModule proxy_module lib/apache/libproxy.so LoadModule cern_meta_module lib/apache/mod_cern_meta.so LoadModule expires_module lib/apache/mod_expires.so LoadModule headers_module lib/apache/mod_headers.so LoadModule usertrack_module lib/apache/mod_usertrack.so LoadModule example_module lib/apache/mod_example.so LoadModule unique_id_module lib/apache/mod_unique_id.so LoadModule setenvif_modulelib/apache/mod_setenvif.so LoadModule php4_modulelib/apache/libphp4.so LoadModule perl_modulelib/apache/libperl.so LoadModule dav_module lib/apache/libdav.so LoadModule gzip_module lib/apache/mod_gzip.so IfDefine SSL LoadModule ssl_module lib/apache/libssl.so /IfDefine ... ClearModuleList AddModule mod_mmap_static.c AddModule mod_vhost_alias.c AddModule mod_env.c AddModule mod_define.c AddModule mod_log_config.c AddModule mod_log_agent.c AddModule mod_log_referer.c AddModule mod_mime_magic.c AddModule mod_mime.c AddModule mod_negotiation.c AddModule mod_status.c AddModule mod_info.c AddModule mod_include.c AddModule mod_autoindex.c AddModule mod_dir.c AddModule mod_cgi.c AddModule mod_asis.c AddModule mod_imap.c AddModule mod_actions.c AddModule mod_speling.c AddModule mod_userdir.c AddModule mod_alias.c AddModule mod_jk.c AddModule auth_ldap.c AddModule mod_rewrite.c AddModule mod_access.c AddModule mod_auth.c AddModule mod_auth_anon.c AddModule mod_auth_dbm.c AddModule mod_auth_db.c AddModule mod_digest.c AddModule mod_proxy.c AddModule mod_cern_meta.c AddModule mod_expires.c AddModule mod_headers.c AddModule mod_usertrack.c AddModule mod_example.c AddModule mod_unique_id.c AddModule mod_so.c AddModule mod_setenvif.c AddModule mod_php4.c AddModule mod_perl.c AddModule mod_dav.c AddModule mod_gzip.c IfDefine SSL AddModule mod_ssl.c /IfDefine
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
GOMEZ Henri wrote: Just did a DSO version and couldn't replicate the problem (BTW, I've recompiled/statically linked Apache and mod_jk again since then and the problem was still there). So, maybe it has to do with statically linking Apache after all... Strange problem. What's the difference between static and dynamic. We may have missed something at mod_jk init time ? This only happens during a graceful restart. Normal restart is OK. What does mod_jk do when it receives SIGUSR1? Does it actually do anything? Maybe that's the problem. Or it could have to do with Apache's behaviour on SIGUSR1. Something like, if DSO then modules are reloaded and therefore reinitialised, if static then the configuration file is re-read and that's it (pure speculative BS, I have no idea what Apache actually does). I kind of remember that people from mod_ssl were saying something about Apache init process being a bit strange (something like two passes) but that I think has to do with startup. Not sure. I still have to distribute this DSO version of Apache and mod_jk to some slower machines to see if that makes any difference at all. The machine that didn't produce any errors is a 1 GHz Athlon, so timing issues could have been resolved by brute force, who knows... Thanks to tell us what happened... Well, once I actually do that I surely will. I don't know if you've been in the position to run your own business, but I can only 'hack' when nothing else requires my immediate attention :-( Is there a recommendation as to the order of loading mod_jk for both statically linked version and the DSO version? Maybe that's important. I load mod_jk after alias and before rewrite and never get any problems. Not I also use php4 Thanks for the tips about module order. I'll see how that affects the whole thing. Bojan
Re: Problem with mod_jk 1.2.0 (latest CVS snapshot)
OOPS! Sorry :-( jk.conf (called from httpd.conf) - ### # Apache JK Configuration File# ### IfModule mod_jk.c JkWorkersFile /etc/tomcat/workers.properties JkLogFile /var/apache/logs/mod_jk.log JkLogLevelerror JkMount *.jsp ajp13 JkMount /*.jsp ajp13 JkMount *.vmajp13 JkMount /*.vm ajp13 JkMount /servlet/* ajp13 JkMount /login/j_security_check ajp13 /IfModule - workers.properties file: - ### #Apache JK Configuration File # ### # # Execution parameters ### # Setup for RedHat Linux system # workers.tomcat_home=/usr/local/tomcat workers.java_home=/usr/local/jdk ps=/ worker.list=ajp13 # Definition for Ajp13 worker # worker.ajp13.port=8009 worker.ajp13.host=localhost worker.ajp13.type=ajp13 - BTW, this is the stuff that was running OK with Tomcat 3.3 M3 and it runs OK with CVS Tomcat 3.3 + mod_jk from M3. As for DSO, I'll have to work on that, so probably some time tomorrow (Sydney time). One of the points that I'm trying to make anyway is that statically linked Apache is supported, so mod_jk should run with it ;-) It runs faster anyhow and it uses less memory. Bojan Gomez Henri wrote: After including it in Apache 1.3.20, statically compiled/linked for RedHat Linux 7.0, the error messages 'child pid x exit signal Segmentation fault (11)' start appearing in the error log file of Apache. mod_jk from Tomcat 3.3 M3 works fine with both Tomcat M3 and the latest 3.3 CVS snapshot. Could you try to rebuild it dynamically (DSO) and see if the problem persist. Thanks to send us mod_jk parts in httpd.conf and workers.properties file :) Here is the scenario to replicate the problem: - start Tomcat and Apache, mod_jk for latest CVS - call a few JSP/Velocity or whichever pages that exercise mod_jk - restart Tomcat - restart Apache (gracefully!) 2 seconds after Tomcat's startup (I'm doing this from my startup script) - try to load any page from the site you were just browsing and you'll get errors along the lines that the web server didn't return anything - log files show segfaults If you stop Tomcat and Apache and start them again, this doesn't happen. Also, when mod_jk shipped with M3 is used everything is sweet. Don't know all that much about mod_jk, but it seems the problem is related to graceful restart of Apache and subsequent reconnection to Tomcat (ie. part of the code that does that was changed and ouch, segfault :-) Hope this helps. Bojan - Henri Gomez ___[_] EMAIL : [EMAIL PROTECTED](. .) PGP KEY : 697ECEDD...oOOo..(_)..oOOo... PGP Fingerprint : 9DF8 1EA8 ED53 2F39 DC9B 904A 364F 80E6
