----- Original Message -----
From: Jeff Trawick <[email protected]>
To: [email protected]
Cc:
Sent: Thursday, July 19, 2012 5:15 PM
Subject: Re: [users@httpd] Apache httpd 2.2.21 Segmentation faults
On Thu, Jul 19, 2012 at 7:32 AM, Abhi Auradkar <[email protected]> wrote:
> Jeff,
>
> The cwmp handler has no functions like 'request_handler' and
>'release_lock'.
> The internal redirect from cwmp handler goes to weblogic apache plugin which
> has to tunnel the request to a weblogic server
cluster.
Interesting...
I suggest trying the latest WLS plugin release (11.1.1.6.0) and, if
that doesn't resolve the problem, contact Oracle support.
>
>
>
> Thanks,
> ~Abhi
>
> ----- Original Message -----
>
> From: Jeff Trawick <[email protected]>
> To: [email protected]
> Cc:
> Sent: Thursday, July 19, 2012 4:49 PM
> Subject: Re: [users@httpd] Apache httpd 2.2.21 Segmentation faults
>
> On Thu, Jul 19, 2012 at 5:18 AM, Abhi Auradkar <[email protected]> wrote:
>> Hi,
>>
>> I tried attaching the debugger to apache running as a single
>>instance(httpd -X) and head also turned on memcheck and accesscheck(check
>>-all) with librtc.
>> I saw a core dump and it had this stack. Is something going wrong in
>> internal_redirect()?
>>
>> (dbx) where
>> current thread: t@3
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/config.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/config.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/modules/http/http_request.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/modules/http/http_request.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/config.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/config.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/modules/http/http_request.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/modules/http/http_request.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/modules/http/http_core.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/modules/http/http_core.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/connection.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/connection.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/mpm/worker/worker.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/mpm/worker/worker.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/mpm/worker/worker.c"
>> dbx: warning: can't find file
>> "/data/build_area/xml-data/build-dir/HDM-APACHEHTTPD2218-APACHE2221DEBUG/server/mpm/worker/worker.c"
>> [1] __rtc_trap_handler(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfee96e24
>> [2] 0x0(0x1205e0, 0x0, 0x0, 0x0, 0x0, 0xb), at 0x0
>
> This is the point of failure. release_lock() is branching to zero.
>
>> [3] release_lock(0x169050, 0x1cc8b0, 0x0, 0x189af0, 0x5, 0x1010101), at
>>0xfe1504a8
>> [4] request_handler(0x1539e0, 0x11d588, 0x1539e0, 0x0, 0x0, 0xfffeb748),
>>at 0xfe15dfa8
>
> request_handler and release_lock are third-party code. (same module
> as the cwmp function seen earlier in the stack?)
>
> This needs to be debugged from the perspective of that third-party
> code -- what was release_lock() trying to do when it branched to
> location zero, and why wasn't a valid function pointer available.
>
>> [5] ap_run_handler(0x1539e0, 0x1cd7a8, 0x1717ca, 0xfef27378, 0xfee96c9c,
>>0xfee93dec), at 0x50314
>> [6] ap_invoke_handler(r = 0x1539e0), at 0x50fa4
>> [7] ap_internal_redirect_handler(new_uri = 0x12bdf0
>>"/cwmp-allow/cwmpWeb/CPEMgt", r = 0x152ed0), at 0x7e34c
>> =>[8] cwmp_throttle_handler(r = 0x152ed0) (optimized), at 0xfe697c2c (line
>> ~2403) in "mod_cwmp.c"
>> [9] ap_run_handler(0x152ed0, 0x159ef0, 0x152ed0, 0x10b098, 0x153068,
>>0x153070), at 0x50314
>> [10] ap_invoke_handler(r = 0x152ed0), at 0x50fa4
>> [11] ap_process_request(r = 0x152ed0), at 0x7d6dc
>> [12] ap_process_http_connection(c = 0x10b098), at 0x78e98
>> [13] ap_run_process_connection(0x10b098, 0x168268, 0x168268, 0x0,
>>0x1cddf0, 0xf79d0), at 0x5cba4
>> [14] ap_process_connection(c = 0x10b098, csd = 0x168268), at 0x5d270
>> [15] process_socket(p = 0x110fe8, sock = 0x168268, my_child_num = 0,
>>my_thread_num = 0, bucket_alloc = 0xf79d0), at 0x91e8c
>> [16] worker_thread(thd = 0x110e40, dummy = 0xe8690), at 0x92a68
>> [17] dummy_worker(opaque = 0x110e40), line 142 in "thread.c"
>> (dbx) threads
>>
>>
>>
>> ----- Original Message -----
>> From: Jeff Trawick <[email protected]>
>> To: [email protected]; Abhi Auradkar <[email protected]>
>> Cc:
>> Sent: Wednesday, June 6, 2012 1:25 AM
>> Subject: Re: [users@httpd] Apache httpd 2.2.21 Segmentation faults
>>
>> On Mon, Jun 4, 2012 at 2:27 AM, Abhi Auradkar <[email protected]> wrote:
>>> Hi All,
>>>
>>>
>>> I am running a apache 2.2.21 on Solaris Sparc. I see the following
>>> segmantation faults every now and then.
>>> Any pointers to whats wrong will be really helpful.
>>
>> Please get the stack of this thread, which is probably the one that crashed:
>>
>> t@68 a l@68 dummy_worker() LWP suspended in in <can't get PC>()
>>
>> As is also visible with pflags sometimes, Solaris can misidentify the
>> main thread of the process as the crasher. That is blocked in read()
>> during steady state and cannot crash.
>>
>>>
>>> Thanks.
>>>
>>> t@1 (l@1) terminated by signal SEGV (Segmentation Fault)
>>> 0xff03e3ac: _read+0x000c: bcc,pt %icc,_read+0x20 ! 0xff03e3c0
>>> (dbx) where
>>> current thread: t@1
>>> =>[1] _read(0x4, 0xffbff84b, 0x1, 0x0, 0x0, 0x0), at 0xff03e3ac
>>> [2] read(0x8, 0xffbff84b, 0x1, 0x0, 0x1cf4, 0x8), at 0xff02ca1c
>>> [3] ap_mpm_pod_check(0xd6d70, 0x6b854, 0xa0000, 0xff232a00, 0xfff7, 0x0),
>>> at 0x6f408
>>> [4] child_main(0x5, 0xffbff8b4, 0x135d98, 0x1, 0x9f0ec, 0x1b0870), at
>>> 0x6cbbc
>>> [5] perform_idle_server_maintenance(0x0, 0x9f000, 0xafb70, 0x2, 0x5, 0x0),
>>> at 0x6d6ac
>>> [6] server_main_loop(0x0, 0x0, 0x5, 0x9f0b0, 0x0, 0xffbffa78), at 0x6dd10
>>> [7] ap_mpm_run(0x20, 0xa07d8, 0x9f0b8, 0x0, 0x20, 0x60), at 0x6e000
>>> [8] main(0x9e800, 0x4b8d0, 0x9ebb4, 0x7ddd0, 0x9d8e8, 0x9e800), at 0x29eac
>>> (dbx) threads
>>> o> t@1 a l@1 ?() signal SIGSEGV in _read()
>>> t@3 a l@3 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@4 a l@4 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@5 a l@5 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@6 a l@6 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@7 a l@7 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@8 a l@8 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@9 a l@9 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@10 a l@10 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@11 a l@11 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@12 a l@12 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@13 a l@13 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@14 a l@14 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@15 a l@15 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@16 a l@16 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@17 a l@17 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@18 a l@18 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@19 a l@19 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@20 a l@20 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@21 a l@21 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@22 a l@22 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@23 a l@23 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@24 a l@24 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@25 a l@25 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@26 a l@26 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@27 a l@27 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@28 a l@28 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@29 a l@29 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@30 a l@30 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@31 a l@31 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@32 a l@32 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@33 a l@33 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@34 a l@34 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@35 a l@35 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@36 a l@36 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@37 a l@37 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@38 a l@38 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@39 a l@39 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@40 a l@40 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@41 a l@41 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@42 a l@42 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@43 a l@43 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@44 a l@44 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@45 a l@45 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@46 a l@46 dummy_worker() LWP suspended in _stat64()
>>> t@47 a l@47 dummy_worker() LWP suspended in __lwp_park()
>>> t@48 a l@48 dummy_worker() LWP suspended in __pollsys()
>>> t@49 a l@49 dummy_worker() LWP suspended in __pollsys()
>>> t@50 a l@50 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@51 a l@51 dummy_worker() LWP suspended in strstr()
>>> t@52 a l@52 dummy_worker() LWP suspended in __pollsys()
>>> t@53 a l@53 dummy_worker() LWP suspended in __pollsys()
>>> t@54 a l@54 dummy_worker() LWP suspended in __pollsys()
>>> t@55 a l@55 dummy_worker() sleep on 0xff0b5a60 in __lwp_park()
>>> t@56 a l@56 dummy_worker() LWP suspended in _lstat64()
>>> t@57 a l@57 dummy_worker() LWP suspended in __pollsys()
>>> t@58 a l@58 dummy_worker() LWP suspended in _brk_unlocked()
>>> t@59 a l@59 dummy_worker() LWP suspended in __pollsys()
>>> t@60 a l@60 dummy_worker() LWP suspended in _lstat64()
>>> t@61 a l@61 dummy_worker() LWP suspended in __pollsys()
>>> t@62 a l@62 dummy_worker() LWP suspended in __pollsys()
>>> t@63 a l@63 dummy_worker() LWP suspended in _lstat64()
>>> t@64 a l@64 dummy_worker() LWP suspended in __lwp_park()
>>> t@65 a l@65 dummy_worker() LWP suspended in __pollsys()
>>> t@66 a l@66 dummy_worker() sleep on 0x135e30 in __lwp_park()
>>> t@67 a l@67 dummy_worker() LWP suspended in __lwp_unpark()
>>> t@68 a l@68 dummy_worker() LWP suspended in in <can't get PC>()
>>> t@69 a l@69 dummy_worker() LWP suspended in ___nanosleep()
>>> (dbx)
>>>
>>>
>>> Apache Version Info
>>> =============
>>>
>>> /usr/local/apache2/bin/httpd -v
>>> Server version: Apache/2.2.21 (Unix)
>>> Server built: Jan 19 2012 09:52:37
>>> bash-3.00# /usr/local/apache2/bin/httpd -V
>>> Server version: Apache/2.2.21 (Unix)
>>> Server built: Jan 19 2012 09:52:37
>>> Server's Module Magic Number: 20051115:30
>>> Server loaded: APR 1.4.5, APR-Util 1.3.12
>>> Compiled using: APR 1.4.5, APR-Util 1.3.12
>>> Architecture: 32-bit
>>> Server MPM: Worker
>>> threaded: yes (fixed thread count)
>>> forked: yes (variable process count)
>>> Server compiled with....
>>> -D APACHE_MPM_DIR="server/mpm/worker"
>>> -D APR_HAS_SENDFILE
>>> -D APR_HAS_MMAP
>>> -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
>>> -D APR_USE_PROC_PTHREAD_SERIALIZE
>>> -D APR_USE_PTHREAD_SERIALIZE
>>> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
>>> -D APR_HAS_OTHER_CHILD
>>> -D AP_HAVE_RELIABLE_PIPED_LOGS
>>> -D DYNAMIC_MODULE_LIMIT=128
>>> -D HTTPD_ROOT="/usr/local/apache2"
>>> -D SUEXEC_BIN="/usr/local/apache2/bin/suexec"
>>> -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
>>> -D DEFAULT_ERRORLOG="logs/error_log"
>>> -D AP_TYPES_CONFIG_FILE="conf/mime.types"
>>> -D SERVER_CONFIG_FILE="conf/httpd.conf"
>>>
>>>
>>
>>
>>
>> --
>> Born in Roswell... married an alien...
>> http://emptyhammock.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
>
>
>
> --
> Born in Roswell... married an alien...
> http://emptyhammock.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
--
Born in Roswell... married an alien...
http://emptyhammock.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
