---- Joe Lewis<j...@joe-lewis.com> wrote:
On 6/21/12 9:39 PM, oh...@cox.net wrote:
---- oh...@cox.net wrote:
---- oh...@cox.net wrote:
---- Joe Lewis<j...@joe-lewis.com> wrote:
On 6/21/12 7:32 PM, oh...@cox.net wrote:
---- oh...@cox.net wrote:
---- Joe Lewis<j...@joe-lewis.com> wrote:
On 6/21/12 6:46 PM, oh...@cox.net wrote:
---- Joe Lewis<j...@joe-lewis.com> wrote:
On 6/21/12 5:49 PM, oh...@cox.net wrote:
---- oh...@cox.net wrote:
---- Sorin Manolache<sor...@gmail.com> wrote:
And I forgot to say: run gdb in some sort of environment
where you see
your current source code line and a couple of surrounding
lines. You
could achieve this with the "list" command, but I prefer
running gdb in
emacs and let emacs do the nice listing of source code in
a different panel.
S
Here's the function from my source. It's the original
from mod_headers.c, plus my printf:
static int header_post_config(apr_pool_t *pconf,
apr_pool_t *plog,
apr_pool_t *ptemp,
server_rec *s)
{
printf("In header_post_config\n");
header_ssl_lookup =
APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
return OK;
}
Jim
Hi,
I was able to get the segfault to go away. Here's what I
had to do:
- Created /etc/ld.so.conf.d/my.conf, and added the
directory where my libobaccess.so was
- Run 'ldconfig' to activate.
- In the apxs command, DON'T include the -L and -l arguments
After that, Apache appears to start ok, without segfault :)!!
Thanks for all of the great help, esp. the suggestion about
checking "ldconfig -p". I still don't understand why, but
I'm just glad that I can get past this piece so now I can
debug my module :)...
Later,
Jim
I'm just glad this list is as good as it is!
FYI, the ldconfig is the dynamic linker control, and those
/etc/ld.so.conf.d files provide additional search
directories for the
linker to check in when loading a library.
Joe
--
http://www.silverhawk.net/
Hi,
Sorry to report, but my earlier report was a "false positive"
:)...
I forgot that the mod_headers.c that I was doing the earlier
testing with had all references to the libobaccess.so removed
:(!!
So, I'm still stuck with basically the same problem, now,
working with my "full" code, with the calls in it:
- If I compile with -L and -l, Apache segfaults when it starts
- If I compile without -L and -l, then I get "undefined
symbol" errors when I try to start Apache, e.g.:
[root@apachemodule build-mod_headers]#
/apps/httpd2222/bin/apachectl -k start -X
httpd: Syntax error on line 84 of
/apps/httpd2222/conf/httpd.conf: Cannot load
/apps/httpd2222/modules/mod_headers.so into server:
/apps/httpd2222/modules/mod_headers.so: undefined symbol:
ObResource_isProtected
That "ObResource_isProtected" should be a symbol in
libobaccess.so, and in fact, if I do "nm --dynamic", I get:
[root@apachemodule build-mod_headers]# nm --dynamic
/apps/netpoint/lib64/libobaccess.so | grep
"ObResource_isProtected"
00000000000a6d80 T ObResource_isProtected
[root@apachemodule build-mod_headers]#
I'm *assuming* that the reason for the "undefined symbol"
error is that libobaccess.so is actually not being loaded,
but then when I try to load libobaccess.so, either via -L and
-l in the apxs, or using LoadFile in httpd.conf, I get the
segfault (same gdb info, BTW).
Catch-22?
Sorry for the false alarm :(!!
Jim
Not a catch-22. The -L and -l specify linker options when
assembling
the code. The ldconfig is a run-time thing. If you are
getting the
stderr messages, you are making it all the way into your
library. I'd
suggest commenting out the following line and see if you get
farther :
header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
That should tell you if the problem is the ssl_var_lookup.
Joe
--
http://www.silverhawk.net/
Hi,
Thanks for the suggestion. I just tried what you suggested,
and got a segfault when I started Apache with the modified module.
Jim
Hi,
As a reminder, here's the gdb with the library loaded:
(gdb) b header_post_config
Function "header_post_config" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (header_post_config) pending.
(gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
-f /apps/httpd2222/conf/httpd.conf
[Thread debugging using libthread_db enabled]
[New Thread 182897610272 (LWP 11317)]
Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
Pending breakpoint "header_post_config" resolved
mod_headers-jl V0.09 - start calling OAM API
In register_hooks
In create_headers_dir_config
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
[Switching to Thread 182897610272 (LWP 11317)]
Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
1121 printf("In header_post_config\n");
(gdb) n
1120 {
(gdb) n
1121 printf("In header_post_config\n");
(gdb) n
In header_post_config
1122 header_ssl_lookup =
APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
(gdb) n
1124 }
(gdb) n
0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
(gdb) n
Program received signal SIGSEGV, Segmentation fault.
0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
So, it's actually blowing up in "BN_num_bits()" in
/lib64/libcrypto.so.4?
Jim
When you see the :
Program received signal SIGSEGV, Segmentation fault.
0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
in gdb, type "bt" and hit enter. It should show the back trace
and how
you got to the BN_num_bits() function.
Joe
--
http://www.silverhawk.net/
Hi,
Here's the bt full:
In header_inout_cmd
In parse_format_tag
In parse_misc_string
[Switching to Thread 182897610272 (LWP 6676)]
Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
1121 printf("In header_post_config\n");
(gdb) n
1120 {
(gdb) n
1121 printf("In header_post_config\n");
(gdb) n
In header_post_config
1122 header_ssl_lookup =
APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
(gdb) n
1124 }
(gdb) n
0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
(gdb) n
Program received signal SIGSEGV, Segmentation fault.
0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
(gdb) bt full
#0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
No symbol table info available.
#1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
/lib64/libcrypto.so.4
No symbol table info available.
#2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
No symbol table info available.
#3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
No symbol table info available.
#4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
/lib64/libcrypto.so.4
No symbol table info available.
#5 0x0000003518dae5f2 in ASN1_template_i2d () from
/lib64/libcrypto.so.4
No symbol table info available.
#6 0x0000003518dae28e in ASN1_item_ex_i2d () from
/lib64/libcrypto.so.4
No symbol table info available.
#7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
No symbol table info available.
#8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
p=0x5a72d8) at ssl_engine_pphrase.c:505
mc = (SSLModConfigRec *) 0x571738
sc = (SSLSrvConfigRec *) 0x668c38
pServ = (server_rec *) 0x65fa48
cpVHostID = 0x60add0 "www.example.com:443"
szPath =
"/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
'\0'<rep eats 15
times>,
"\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
'\0'<repeats 35 times>, "à\224k",
' ---Type<return>
to continue, or q<return> to quit---
\0'<repeats 13 times>,
"øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
'\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
'\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
'\0'<repeats 35 times>...
pPrivateKey = (EVP_PKEY *) 0x6ba670
asn1 = Variable "asn1" is not available.
(gdb)
Not really sure what to make of that though :(...
Jim
Hi,
Ok, something interesting. I noticed that the servername in the
gdb output was example.whatever.com, so I changed ServerName in
extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
segfaulted after that.
I noticed that igdb was showing my router IP address for
"mod_unique_id" (whatever that is), so I thought it was because I
didn't have the hostname in /etc/hosts. So, I added
"apache1.whatever.com" to /etc/hosts, and also turned Apache
LogLevel to debug, and now I get a different error:
[root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
mod_headers-jl V0.09 - start calling OAM API
In register_hooks
In create_headers_dir_config
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In header_post_config
*** glibc detected *** corrupted double-linked list:
0x00000000006b9710 ***
/apps/httpd2222/bin/apachectl: line 78: 7599
Aborted $HTTPD $ARGV
[root@apachemodule dev]#
Here's the Apache error_log:
[Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
72.215.225.9
[Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
of entropy
[Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
key of SSL-aware server
[Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
72.215.225.9
[Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
of entropy
[Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
key of SSL-aware server
[Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
72.215.225.9
[Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
of entropy
[Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
key of SSL-aware server
[root@apachemodule dev]#
and here's the gdb backtrace:
(gdb) b header_post_config
Function "header_post_config" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (header_post_config) pending.
(gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
/apps/httpd2222/conf/httpd.conf
[Thread debugging using libthread_db enabled]
[New Thread 182897610272 (LWP 7644)]
Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
Pending breakpoint "header_post_config" resolved
mod_headers-jl V0.09 - start calling OAM API
In register_hooks
In create_headers_dir_config
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
[Switching to Thread 182897610272 (LWP 7644)]
Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
1121 printf("In header_post_config\n");
(gdb) n
1120 {
(gdb) n
1121 printf("In header_post_config\n");
(gdb) n
In header_post_config
1122 header_ssl_lookup =
APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
(gdb) n
1124 }
(gdb) n
0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
(gdb) n
*** glibc detected *** corrupted double-linked list:
0x00000000006b9710 ***
Program received signal SIGABRT, Aborted.
0x000000351432e26d in raise () from /lib64/tls/libc.so.6
(gdb) bt full
#0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
No symbol table info available.
#1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
No symbol table info available.
#2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
No symbol table info available.
#3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
No symbol table info available.
#4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
No symbol table info available.
#5 0x0000002a9790b6ba in R_free () from
/apps/netpoint/lib64/libobaccess.so
No symbol table info available.
#6 0x0000002a9792bc41 in X509_CINF_free () from
/apps/netpoint/lib64/libobaccess.so
No symbol table info available.
#7 0x0000002a9790ed98 in X509_free () from
/apps/netpoint/lib64/libobaccess.so
No symbol table info available.
#8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
p=0x5a72d8) at ssl_engine_pphrase.c:243
mc = (SSLModConfigRec *) 0x571738
sc = (SSLSrvConfigRec *) 0x668c38
pServ = (server_rec *) 0x65fa48
cpVHostID = 0x60add0 "apache1.whatever.com:443"
szPath =
"/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
'\0'<repeats 15 times>,
"\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
'\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
"øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
'\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
'\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
'\0'<repeats 35 times>...
pPrivateKey = Variable "pPrivateKey" is not available.
(gdb)
Jim
I guess the thing that I'm puzzled about is why it goes from
ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
in the libobaccess.so? Apache shouldn't even be aware of the
libobaccess stuff yet (since I think that Apache is still trying to
initialize SSL processing)?
Jim
It shouldn't unless you've declared a shutdown hook or a clean up hook
of sorts. The looks of that stack trace look strikingly like an
exception handler firing off. What are all of the hooks you have
running?
Joe
--
http://www.silverhawk.net/
Hi,
I'm basically using the original mod_headers.c as a starter code, and
stuck some stuff in there. Haven't done anything with the hooks etc.
yet, so whatever was there is there:
static void register_hooks(apr_pool_t *p)
{
printf("mod_headers-jl V0.09 - start calling OAM API\n");
printf("In register_hooks\n");
ap_register_output_filter("FIXUP_HEADERS_OUT",
ap_headers_output_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_register_output_filter("FIXUP_HEADERS_ERR",
ap_headers_error_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
NULL, APR_HOOK_LAST);
ap_hook_insert_error_filter(ap_headers_insert_error_filter,
NULL, NULL, APR_HOOK_LAST);
ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
ap_hook_post_read_request(ap_headers_early, NULL, NULL,
APR_HOOK_FIRST);
}
module AP_MODULE_DECLARE_DATA headers_module =
{
STANDARD20_MODULE_STUFF,
create_headers_dir_config, /* dir config creater */
merge_headers_config, /* dir merger --- default is to
override */
NULL, /* server config */
NULL, /* merge server configs */
headers_cmds, /* command apr_table_t */
register_hooks /* register hooks */
};
I guess that there is a:
ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
in that register_hooks(), but it was there from the original code.
BTW, I was checking, and it looks like there's an X509_free() function
in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
I'm wondering if things are confused, and maybe Apache code expects to
call the openssl X509_free, but somehow ends up calling the
X509_free() in libobaccess.so?
What would cause something like that? If two .so files have the same
function, how to avoid conflicts?
Jim
