---- oh...@cox.net wrote:
>
> ---- oh...@cox.net wrote:
> >
> > ---- Joe Lewis <j...@joe-lewis.com> wrote:
> > > On 6/21/12 7:32 PM, oh...@cox.net wrote:
> > > > ---- oh...@cox.net wrote:
> > > >> ---- Joe Lewis<j...@joe-lewis.com> wrote:
> > > >>> On 6/21/12 6:46 PM, oh...@cox.net wrote:
> > > >>>> ---- Joe Lewis<j...@joe-lewis.com> wrote:
> > > >>>>> On 6/21/12 5:49 PM, oh...@cox.net wrote:
> > > >>>>>> ---- oh...@cox.net wrote:
> > > >>>>>>> ---- Sorin Manolache<sor...@gmail.com> wrote:
> > > >>>>>>>> And I forgot to say: run gdb in some sort of environment where
> > > >>>>>>>> you see
> > > >>>>>>>> your current source code line and a couple of surrounding lines.
> > > >>>>>>>> You
> > > >>>>>>>> could achieve this with the "list" command, but I prefer running
> > > >>>>>>>> gdb in
> > > >>>>>>>> emacs and let emacs do the nice listing of source code in a
> > > >>>>>>>> different panel.
> > > >>>>>>>>
> > > >>>>>>>> S
> > > >>>>>>> Here's the function from my source. It's the original from
> > > >>>>>>> mod_headers.c, plus my printf:
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> > > >>>>>>> apr_pool_t *ptemp, server_rec *s)
> > > >>>>>>> {
> > > >>>>>>> printf("In header_post_config\n");
> > > >>>>>>> header_ssl_lookup =
> > > >>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > > >>>>>>> return OK;
> > > >>>>>>> }
> > > >>>>>>>
> > > >>>>>>> Jim
> > > >>>>>> Hi,
> > > >>>>>>
> > > >>>>>> I was able to get the segfault to go away. Here's what I had to
> > > >>>>>> do:
> > > >>>>>>
> > > >>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where
> > > >>>>>> my libobaccess.so was
> > > >>>>>> - Run 'ldconfig' to activate.
> > > >>>>>> - In the apxs command, DON'T include the -L and -l arguments
> > > >>>>>>
> > > >>>>>> After that, Apache appears to start ok, without segfault :)!!
> > > >>>>>>
> > > >>>>>> Thanks for all of the great help, esp. the suggestion about
> > > >>>>>> checking "ldconfig -p". I still don't understand why, but I'm
> > > >>>>>> just glad that I can get past this piece so now I can debug my
> > > >>>>>> module :)...
> > > >>>>>>
> > > >>>>>> Later,
> > > >>>>>> Jim
> > > >>>>> I'm just glad this list is as good as it is!
> > > >>>>>
> > > >>>>> FYI, the ldconfig is the dynamic linker control, and those
> > > >>>>> /etc/ld.so.conf.d files provide additional search directories for
> > > >>>>> the
> > > >>>>> linker to check in when loading a library.
> > > >>>>>
> > > >>>>> Joe
> > > >>>>> --
> > > >>>>> http://www.silverhawk.net/
> > > >>>> Hi,
> > > >>>>
> > > >>>> Sorry to report, but my earlier report was a "false positive" :)...
> > > >>>>
> > > >>>> I forgot that the mod_headers.c that I was doing the earlier testing
> > > >>>> with had all references to the libobaccess.so removed :(!!
> > > >>>>
> > > >>>> So, I'm still stuck with basically the same problem, now, working
> > > >>>> with my "full" code, with the calls in it:
> > > >>>>
> > > >>>> - If I compile with -L and -l, Apache segfaults when it starts
> > > >>>>
> > > >>>> - If I compile without -L and -l, then I get "undefined symbol"
> > > >>>> errors when I try to start Apache, e.g.:
> > > >>>>
> > > >>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl
> > > >>>> -k start -X
> > > >>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf:
> > > >>>> Cannot load /apps/httpd2222/modules/mod_headers.so into server:
> > > >>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
> > > >>>> ObResource_isProtected
> > > >>>>
> > > >>>> That "ObResource_isProtected" should be a symbol in libobaccess.so,
> > > >>>> and in fact, if I do "nm --dynamic", I get:
> > > >>>>
> > > >>>> [root@apachemodule build-mod_headers]# nm --dynamic
> > > >>>> /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> > > >>>> 00000000000a6d80 T ObResource_isProtected
> > > >>>> [root@apachemodule build-mod_headers]#
> > > >>>>
> > > >>>>
> > > >>>> I'm *assuming* that the reason for the "undefined symbol" error is
> > > >>>> that libobaccess.so is actually not being loaded, but then when I
> > > >>>> try to load libobaccess.so, either via -L and -l in the apxs, or
> > > >>>> using LoadFile in httpd.conf, I get the segfault (same gdb info,
> > > >>>> BTW).
> > > >>>>
> > > >>>> Catch-22?
> > > >>>>
> > > >>>> Sorry for the false alarm :(!!
> > > >>>>
> > > >>>> Jim
> > > >>>>
> > > >>>>
> > > >>> Not a catch-22. The -L and -l specify linker options when assembling
> > > >>> the code. The ldconfig is a run-time thing. If you are getting the
> > > >>> stderr messages, you are making it all the way into your library. I'd
> > > >>> suggest commenting out the following line and see if you get farther :
> > > >>>
> > > >>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > > >>>
> > > >>> That should tell you if the problem is the ssl_var_lookup.
> > > >>>
> > > >>> Joe
> > > >>> --
> > > >>> http://www.silverhawk.net/
> > > >>
> > > >> Hi,
> > > >>
> > > >> Thanks for the suggestion. I just tried what you suggested, and got a
> > > >> segfault when I started Apache with the modified module.
> > > >>
> > > >> Jim
> > > >>
> > > >>
> > > > Hi,
> > > >
> > > > As a reminder, here's the gdb with the library loaded:
> > > >
> > > >
> > > > (gdb) b header_post_config
> > > > Function "header_post_config" not defined.
> > > > Make breakpoint pending on future shared library load? (y or [n]) y
> > > > Breakpoint 1 (header_post_config) pending.
> > > > (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > > > Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
> > > > /apps/httpd2222/conf/httpd.conf
> > > > [Thread debugging using libthread_db enabled]
> > > > [New Thread 182897610272 (LWP 11317)]
> > > > Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> > > > Pending breakpoint "header_post_config" resolved
> > > > mod_headers-jl V0.09 - start calling OAM API
> > > > In register_hooks
> > > > In create_headers_dir_config
> > > > In create_headers_dir_config
> > > > In header_cmd
> > > > In header_inout_cmd
> > > > In parse_format_tag
> > > > In parse_misc_string
> > > > In create_headers_dir_config
> > > > In header_cmd
> > > > In header_inout_cmd
> > > > In parse_format_tag
> > > > In parse_misc_string
> > > > [Switching to Thread 182897610272 (LWP 11317)]
> > > >
> > > > Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> > > > ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > > > 1121 printf("In header_post_config\n");
> > > > (gdb) n
> > > > 1120 {
> > > > (gdb) n
> > > > 1121 printf("In header_post_config\n");
> > > > (gdb) n
> > > > In header_post_config
> > > > 1122 header_ssl_lookup =
> > > > APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > > > (gdb) n
> > > > 1124 }
> > > > (gdb) n
> > > > 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> > > > plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > > > 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > > > (gdb) n
> > > >
> > > > Program received signal SIGSEGV, Segmentation fault.
> > > > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > > >
> > > >
> > > > So, it's actually blowing up in "BN_num_bits()" in
> > > > /lib64/libcrypto.so.4?
> > > >
> > > > Jim
> > > When you see the :
> > >
> > > Program received signal SIGSEGV, Segmentation fault.
> > > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > >
> > >
> > > in gdb, type "bt" and hit enter. It should show the back trace and how
> > > you got to the BN_num_bits() function.
> > >
> > > Joe
> > > --
> > > http://www.silverhawk.net/
> >
> >
> > Hi,
> >
> > Here's the bt full:
> >
> >
> > In header_inout_cmd
> > In parse_format_tag
> > In parse_misc_string
> > [Switching to Thread 182897610272 (LWP 6676)]
> >
> > Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> > ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > 1121 printf("In header_post_config\n");
> > (gdb) n
> > 1120 {
> > (gdb) n
> > 1121 printf("In header_post_config\n");
> > (gdb) n
> > In header_post_config
> > 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > (gdb) n
> > 1124 }
> > (gdb) n
> > 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8,
> > ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > (gdb) n
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > (gdb) bt full
> > #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
> > /lib64/libcrypto.so.4
> > No symbol table info available.
> > #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #5 0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #6 0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at
> > ssl_engine_pphrase.c:505
> > mc = (SSLModConfigRec *) 0x571738
> > sc = (SSLSrvConfigRec *) 0x668c38
> > pServ = (server_rec *) 0x65fa48
> > cpVHostID = 0x60add0 "www.example.com:443"
> > szPath =
> > "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177",
> > '\0' <rep eats 15 times>,
> > "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177",
> > '\0' <repeats 35 times>, "à\224k", '
> > ---Type <return> to continue, or q <return> to quit---
> > \0' <repeats 13 times>,
> > "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> > '\0' <repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'
> > <repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0' <repeats
> > 35 times>...
> > pPrivateKey = (EVP_PKEY *) 0x6ba670
> > asn1 = Variable "asn1" is not available.
> > (gdb)
> >
> >
> > Not really sure what to make of that though :(...
> >
> > Jim
>
>
> Hi,
>
> Ok, something interesting. I noticed that the servername in the gdb output
> was example.whatever.com, so I changed ServerName in extras/httpd-ssl.conf to
> "apache1.whatever.com". Apache still segfaulted after that.
>
> I noticed that igdb was showing my router IP address for "mod_unique_id"
> (whatever that is), so I thought it was because I didn't have the hostname in
> /etc/hosts. So, I added "apache1.whatever.com" to /etc/hosts, and also
> turned Apache LogLevel to debug, and now I get a different error:
>
> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> mod_headers-jl V0.09 - start calling OAM API
> In register_hooks
> In create_headers_dir_config
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> In header_post_config
> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
> /apps/httpd2222/bin/apachectl: line 78: 7599 Aborted $HTTPD
> $ARGV
> [root@apachemodule dev]#
>
>
> Here's the Apache error_log:
>
> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> [Thu Jun 21 23:24:12 2012] [info] Loading certificate & private key of
> SSL-aware server
> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> [Thu Jun 21 23:29:52 2012] [info] Loading certificate & private key of
> SSL-aware server
> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> [Thu Jun 21 23:29:57 2012] [info] Loading certificate & private key of
> SSL-aware server
> [root@apachemodule dev]#
>
>
> and here's the gdb backtrace:
>
> (gdb) b header_post_config
> Function "header_post_config" not defined.
> Make breakpoint pending on future shared library load? (y or [n]) y
> Breakpoint 1 (header_post_config) pending.
> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
> /apps/httpd2222/conf/httpd.conf
> [Thread debugging using libthread_db enabled]
> [New Thread 182897610272 (LWP 7644)]
> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> Pending breakpoint "header_post_config" resolved
> mod_headers-jl V0.09 - start calling OAM API
> In register_hooks
> In create_headers_dir_config
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> [Switching to Thread 182897610272 (LWP 7644)]
>
> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> 1121 printf("In header_post_config\n");
> (gdb) n
> 1120 {
> (gdb) n
> 1121 printf("In header_post_config\n");
> (gdb) n
> In header_post_config
> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> (gdb) n
> 1124 }
> (gdb) n
> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8,
> ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> (gdb) n
> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
>
> Program received signal SIGABRT, Aborted.
> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> (gdb) bt full
> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> No symbol table info available.
> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> No symbol table info available.
> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> No symbol table info available.
> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> No symbol table info available.
> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
> No symbol table info available.
> #5 0x0000002a9790b6ba in R_free () from /apps/netpoint/lib64/libobaccess.so
> No symbol table info available.
> #6 0x0000002a9792bc41 in X509_CINF_free () from
> /apps/netpoint/lib64/libobaccess.so
> No symbol table info available.
> #7 0x0000002a9790ed98 in X509_free () from
> /apps/netpoint/lib64/libobaccess.so
> No symbol table info available.
> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at
> ssl_engine_pphrase.c:243
> mc = (SSLModConfigRec *) 0x571738
> sc = (SSLSrvConfigRec *) 0x668c38
> pServ = (server_rec *) 0x65fa48
> cpVHostID = 0x60add0 "apache1.whatever.com:443"
> szPath =
> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177",
> '\0' <repeats 15 times>,
> "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177",
> '\0' <repeats 35 times>, "à\224k", '\0' <repeats 13 times>,
> "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> '\0' <repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'
> <repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0' <repeats 35
> times>...
> pPrivateKey = Variable "pPrivateKey" is not available.
> (gdb)
>
>
> Jim
>
I guess the thing that I'm puzzled about is why it goes from
ssl_pphrase_Handle (which is part of Apache code?) to X509_free () in the
libobaccess.so? Apache shouldn't even be aware of the libobaccess stuff yet
(since I think that Apache is still trying to initialize SSL processing)?
Jim
