RE: segfault when using crypto library inside netscape plugin (Solaris 2.6/Sparc/openssl-0.9.5a)

[Steve Bazyl]

One more thing...I also tried adding lock callbacks to make sure its not a threading problem.  Made no difference (was getting lock requests as I should, and only from a single thread as expected).

 

 -----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Steve Bazyl
Sent: Tuesday, June 13, 2000 7:41 PM
To: [EMAIL PROTECTED]
Subject: segfault when using crypto library inside netscape plugin (Solaris 2.6/Sparc/openssl-0.9.5a)

We're having a really strange problem with the openssl crypto library -- it keeps segfaulting down in SHA1_Update when called from an NSAPI plugin (running in NES 3.6). 

 

I've tried building the library with optimizations off and all that fun stuff, and have run the test suite which it passes with flying colors.  I've also written various pieces of test code which drive the crypto lib with both static and dynamic linking, all works fine.  However, every time we run it inside NES, it crashes.

 

I've reduced it down to a simple piece of test code which promptly crashes the web server when invoked.

 

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <nsapi.h> 

 

NSAPI_PUBLIC int openssl_test( pblock *param, Session *sn, Request *rq )
{
  char seed[] = "some arbitrary data to seed the random number generator";

 

  printf ( "seeding..." );
  RAND_seed( seed, sizeof seed - 1); // Probably don't need the -1, but I'm getting paranoid :)
  printf ( "done\n" );
  return REQ_PROCEED;
}

 

To build (assuming you have openssl and netscape libs in appropriate places...):

gcc -G -o test_plugin.so test_plugin.c -lcrypto -lnsl -lsocket -DUNIX -DXP_UNIX -D_REENTRANT

 

Add to the server's obj.conf:

 

Init fn="load-modules" funcs="openssl_test" shlib="wherever_you_put_your_library"

Init fn="openssl_test"

 

Interestingly, if I link against the old SSLeay crypto library it works fine!!!!! (ok...I have an old binary, not quite sure how it was built...maybe something in the build options...probably not since all the tests pass fine?).

 

Any and all help is greatly appreciated :)