So you had a bug in your code. So what?
No bug - read this:
http://www.unix.org/version2/whatsnew/threadspaper.ps :
Registration of fork handlers (pthread_atfork( )). The fork handlers are
routines that are to
be executed in association with calls to the fork( ) function. There are
three
Paul Sheer, 2008-01-29:
Let's say you have 1600 clients. Let's say that you have 40 threads, and
each thread
handles 40 connections. Now let's say that each thread initializes it's own
SSL_CTX structure.
The SSL_CTX structure contains most of the data required for SSL
functionality.
Leandro Santi, 2008-01-29:
I won't argue with you about using the library in an
undocumented manner; but I *do* think it'd be interesting
to get some real quantitative data: we could use it as a
basis to discuss possible future library modifications,
more compatible with your requests.
One
This behavior, by itself, does not necessary guarantee
that your OpenSSL library code won't race against itself,
won't corrupt its own data, or crash (hint: learn about
the MySQL case, search the archives).
it's own data?? - well this is exactly why I asked on this
list :-) I wanted to get a
Further, on some systems you can't link with libpthread
if you intend to use fork(). I have two builds of my
software, one that does fork()ing and one that does
pthread_create()ing. So I am trying to avoid having to have two
installations of OpenSSL on every build platform.
I find it hard
I find it hard to believe that there exists a platform where:
On FreeBSD/OpenBSD my program outright core dumped and I could not
figure out why for days and days. Now I have two separate builds - one built
with -D_REENTRANT -DTHREADS ... -lpthread and one without.
Only with Linux do you have
On Tue, Jan 29, 2008 at 10:22:16PM +0200, Paul Sheer wrote:
I find it hard to believe that there exists a platform where:
On FreeBSD/OpenBSD my program outright core dumped and I could not
figure out why for days and days. Now I have two separate builds - one built
with -D_REENTRANT
On FreeBSD/OpenBSD my program outright core dumped and I could not
figure out why for days and days.
So you had a bug in your code. So what?
Now I have two separate builds - one built
with -D_REENTRANT -DTHREADS ... -lpthread and one without.
Only with Linux do you have the freedom of
Any argument which begins with on Linux or (generalized) on
[platform] is automatically suspect, regardless of whether there is
any currently-extant platform which violates the assumptions put
forward.
For an example of why this is a problem, remember the assumption on 32-
bit platforms
On Mon, Jan 21, 2008 at 05:34:43PM -0800, David Schwartz wrote:
- there is no difference between
multithreaded and non-multithreaded _compilation_ (surely not for errno
and malloc).
Really? So 'errno' refers to a process global in both cases?! (Note that I
said the definition, not the
errno is stored in Thread Local Storage (TLS). You can't link to the
global errno anymore.
For a single-threaded process, there is no distinction between thread-local
storage and a global variable. For a multi-threaded process, there is.
The same code can have a different semantic meaning
Tomas Mraz, 2008-01-24:
So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given
that there won't be any static and/or global variables in the OpenSSL
code called is 100% safe thing if the threads do not share any data
manipulated within the OpenSSL library.
As mentioned in
On Mon, Jan 28, 2008 at 02:22:09PM -0800, David Schwartz wrote:
errno is stored in Thread Local Storage (TLS). You can't link to the
global errno anymore.
For a single-threaded process, there is no distinction between thread-local
storage and a global variable. For a multi-threaded
Let's say you have 1600 clients. Let's say that you have 40 threads, and
each thread
handles 40 connections. Now let's say that each thread initializes it's own
SSL_CTX structure.
The SSL_CTX structure contains most of the data required for SSL
functionality.
Because each SSL_CTX structure has
Paul Sheer wrote:
Locking with no contention is not pretty expensive, it's darn
near free.
Oh? If this is true it changes things somewhat.
But I must say that I believe that no-one has ever used OpenSSL with
10'000 concurrent SSL objects. So I'm not going to take the chance
that
On Mon, 2008-01-21 at 17:34 -0800, David Schwartz wrote:
On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote:
Most definitely not. At a minimum, the definition of things
like 'errno' and
'malloc' might be different between a multithreaded build and a
non-multithreaded build.
So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given
that there won't be any static and/or global variables in the OpenSSL
code called is 100% safe thing [] if [] the threads do not share
any data
manipulated within the OpenSSL library.
Quite a big if there!! :-)
Really? So 'errno' refers to a process global in both cases?!
(Note that I
said the definition, not the implementation.)
Maybe we didn't understand each other - I don't say, that glibc without
multithread support and with it is the same. I say that linking single
threaded library which
So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given
that there won't be any static and/or global variables in the OpenSSL
code called is 100% safe thing if the threads do not share any data
manipulated within the OpenSSL library.
But that's not what he's doing. He's
Locking with no contention is not pretty expensive, it's darn near
free.
On systems with only one processor and nothing like hyperthreading.
/r$
--
STSM, DataPower Chief Programmer
WebSphere DataPower SOA Appliances
http://www.ibm.com/software/integration/datapower/
But I must say that I believe that no-one has ever used OpenSSL
with 10'000 concurrent SSL objects.
Umm, what?! We've last tested to almost 16,384. Our first test to 10,000 was
many years ago, on servers with Pentium processors and 128MB of RAM. We've
tested on operating systems from Windows
Locking with no contention is not pretty expensive, it's darn near
free.
On systems with only one processor and nothing like hyperthreading.
Did you miss the with no contention part? An uncontended lock costs about
the same on an SMP system as on an MP system. AFAIK, hyperthreading doesn't
I'm replying to David's email off-list...
-paul
On Jan 24, 2008 8:44 PM, David Schwartz [EMAIL PROTECTED] wrote:
But I must say that I believe that no-one has ever used OpenSSL
with 10'000 concurrent SSL objects.
Umm, what?! We've last tested to almost 16,384. Our first test to 10,000
On systems with only one processor and nothing like hyperthreading.
Did you miss the with no contention part?
No. I didn't realize you meant it as 'no possible contention.'
/r$
--
STSM, DataPower Chief Programmer
WebSphere DataPower SOA Appliances
I'm only familiar with Solaris. In that system the real stuff in a mutex is a
byte about 12 bytes into the lock structure. On SPARC the mutex_lock function
accesses it with an LDSTUB instruction, which is a special atomic instruction
that loads the old value into a register, and stores 0xff
On Thu, 2008-01-24 at 09:11 -0800, David Schwartz wrote:
Really? So 'errno' refers to a process global in both cases?!
(Note that I
said the definition, not the implementation.)
Maybe we didn't understand each other - I don't say, that glibc without
multithread support and with it
David Jacobson wrote:
I'm only familiar with Solaris. In that system the real stuff
in a mutex is a byte about 12 bytes into the lock structure.
On SPARC the mutex_lock function accesses it with an LDSTUB
instruction, which is a special atomic instruction that loads
the old value into a
Well my hybrid threaded app now seems to be stable - even under extreme
loads.
Here is what I did for others to refer:
Comments *most* welcome.
These steps allow me to link both builds of my program with the same
non-threaded
OpenSSL build. I.e. both the fork() and pthread_create() builds of my
Paul Sheer wrote:
I valgrind'ed OpenSSL as follows:
I compiled OpenSSL (0.9.8g) with my own random number engine - in
order to generate
pseudo random numbers that are not based on unitialized values (if you
run openssl
without doing this you get infinite warnings - of course).
The results
Am I doing anything completely insane here?
IMO, writing security software by doing something that is specifically not
documented or guaranteed to work and then trying to fix every problem it
creates (at least, that you can find) is completely insane.
DS
IMO, writing security software by doing something that is specifically
not
documented or guaranteed to work and then trying to fix every problem it
creates (at least, that you can find) is completely insane.
Guaranteed to work? Who's doing the indemnification?
Security's all about
IMO, writing security software by doing something that is specifically not
documented or guaranteed to work and then trying to fix every problem it
creates (at least, that you can find) is completely insane.
Ok, I managed to find another problem: error setting/getting (eg.
ERR_clear_error)
Paul Sheer wrote:
Well my hybrid threaded app now seems to be stable - even under extreme
loads.
2. Use my own RAND object (RAND_set_rand_method) so that
OpenSSL does
not try lock static globals.
How are you sure of this ? Did you manually remove the object code
Guaranteed to work? Who's doing the indemnification?
The point of a guarantee is that it is much less likely to change on another
machine or if a library is upgraded and compatability is claimed. Of course,
things can still go wrong.
When bugs are fixed in a library or a new version claims
: David Schwartz [EMAIL PROTECTED]
Date: Sun, 20 Jan 2008 11:59:00
To:openssl-dev@openssl.org
Subject: RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -
possible bug???)
I should be able to create a multithreaded application using
a non-multithreaded openssl build provided
On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote:
I should be able to create a multithreaded application using
a non-multithreaded openssl build provided that I have an ssl
context per thread.
Most definitely not. At a minimum, the definition of things like 'errno' and
'malloc'
On Mon, Jan 21, 2008 at 09:24:34AM +0100, Tomas Mraz wrote:
On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote:
I should be able to create a multithreaded application using
a non-multithreaded openssl build provided that I have an ssl
context per thread.
Most definitely not. At
On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote:
Most definitely not. At a minimum, the definition of things
like 'errno' and
'malloc' might be different between a multithreaded build and a
non-multithreaded build. There is no supported way to combine
multithreaded
code and
I should be able to create a multithreaded application using
a non-multithreaded openssl build provided that I have an ssl
context per thread.
Most definitely not. At a minimum, the definition of things like 'errno' and
'malloc' might be different between a multithreaded build and a
I valgrind'ed OpenSSL as follows:
I compiled OpenSSL (0.9.8g) with my own random number engine - in order to
generate
pseudo random numbers that are not based on unitialized values (if you run
openssl
without doing this you get infinite warnings - of course).
The results are as follows
Jan 2008 15:40:12
To:openssl-dev@openssl.org
Subject: Re: Two valgrind warnings in OpenSSL - possible bug???
I compiled OpenSSL (0.9.8g) with my own random number engine - in order
to generate
pseudo random numbers that are not based on unitialized values (if you
run openssl
without doing
On Sat, Jan 19, 2008 at 03:40:12PM -0500, Brad House wrote:
I compiled OpenSSL (0.9.8g) with my own random number engine - in order to
generate
pseudo random numbers that are not based on unitialized values (if you run
openssl
without doing this you get infinite warnings - of course).
The
Message-
From: Kurt Roeckx [EMAIL PROTECTED]
Date: Sun, 20 Jan 2008 00:17:57
To:openssl-dev@openssl.org
Subject: Re: Two valgrind warnings in OpenSSL - possible bug???
On Sat, Jan 19, 2008 at 03:40:12PM -0500, Brad House wrote:
I compiled OpenSSL (0.9.8g) with my own random number engine
43 matches
Mail list logo
