[RFC PATCH] s_client/s_server: support unix domain sockets
The -unix path argument allows s_server and s_client to use a unix
domain socket in the filesystem instead of IPv4 (-connect, -port,
-accept, etc). If s_server exits gracefully, such as when -naccept
is used and the requested number of SSL/TLS connections have occurred,
then the domain socket file is removed. On ctrl-C, it is likely that
the stale socket file will be left over, such that s_server would
normally fail to restart with the same arguments. For this reason,
s_server also supports an -unlink option, which will clean up any
stale socket file before starting.
If you have any reason to want encrypted IPC within an O/S instance,
this concept might come in handy. Otherwise it just demonstrates that
there is nothing about SSL/TLS that limits it to TCP/IP in any way.
(There might also be benchmarking and profiling use in this path, as
unix domain sockets are much lower overhead than connecting over local
IP addresses).
Signed-off-by: Geoff Thorpe ge...@openssl.org
---
This is just a request for comments. Anyone think this is worth
putting in? Eg.
[In one shell]
cd apps openssl s_server -unix foobar -naccept 1
[In another shell]
cd apps openssl s_client -unix foobar
apps/s_apps.h | 12 -
apps/s_client.c | 17 ++-
apps/s_server.c | 45 --
apps/s_socket.c | 138 +++-
e_os.h | 10
5 files changed, 214 insertions(+), 8 deletions(-)
diff --git a/apps/s_apps.h b/apps/s_apps.h
index 3b9f9a0..9d16e45 100644
--- a/apps/s_apps.h
+++ b/apps/s_apps.h
@@ -148,7 +148,14 @@ typedef fd_mask fd_set;
#define PORT_STR4433
#define PROTOCOLtcp
-int do_server(int port, int type, int *ret, int (*cb) (char *hostname, int s,
int stype, unsigned char *context), unsigned char *context, int naccept);
+int do_server(int port, int type, int *ret,
+ int (*cb)(char *hostname, int s, int stype, unsigned char
*context),
+ unsigned char *context, int naccept);
+#ifndef NO_SYS_UN_H
+int do_server_unix(const char *path, int *ret,
+ int (*cb)(char *hostname, int s, int stype, unsigned char
*context),
+ unsigned char *context, int naccept);
+#endif
#ifdef HEADER_X509_H
int MS_CALLBACK verify_callback(int ok, X509_STORE_CTX *ctx);
#endif
@@ -162,6 +169,9 @@ int ssl_print_curves(BIO *out, SSL *s, int noshared);
#endif
int ssl_print_tmp_key(BIO *out, SSL *s);
int init_client(int *sock, const char *server, int port, int type);
+#ifndef NO_SYS_UN_H
+int init_client_unix(int *sock, const char *server);
+#endif
int should_retry(int i);
int extract_port(const char *str, short *port_ptr);
int extract_host_port(char *str,char **host_ptr,unsigned char *ip,short *p);
diff --git a/apps/s_client.c b/apps/s_client.c
index f8c059a..eee0e2e 100644
--- a/apps/s_client.c
+++ b/apps/s_client.c
@@ -323,7 +323,8 @@ static void sc_usage(void)
BIO_printf(bio_err,\n);
BIO_printf(bio_err, -host host - use -connect instead\n);
BIO_printf(bio_err, -port port - use -connect instead\n);
- BIO_printf(bio_err, -connect host:port - who to connect to (default is
%s:%s)\n,SSL_HOST_NAME,PORT_STR);
+ BIO_printf(bio_err, -connect host:port - connect over TCP/IP (default
is %s:%s)\n,SSL_HOST_NAME,PORT_STR);
+ BIO_printf(bio_err, -unix path- connect over unix domain
sockets\n);
BIO_printf(bio_err, -verify arg - turn on peer certificate
verification\n);
BIO_printf(bio_err, -cert arg - certificate file to use, PEM
format assumed\n);
BIO_printf(bio_err, -certform arg - certificate format (PEM or DER)
PEM default\n);
@@ -627,6 +628,7 @@ int MAIN(int argc, char **argv)
short port=PORT;
int full_log=1;
char *host=SSL_HOST_NAME;
+ const char *unix_path = NULL;
char *xmpphost = NULL;
char *cert_file=NULL,*key_file=NULL,*chain_file=NULL;
int cert_format = FORMAT_PEM, key_format = FORMAT_PEM;
@@ -760,6 +762,11 @@ static char *jpake_secret = NULL;
if (!extract_host_port(*(++argv),host,NULL,port))
goto bad;
}
+ else if (strcmp(*argv,-unix) == 0)
+ {
+ if (--argc 1) goto bad;
+ unix_path = *(++argv);
+ }
else if (strcmp(*argv,-xmpphost) == 0)
{
if (--argc 1) goto bad;
@@ -1155,6 +1162,11 @@ bad:
goto end;
}
+ if (unix_path (socket_type != SOCK_STREAM))
+ {
+ BIO_printf(bio_err, Can't use unix sockets and datagrams
together\n);
+ goto end;
+ }
#if !defined(OPENSSL_NO_JPAKE) !defined(OPENSSL_NO_PSK)
if (jpake_secret)
{
@@ -1499,7 +1511,8 @@ bad:
re_start
[PATCH] bignum: allow concurrent BN_MONT_CTX_set_locked()
The lazy-initialisation of BN_MONT_CTX was serialising all threads, as
noted by Daniel Sands and co at Sandia. This was to handle the case that
2 or more threads race to lazy-init the same context, but stunted all
scalability in the case where 2 or more threads are doing unrelated
things! We favour the latter case by punishing the former. The init work
gets done by each thread that finds the context to be uninitialised, and
we then lock the set logic after that work is done - the winning
thread's work gets used, the losing threads throw away what they've done.
Signed-off-by: Geoff Thorpe ge...@openssl.org
---
Hi Daniel,
This patch is relative to the current 'master' branch, but it applies
cleanly (git am) to the 1.0.2 and 1.0.1 branches too, so should
presumably be workable if you're using something similar (like a
patched source tree from some distro).
make test in openssl still passes after this, so I'm reasonably sure
it doesn't logically break anything, but as you have some scalability
use-case to hand I'd be grateful if you could confirm for me that it
improves the performance issue too.
Thanks,
Geoff
crypto/bn/bn_mont.c | 46 ++
1 file changed, 26 insertions(+), 20 deletions(-)
diff --git a/crypto/bn/bn_mont.c b/crypto/bn/bn_mont.c
index e6f6e3f..799763e 100644
--- a/crypto/bn/bn_mont.c
+++ b/crypto/bn/bn_mont.c
@@ -480,32 +480,38 @@ BN_MONT_CTX *BN_MONT_CTX_copy(BN_MONT_CTX *to,
BN_MONT_CTX *from)
BN_MONT_CTX *BN_MONT_CTX_set_locked(BN_MONT_CTX **pmont, int lock,
const BIGNUM *mod, BN_CTX *ctx)
{
- int got_write_lock = 0;
BN_MONT_CTX *ret;
CRYPTO_r_lock(lock);
- if (!*pmont)
+ ret = *pmont;
+ CRYPTO_r_unlock(lock);
+ if (ret)
+ return ret;
+
+ /* We don't want to serialise globally while doing our lazy-init math in
+* BN_MONT_CTX_set. That punishes threads that are doing independent
+* things. Instead, punish the case where more than one thread tries to
+* lazy-init the same 'pmont', by having each do the lazy-init math work
+* independently and only use the one from the thread that wins the race
+* (the losers throw away the work they've done). */
+ ret = BN_MONT_CTX_new();
+ if (!ret)
+ return NULL;
+ if (!BN_MONT_CTX_set(ret, mod, ctx))
{
- CRYPTO_r_unlock(lock);
- CRYPTO_w_lock(lock);
- got_write_lock = 1;
+ BN_MONT_CTX_free(ret);
+ return NULL;
+ }
- if (!*pmont)
- {
- ret = BN_MONT_CTX_new();
- if (ret !BN_MONT_CTX_set(ret, mod, ctx))
- BN_MONT_CTX_free(ret);
- else
- *pmont = ret;
- }
+ /* The locked compare-and-set, after the local work is done. */
+ CRYPTO_w_lock(lock);
+ if (*pmont)
+ {
+ BN_MONT_CTX_free(ret);
+ ret = *pmont;
}
-
- ret = *pmont;
-
- if (got_write_lock)
- CRYPTO_w_unlock(lock);
else
- CRYPTO_r_unlock(lock);
-
+ *pmont = ret;
+ CRYPTO_w_unlock(lock);
return ret;
}
--
1.9.1
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majord...@openssl.org
[openssl.org #2608] bug report: segfault from base64 decoding
David (thanks!) re-raised this ticket with the following details. (I am also attaching his 'base64-bug.txt' file.) Our UC-KLEE tool found a buffer overflow in OpenSSL's base64 decoder that results in a negative length passed to memcpy. This is likely exploitable, depending on where PEM-encoded keys/certs are originating. Someone else reported the same bug in 2011 and it was never addressed (sensing a pattern?): https://rt.openssl.org/Ticket/Display.html?id=2608user=guestpass=guest The attacker can control the negative value, but only in the range -1 to -16, so it's likely to segfault and trigger a DoS, but I suppose there's a small chance of more serious consequences. Attached is a sample attack input that yields a length of -16: $ openssl base64 -d -in base64-bug.txt Backtrace: frame #1: 0x93550e38 libsystem_c.dylib`__memcpy_chk + 37 frame #2: 0x001c5017 openssl`b64_read(b=0x00489700, out=0x00b2cc00, outl=8192) + 1975 at bio_b64.c:357 frame #3: 0x0019f072 openssl`BIO_read(b=0x00489700, out=0x00b2cc00, outl=8192) + 434 at bio_lib.c:212 frame #4: 0x000134d0 openssl`enc_main(argc=0, argv=0xba8c) + 9536 at enc.c:662 frame #5: 0x2e02 openssl`do_cmd(prog=0x00488e00, argc=4, argv=0xba7c) + 274 at openssl.c:490 frame #6: 0x25a4 openssl`main(Argc=4, Argv=0xba7c) + 1316 at openssl.c:382 -
[openssl.org #2608] bug report: segfault from base64 decoding
I'm attaching a patch that I think is the right fix, but would appreciate feedback from people who understand evp/b64 better than me. It passes make test, and ceases to dump-core with David's sample input (also attached to this ticket). BTW, this patch seems to apply to all branches as far back as I've looked (0.9.7-stable), so presumably the problem is as old as that too. 0001-evp-prevent-underflow-in-base64-decoding.patch Description: Binary data
[openssl.org #695] [PATCH] DSO: dlfcn support for MacOS X
I contacted the original submitter and he said there is unlikely to be any value in this line of enquiry now. He asked me to close the ticket and I'm happy to oblige. -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH] Fix RSA blinding locking hole
On Thursday 02 April 2009 04:44:55 Marc Haisenko wrote: Just a little load thinking... my patch *seems* to fix the blinding issue. All the patch does is to make sure that once the thread/process id's are different locking is *always* done and rsa-mt_blinding is always used. But since locking seems to be a NOP in our case this may not fix our problem. Yet it seems to as we have no more Bad Record MACs since I have introduced the patch. I don't get why... Well the fact that rsa-blinding and rsa-mt_blinding are both initialised by *locked* lazy-initialisation, could explain why you're having grief when the locking is NOP'd out? Anyway, I'll do the locking implementation in OpenSER and report back in a few days. Good luck. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH] Fix RSA blinding locking hole
On Wednesday 01 April 2009 04:48:01 Marc Haisenko wrote: Forgive my frustration, but which god do I need to sacrifice to to get some attention ? Even a go away, we don't care would be OK... I have your email tagged for attention, but haven't had a good enough moment to look at it. But please go ahead and sacrifice any god that pleases you, it'll give you something to do while I try to take a look at this, and it will no doubt amuse any passers-by. :-) Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH] Fix RSA blinding locking hole
OK, I've taken a look at this, and scratched my head a little. It's a
touch complicated by the fact that thread-ids have changed in the head
of development relative to what you're looking at in 0.9.8. But I'm now
wondering if you haven't misunderstood the nature of openssl's threading
support;
On Tuesday 31 March 2009 04:52:19 Marc Haisenko wrote:
Hi folks,
last week I described a problem with RSA blinding and its locking[1].
Coincidentally another user ran into the same problem in a totally
different scenario, the work-around of disabling the blinding fixed
the issue for him[2].
So here's a patch to fix this issue.
The issue is that the rsa_get_blinding method checks whether there is
concurrent access to the blinding and tells its caller whether locking
needs to be done. The check is done by comparing thread/process IDs.
If they differ locking needs to be done. The problem is that locking
is not done if the IDs are same. It is assumed that the creator
doesn't need to do locking which is correct for a single-threaded
application but is wrong in a multi- threaded/processed environment
once another process has accessed the blinding.
We need to separate multi-threading from multi-processing here. Once
you've forked, there should be no locking relationship whatsoever
between processes, irrespective of any threads (and thread-ids) within
those processes. Do you have some reason to believe this isn't the case?
The fork has essentially duplicated the process and created isolation
between the two (copy-on-write or otherwise). So our only concern
*should* be between threads within the same process.
So ... multi-threading. My understanding of the RSA blinding (which I
didn't write) is that rsa-blinding is lazy-initialised by whichever
thread gets to it first. The call to RSA_setup_blinding() creates the
blinding struct and tags it with the current thread-id. Subsequent
accesses to RSA blinding will use rsa-blinding if and only if the
caller is the thread that lazy-initialised it (ipso facto, if processing
of a given RSA struct is always in one thread - this is optimal).
Accesses to RSA blinding from other threads should use rsa-mt_blinding
(which is lazy-initialised by the first such thread) and this places
obligations on the caller - ie. do your own locking, don't use a common
blinding factor, etc.
This should work. I'm wondering if I understood what you meant by The
problem is that locking is not done if the IDs are same. ... If the IDs
are the same, that means you're in the same thread, period!! If that's
not the case, then the problem is that your thread-id callback isn't set
up correctly. For threading support to work correctly, you need to
provide hooks in order to make openssl compatible with your thread model
(pthreads or otherwise). For 0.9.8, the APIs include;
void CRYPTO_set_locking_callback(void (*func)(int mode,int type,
const char *file,int line));
void CRYPTO_set_id_callback(unsigned long (*func)(void));
Locking callbacks default to nops, and id callbacks default to different
things depending on platform, but one possibility is errno (ie. the
address of the errno var, in the hopes that it's thread-local) and
another is getpid() (ie. in the hopes that the threads have distinct
pids too). It may be that the default behaviour is degenerating on your
platform and not correctly differentiating between threads.
As a fix I've introduced a new flag for struct rsa_st,
RSA_FLAG_BLINDING_NEEDS_LOCKING. Once it has been noticed that locking
has to be done the flag is set and from then on we always do locking,
whether we are the creator or not.
However, I'm scratching my head why this fixes the issue given the
fact that there are actually two blindings: one when doing no locking
(rsa-blinding) and another one when doing locking (rsa-mt_blinding).
Yet our load test indicates that this really does fix the issue:
without it we get about a dozen Bad Record MAC per day due to the
blinding. Since I've implemented the attached patch none has occured
(which is about 1.5 days of load test now).
As before, the thread detection ('id_callback') might be failing you such
that all threads appear to be the same thread? Try calling
CRYPTO_thread_id() from different threads in the same process and see
whether they return the same value. This might explain why you fixed
the problem by locking all callers even when they all have the same
thread ID...
Cheers,
Geoff
--
Un terrien, c'est un singe avec des clefs de char...
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majord...@openssl.org
Re: OpenSSL v1.0 and API/ABI compatibily.
Fair comment, I'll respond to this as best I can, but this is not any kind of official statement. On Wednesday 01 April 2009 14:01:18 Kurt Roeckx wrote: Hi, I was under the impression that for the 1.0 version you would change the API so that the ABI doesn't break all the time, and I see no changed related to that. No, and please don't expect it to happen for v1.0. I speak only for myself, the others can agree/disagree as they please, but I think ABI compatibility is a fairly unrealistic dream when you take into account the SSLeay and OpenSSL legacy interfaces, current release management mechanisms, and community expectations (specifically souce compatibility, and in the case of stable-releases, even binary/shared-lib compatibility). The problem is circular - to get to ABI compatibility (and other desirable but far-away goals) would first require a large number of significant breaks in API compatibility. Resistance to the latter has historically been more evident, and certainly more immediate, than support for the former. However, we have been mulling over a new way of managing versions that would allow openssl to deliberately engage in API breakages across major releases (in ways that can be tracked, and which allow dependent libs/apps to prepare, etc), and to hopefully combine this with a more orderly/predictable release time-frame. The idea then is that working towards goals like ABI-stability would be possible in iterations that explicitly break certain legacy interfaces and behaviours, but do so in a clearly-defined manner. There would also be appropriate assurances for continued support of prior versions + adequate lead times to prepare for interface changes prior to major releases. The whole point is that there are types of progress we can't make by purely incremental and backwards-compatible steps, so this would be an attempt to address that without raising the ire of disgruntled users. But as for 1.0, the real issue there is that it has been baking for far too long, it is now neary 4 years since the last stable branch was cut (0.9.8) and the accumulated changes in the development head are overdue for a stable release branch of their own. In fact, the changes are quite wide-ranging even if they're mostly under the covers, so v1.0 it will be. I believe/hope we will follow that up with improved process for subsequent releases in order to make more meaningful progress on the sort of significant issues/goals you hint at. The 0.9s have been incubating for ages now with a fairly strong maintenance focus, so even if the 1.0 release is not unlike another 0.9 release, the hope is that it marks a change towards an iterative process more focussed on progress. At least that's my take on things. :-) Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: Exportable C functions, not macros.
On Tuesday 24 March 2009 09:45:20 Maxim Masiutin wrote: Hello All, We have a BN_GENCB structure BN_GENCB_set_*** macros to set fields. Problem is that when compiling OpenSSL as a library and using it from a different application written in different language, converting structures and unions is problematic due to different data sizes and structure alignment issues. Some parts of the OpenSSL already have exportable functions to operate with structures, e.g. EC_KEY_new, EC_KEY_set_group, EC_KEY_set_private_key, EC_GROUP_get_point_conversion_form, EC_KEY_free, and so on. In the past, all SSLeay/LIBeay was heavily relying on macros. I wrote to Eric Young about this issue. He (and later other developers) did converted most of the macros to exportable functions. Can we also convert BN_GENCB_set_*** macros to functions? Thank you in advance. I'm assuming here that converting the macros to inlines is also not going to help? If so, it seems you'll have problems in lots of places besides BN_GENCB, as lots of structures are exposed. BN_GENCB is not something we want to turn into an API(tm) as such ... If you are trying to thunk between languages, why don't you implement the missing protections/wrappers in your language glue? You'll presumably need similar wrapper functions for other structure accessors. Moreover, if the user behind your language glue wants to set a callback in the BN_GENCB struct, you presumably have to insert your own trampolining to handle that? If so, using your own wrappers will allow you to extend the C-native BN_GENCB struct with any extra state you need to bridge the callback into your particular language domain? In any case, this is not stuff that affects C users, so I don't think it (alone) justifies clamping down the C interface itself. Please let me know if I overlooked other issues. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH RFC -v2] Add support to Intel AES-NI instruction set for x86_64 platform
On Tuesday 23 December 2008 02:01:38 Huang Ying wrote: This patch adds support to Intel AES-NI instruction set for x86_64 platform. Cool. I'm relying on Andy to provide a more thorough review than my quick scan - I don't do perl-asm :-) In particular, I haven't tried patching and building this. (Andy, let me know if you need any off-platform testing - I presume not, but ...) Quick comment: Signed-off-by: Huang Ying ying.hu...@intel.com --- crypto/engine/Makefile | 11 crypto/engine/eng_all.c|3 crypto/engine/eng_intel.c | 589 ++ crypto/engine/eng_intel_asm.pl | 918 + 4 files changed, 1519 insertions(+), 2 deletions(-) Are you using git to prepare this patch, and if so, which git repo+branch are you tracking? +#define INTEL_AES_MIN_ALIGN 16 +#define ALIGN(x,a) (((unsigned long)(x)+(a)-1)(~((a)-1))) +#define INTEL_AES_ALIGN(x) ALIGN(x,INTEL_AES_MIN_ALIGN) You don't seem to need the ALIGN() macro anywhere, just INTEL_AES_ALIGN(), so I'd personally prefer it if you didn't use ALIGN as this is tempting fate with respect to possible symbol conflicts. Also, if you have no philosophical objection, I think the file and symbol naming should be based on the interface rather than the manufacturer (particularly for intel, who provide lots of h/w and interfaces that have nothing to do with AES-NI). Perhaps eng_aesni.c rather than eng_intel.c. If it's absolutely certain no other manufacturer will support the same instructions in the future, we could live with eng_intel_aesni.c, but it still needs to be clear that the engine targets the AES-NI interface rather than (any) intel interface. (I don't want to handle support questions from x86 noobs who presume the eng_intel.c engine accelerates any intel cpu ...) As your use of INTEL_AES_ALIGN() was always to cast it to a pointer, please also rephrase the macro to not need casting every time; #define AESNI_MIN_ALIGN 16 #define AESNI_ALIGN(x) \ (void *)(((unsigned long)(x) + AESNI_MIN_ALIGN - 1) \ (~(unsigned long)(AESNI_MIN_ALIGN - 1))) Finally - did you omit a patch to engine.h? Your changes to eng_all.c include a call to ENGINE_load_intel_aes_ni(), which is in eng_intel.c, but this doen't appear to be declared in any header. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH RFC -v2] Add support to Intel AES-NI instruction set for x86_64 platform
On Tuesday 23 December 2008 11:58:52 Andy Polyakov wrote: Out of curiosity, what does NI stand for anyway? Or is it just something the knights kept saying? But didn't they stop doing so? ROFL :-) Next up, eng_shrubbery.c ... Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [CVS] OpenSSL: openssl/crypto/ crypto-lib.com install.com openssl/engi...
On Tuesday 16 December 2008 05:54:28 Richard Levitte wrote: OpenSSL CVS Repository http://cvs.openssl.org/ __ __ Server: cvs.openssl.org Name: Richard Levitte He's bck! :-) How's things? Merry christmas, happy new year, and all that. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [CVS] OpenSSL: openssl/crypto/ crypto-lib.com install.com openssl/engi...
Oops, chalk up another victim to reply-to ... sorry for the noise (and the follow-up noise of this apology). Cheers, Geoff On Tuesday 16 December 2008 14:52:34 Geoff Thorpe wrote: On Tuesday 16 December 2008 05:54:28 Richard Levitte wrote: OpenSSL CVS Repository http://cvs.openssl.org/ __ __ Server: cvs.openssl.org Name: Richard Levitte He's bck! :-) How's things? Merry christmas, happy new year, and all that. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH RFC] Add support to Intel AES-NI instruction set for x86_64 platform
On Thursday 11 December 2008 05:04:36 Peter Waltenberg wrote: Anything in memory could end up swapped out, but stack is the least likely since it's more often in use, the best you can do is zero the area ASAP. My other objection to putting all of this into an engine is that the engine code is unusable in quite a few cases. Export approvals, and certifications like FIPS and Common Criteria all pretty much insist that the crypto. isn't replaceable by some random chunk of code, that's not an OpenSSL issue as such, but it's going to be awkward for some subset of OpenSSL consumers. There are ways around those issues, but I doubt you really want to add the option of signature checking engine plugins ?. Engines like eng_cryptodev.c *are* built in (they're in ./crypto/engine/ rather ./engines/) and the intention is that they should be the implementation de base for those build targets to which they apply. Cryptodev is the only one so far, but there could be others. In fact, the padlock support for VIA chips (which is comparable to what's being discussed here, with all due respect to the intel instruction-set faithful) sits in ./engines like any other h/w support - a similar argument could be made that, on chips that support it, it should provide the default implementation(s), but right now they've been happy enough to make it a non-default option. In any case, the solution is not to say nah, that's not quite what I need - I'd rather patch into the default s/w implementation instead and disable engine support. The default s/w implementation is a legacy notion maintained for backward compatibility (in an ideal world, that would be an engine too, in fact it would be multiple engines - asm, no-asm, [etc]...) The issues of signatures and dynamic loading are questions of build configuration, which distinguish between what is (a) built in to the libcrypto image and registered/enabled by ENGINE_load_builtin_engines(), and (b) possibly external to libcrypto and must be explicitly enabled/loaded. If you have issues with those semantics (and I know of a few, but fixing and simplifying them will require deprecating the notion of s/w fallback), then patches to crypto/engine/ are welcome (as they would be for ./Configure). Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH RFC] Add support to Intel AES-NI instruction set for?x86_64 platform
On Thursday 11 December 2008 10:52:36 Thor Lancelot Simon wrote: On Thu, Dec 11, 2008 at 10:03:32AM -0500, Geoff Thorpe wrote: Engines like eng_cryptodev.c *are* built in (they're in ./crypto/engine/ rather ./engines/) and the intention is that they should be the implementation de base for those build targets to which they apply. I'm surprised this can be certified for FIPS. Are you sure it is the case for the FIPS module? Consider that eng_cryptodev will in many cases end up using unknown -- and thus presumptively unvalidated -- hardware implementations of most of the core algorithms, in some cases even software implementations in the kernel. I would be surprised that the test lab would allow 'hooks' like this in the FIPS module. I'm not saying they did, do, would, nor should. But your point about unvalidated hardware implementations would equally cover the alternative intel code-path, just as it would(/might/could/has) cover(ed) the via-padlock code-path. The engine issue is about how the C code is laid out, not about what was, is, or should be validated, nor is it *necessarily* about what is static versus dynamic. My point is that there's zero technical argument for bypassing a mechanism whose sole purpose is to organise algorithm implementations, so that one can instead directly hack/patch a pre-existing implementation. If the engine mechanism needs work to meet or beat requirements, that's where the effort should go. Andy can speak for himself, but I *think* that was his point too. Caveat: I haven't been involved in FIPS at all BTW, so I find myself in the couldn't care less category regarding the details. But if there's something that needs to be locked down (ie. fixed in the C code), we can surely lock it down as a part *of* the engine code rather than locking out engine code altogether. If this isn't the case then long-term, FIPS work will diverge significantly from (a) ongoing development, and (b) common sense. Side-note: I've heard people who *are* involved in the FIPS work speak of it in fairly colourful ways - I suspect they'd agree that the standard (and its process) is little more than bureaucratic, superstitious B-S. IANAL, FWIW, etc. :-) Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH RFC] Add support to Intel AES-NI instruction set for x86_64 platform
On Thursday 11 December 2008 20:39:41 Huang Ying wrote: On Thu, 2008-12-11 at 23:03 +0800, Geoff Thorpe wrote: Engines like eng_cryptodev.c *are* built in (they're in ./crypto/engine/ rather ./engines/) and the intention is that they should be the implementation de base for those build targets to which they apply. Cryptodev is the only one so far, but there could be others. In fact, the padlock support for VIA chips (which is comparable to what's being discussed here, with all due respect to the intel instruction-set faithful) sits in ./engines like any other h/w support - a similar argument could be made that, on chips that support it, it should provide the default implementation(s), but right now they've been happy enough to make it a non-default option. The difference between Intel AES-NI and padlock is that padlock provide support for different modes directly including ecb, cbc, cfb and ofb, while Intel AES-NI just provides instructions for AES core block algorithm NOT for modes directly. At the same time, AES-NI pipelining implementation can benefit ecb encrypt and cbc decrypt and counter mode. If we implement AES-NI with branch, we can get full power of AES-NI except ecb and ctr mode. If we implement AES-NI with engine, we can get full power of AES-NI for all modes. But we must duplicate mode implementations that can not benefit from AES-NI, such as cfb, ofb, etc. Do you OK with code duplication? The cipher and digest support is at the granularity of nids, and these combine algorithm, key-length, and mode. So if you implement support for those cipher,length,mode combinations that can be accelerated by AES-NI, your engine will only be invoked for those combinations. You're not obliged to implement anything else, and indeed there is nothing to be gained by doing so. Please look at the padlock engine implementation, particularly the use of the DECLARE_AES_EVP macro. I humbly suggest this is a better way to go. However I see no reason why it couldn't go into crypto/engine/ (like eng_cryptodev.c) rather than in engines/ (like e_padlock.c) so that it is compiled in and loaded by default. Also, the load function for your engine could perform the run-time check for processor type, and only register its implementations with the engine infrastructure if the processor supports the AES-NI extensions (similar to padlock_available() for example). It would then suffice to add the -DHAVE_INTEL_AES_NI define to applicable build targets in Configure (presumably x86_64-related targets). Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH RFC] Add support to Intel AES-NI instruction set for x86_64 platform
On Thursday 11 December 2008 23:02:12 Huang Ying wrote: On Fri, 2008-12-12 at 11:38 +0800, Geoff Thorpe wrote: The cipher and digest support is at the granularity of nids, and these combine algorithm, key-length, and mode. So if you implement support for those cipher,length,mode combinations that can be accelerated by AES-NI, your engine will only be invoked for those combinations. You're not obliged to implement anything else, and indeed there is nothing to be gained by doing so. The situation is: - We implement cbc and ecb mode in engine - If we implement cfb and ofb in engine too, we will duplicate code of cfb and ofb mode itself. - If we do not implement cfb and ofb in engine, no code duplication, BUT we can NOT get AES-NI acceleration for AES core block algorithm (which benefit cfb and ofb too) until we have a branch version. OK, I (mis)understood from your original mail that you could only accelerate a subset of modes. If you can accelerate them all, then please do so by implementing an intel/aes-ni engine. But not by branching in the vanilla implementation. So my suggestion is: - Accelerate AES core block algorithm with branch version. Which is used by cbc, cfb and ofb too. - Accelerate AES ecb and ctr? with engine version. And my suggestion is: - write an engine for your hardware. It is my intention to eventually engineify the default s/w implementations anyway - their static existence is a legacy thing that will get deprecated eventually (that way, if you don't want them and don't load them, or only load a subset of them, your linker and run-time footprint will shrink). At that point the whole issue of duplication is moot because someone will use the vanilla implementation *OR* your AES-NI engine, but not both. Even for now, the duplication issue is a red herring (especially when weighed against the inelegant mess it'll make of the code). So please, just write an engine and see how that goes. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [PATCH RFC] Add support to Intel AES-NI instruction set for x86_64 platform
I doubt the OS vendors would bother to enable an engine by default, testing of the possible configurations is expensive and the costs of support calls if they mess up makes autodetecting the engine to use a very unattractive proposition. One can discuss loading selected engines by default, i.e. you'd have to work to not load it:-) Then it wouldn't be any different, yet provide I am new to OpenSSL. Can you tell me how to do that? how to use the proper engine automatically? I seem to recall that the cryptodev engine (for use on *BSDs) is loaded by default if HAVE_CRYPTODEV is defined, and if so, the load function will bind the engine at run time if /dev/crypto is alive and well. This means it'll get used by default for those algorithms/modes it supports. Isn't this precisely what you'd want to do for processor-specific enhancements? Enable compilation on platforms that might have your processor by setting the corresponding -Dfoo in Configure, and then have your load function bind the engine only if a run-time check shows you're running on a compatible chip. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH] keep CHIL engine from breaking in non-threaded situations [openssl.org #1736]
On Tuesday 18 November 2008 19:22:55 Sander Temme wrote: On Nov 11, 2008, at 12:28 AM, Sander Temme wrote: On Nov 2, 2008, at 9:19 AM, Sander Temme wrote: The following compiles and tests cleanly on trunk and on the latest snapshot of 0.9.8-stable (with offset 1 in e_chil.c). Also attached to prevent line wrap: Ping? Any chance to squeeze this simple, six line patch, that doesn't touch but a single source file, in before 0.9.8j is tagged? Do we need more than Geoff Thorpe's ACK from 10/27/08? It's in now, please verify the next nightly snapshot. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH] keep CHIL engine from breaking in non-threaded situations [openssl.org #1736]
On Tuesday 28 October 2008 02:07:18 Sander Temme wrote: On Oct 27, 2008, at 8:20 AM, Geoff Thorpe wrote: Well, FWIW, I will ACK the patch. And if nobody gets round to putting it in (and doing any other admin that goes with it, like regenerating the error definitions which probably change as a consequence of this, etc), then I'll try to do this myself this some time this week. Thanks Geoff. I ran perl util/mkerror.pl and it didn't come up with any new ones. Should it do anything since I'm only taking an occurrence of an error away? If you want to help edge this forward in the mean-time - try patching the latest CVS snapshots of HEAD and/or 0.9.8-stable, and preferably after running make update so that your patch(es) require(s) less man-handling. TIA. Same patch, applies clean to trunk and to 0.9.8-stable. I actually took it against trunk. Doesn't seem to be touched by make update, unless I'm missing something. It didn't remove the error string? Can you please try removing it manually and verifying everything builds ok? AFAICT, you are orphaning an error string that no longer needs to take up space in the code or compiled image. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH] keep CHIL engine from breaking in non-threaded situations [openssl.org #1736]
Well, FWIW, I will ACK the patch. And if nobody gets round to putting it in (and doing any other admin that goes with it, like regenerating the error definitions which probably change as a consequence of this, etc), then I'll try to do this myself this some time this week. If you want to help edge this forward in the mean-time - try patching the latest CVS snapshots of HEAD and/or 0.9.8-stable, and preferably after running make update so that your patch(es) require(s) less man-handling. TIA. Cheers, Geoff On Monday 27 October 2008 11:11:29 Sander Temme wrote: On Oct 20, 2008, at 11:21 AM, Sander Temme via RT wrote: Dear OpenSSL developers, I would like to propose the following patch to engines/e_chil.c: -- Polite nag -- I have been trying to raise this topic, on the list and through RT, since August. I have had no reaction from any of the folks I see commit to OpenSSL CVS. [snip] -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: DSO_load using DSO_method_dlfcn
Which version of openssl are you using? Cheers, Geoff On Monday 13 October 2008 16:08:53 Pirasenna Velandai Thiyagarajan wrote: How to load a DSO from within an engine? I call DSO_load(NULL, mylibname, NULL, 0); The code path I debugged is as follows: #0 DSO_ctrl (dso=0x48ab98, cmd=2, larg=0, parg=0x0) at dso_lib.c:338 #1 0x2abb9c90 in DSO_load (dso=0x0, filename=0x2ae00888 mylibname, meth=0x0, flags=0) at dso_lib.c:209 [snip] -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: DSO_load using DSO_method_dlfcn
On Monday 13 October 2008 18:05:17 David Woodhouse wrote: On Mon, 2008-10-13 at 13:08 -0700, Pirasenna Velandai Thiyagarajan wrote: How to load a DSO from within an engine? See the code that this patch is mostly ripping out in favour of direct linking: http://git.infradead.org/users/dwmw2/openssl-tpm-engine.git?a=commitdi ff;h=624a38a1 I suspect the original poster's issue is the bind_engine() logic in engine.h, or rather something missing from it (maybe DSO, but I'm not sure). That could be total rubbish too, obviously. Which leads to me wondering what your patch is about ... What's the relation between this patch and the original poster's question? I suspect that I haven't fully understood what your patch is trying to resolve. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: help regarding random numbers in openssl
I see that you've cross-posted to both lists a few times, please don't. Most of your posts (if not all) belong on openssl-users. openssl-dev is for discussing the development of openssl itself, whereas openssl-users is for discussing development *using* openssl (or anything else related to openssl). Thanks, Geoff On Tuesday 07 October 2008 11:47:42 prashanth s joshi wrote: Hi all, In openssl code which part actually handles catching of the random numbers exchanged during the handshake? Regards, Prashanth.. -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Fix VIA Padlock RNG support ?
Hi again, On Thursday 11 September 2008 09:32:14 Geoff Thorpe wrote: On Thursday 11 September 2008 09:06:39 Harald Welte wrote: On Thu, Sep 11, 2008 at 10:22:38PM +1200, Michal Ludvig wrote: Have a look here: http://marc.info/?l=openssl-devm=109113625526391w=2 and in the corresponding thread for the discussion. FWIW even in the Linux kernel the hardware RNGs (incl. VIA PadLock) are not used directly for RNG output but instead to feed the entropy pool. I quite agree with the reasons why OpenSSL shouldn't use whatever it gets from PadLock RNG *directly* as a stream of random numbers. Unfortunately as soon as PadLock engine registers as a RNG provider OpenSSL won't do any post-processing of the random data and therefore the best bet at that time was not to use PadLock's RNG. Yes, after reviewing the discussion and documentation I tend to agree. So the best option really is to make OpenSSL use the userspace interface for the kernel random number generator, and feed that kernel RNG's entropy pool from the hardware RNG. O, right, I see know. Yes this is a bit crap. The problem IMHO is that RAND_METHOD is the wholesale replacement interface. Ie. the entire software PRNG sits behind that interface, no matter how it obtains its entropy, and using an alternative RAND_METHOD will completely bypass the software PRNG. This makes sense for the other ***_METHOD interfaces, because they're alternatives to the s/w implementations - they can be used simultaneously, but work on completely separated contexts. In the case of the PRNG, there are no separated contexts nor point to having s/w and replacement PRNGs working independently. I think the sensible thing would be to stick with the s/w PRNG implementation and have it expose an inner interface for its entropy sources and let engines plug themselves into *that*, rather than acting as alternatives/replacements for the entire PRNG. Looking at this in more detail, the current s/w PRNG implementation keeps a running 'entropy' count and when that reaches a certain threshold, it stops maintaining an entropy counter because the PRNG is considered sufficiently seeded. Each platform (roughly speaking) has its own implementation of RAND_poll() which does some canonical seeding, which may be enough to get the PRNG off the ground, or if not, the application will need to RAND_add() (or RAND_seed()) some more entropy before the PRNG is ready. In any case, this doesn't adapt so well to a model where entropy sources live as callbacks and get called by the PRNG when required. It's more a model where an entropy source should just stuff its entropy into the PRNG as soon as it gets a chance, and preferably as much of the stuff as it has handy. It can always add more later and no harm will be done, but there's no obvious way to add a hook to ask for entropy automatically. With this in mind, I'm wondering if the simplest thing to do isn't just to have the padlock (or any other) engine call RAND_add() with some entropy during the init() handler of the ENGINE itself (rather than in a RAND_METHOD). That doesn't solve the problem of adding more entropy as time goes by, but it's better than the current situation (only having a RAND_METHOD mechanism you can't use at all), and moreover it requires no interface changes, just implementation... Thoughts? Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OPenssl 0.9.8j dev
Please refrain from cross-posting to all the openssl lists. In particular, please don't post to openssl-cvs, that's not what it's for (the Reply-To for cvs mailouts is intentionally set to openssl-dev). Thanks, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Fix VIA Padlock RNG support ?
On Thursday 11 September 2008 09:06:39 Harald Welte wrote: On Thu, Sep 11, 2008 at 10:22:38PM +1200, Michal Ludvig wrote: And finally the one you already knew about. That's the final works for me version ready to be committed to openssl tree current at that time (may not apply smoothly anymore, I don't know): http://marc.info/?l=openssl-devm=115243758508970w=4 Ok, let me dig that out, merge it with current ssl version of openssl, give it some testing and resubmit it to the mailinglist for inclusion. Thanks, let me know how you get on. I don't think there's any taboo or a strong opposition against the patch. It's just that Andy hasn't followed up, I sort of given up and moved to other projects and the whole thing has gone forgotten. Ok. I hope after my re-merge and testing we can get it integrated this time. BTW, my memory is vague here, is this Padlock block only able to do one-shot hashing? Have a look here: http://marc.info/?l=openssl-devm=109113625526391w=2 and in the corresponding thread for the discussion. FWIW even in the Linux kernel the hardware RNGs (incl. VIA PadLock) are not used directly for RNG output but instead to feed the entropy pool. I quite agree with the reasons why OpenSSL shouldn't use whatever it gets from PadLock RNG *directly* as a stream of random numbers. Unfortunately as soon as PadLock engine registers as a RNG provider OpenSSL won't do any post-processing of the random data and therefore the best bet at that time was not to use PadLock's RNG. Yes, after reviewing the discussion and documentation I tend to agree. So the best option really is to make OpenSSL use the userspace interface for the kernel random number generator, and feed that kernel RNG's entropy pool from the hardware RNG. O, right, I see know. Yes this is a bit crap. The problem IMHO is that RAND_METHOD is the wholesale replacement interface. Ie. the entire software PRNG sits behind that interface, no matter how it obtains its entropy, and using an alternative RAND_METHOD will completely bypass the software PRNG. This makes sense for the other ***_METHOD interfaces, because they're alternatives to the s/w implementations - they can be used simultaneously, but work on completely separated contexts. In the case of the PRNG, there are no separated contexts nor point to having s/w and replacement PRNGs working independently. I think the sensible thing would be to stick with the s/w PRNG implementation and have it expose an inner interface for its entropy sources and let engines plug themselves into *that*, rather than acting as alternatives/replacements for the entire PRNG. I'll submit a patch to OpenSSL which gives a more detailed description in the comment since I think it is sort of like a FAQ for those people who actually discover the padlocn no-RNG flag :) I will see if I can sketch an ENTROPY_METHOD that would improve this situation. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: VIA Padlock Hashing Engine [Was: Fix VIA Padlock RNG support ?]
On Thursday 11 September 2008 15:16:48 Andy Polyakov wrote: BTW, my memory is vague here, is this Padlock block only able to do one-shot hashing? Yes, but a technique bypassing this limitation was proposed and proven to work (as per end of SHA1 thread mentioned earlier). Technique involved crashing of hashing instruction into non-accessible page. And that was what I wanted to pursue, but never got time to. Which is why there was no real follow-up:-( For reference, the plan was to setup intermediate buffer followed by non-accessible page upon engine setup, i.e. once, and then serialize access to it with thread synchronizing primitives. I reckon that serializing threads is OK, because system is more likely to starve for data than for hashing compute power (1Gbps NIC vs. ~2Gbps hashing rate). A. Strangely enough, I've got uncommitted code to add support for thread-local-storage ... :-) (It's one of the components in the work I'm doing for async-crypto.) So if you take the approach you suggesting above, it should be relatively simple to generalise it later, to avoid locking/serialisation. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Fix VIA Padlock RNG support ?
* Harald Welte ([EMAIL PROTECTED]) wrote: Hi guys, ist has been 10 days since I posted this mail about certain questions with regard to the suboptimal integration of VIA padlock support in OpenSSL. Is there some kind of taboo against this topic or some bad history that I'm missing? If yes, I'm sorry to hear that. In any case, I am here, I have time, and I will do whatever it takes to the code to make you guys happy with it for integration. So please talk to me ;) Hi Harald, No taboo, no bad history that I'm aware of, just plain old open-source, everyone's-always-got-something-else-less-free-to-do indifference. Don't take it personally :-) I just took a look at Michal's SHA patch and nothing lept out as overly terrifying. Perhaps Michal will comment if he's aware of any discussion about it? (I don't recall.) Otherwise did you happen to search the request tracker or mail archives about this? (Ie. beyond the fact that Michal's post didn't have a threaded response.) As for the RNG stuff, if you are able to find any references to discussion and/or cvs commits regarding the deactivation by OpenSSL maintainers then that would make it easier for me to follow up too. TIA. Cheers, Geoff Thanks again. On Mon, Sep 01, 2008 at 09:51:30PM +0800, Harald Welte wrote: Hi Michal, Hi OpenSSL developers, as part of my work for VIA, I am trying to find out what we can do to make sure the VIA Padlock RNG is activated by default. I have read the comments in the source code, referring that the RNG is not used the way that VIA recommends for secure applications. I have also read the padlock programming guides from http://linux.via.com.tw/support/beginDownload.action?eleid=181fid=261 and http://linux.via.com.tw/support/beginDownload.action?eleid=181fid=281 So from what I can tell, Michal Ludvig originally included RNG support in his patch, but it was deactivated by the OpenSSL maintainers due to security concerns. Can somebody please indicate what exactly those concerns were? I would be willing to put in some of my own time to come up with a patch to address the concerns, and then have that patch reviewed by OpenSSL guys, Michal as well as the respective Padlock security expert inside VIA. I also have a question about Michal's SHA1/224/256 patch at http://marc.info/?l=openssl-devm=115243758508970w=2 It never received any feedback on the list, and it wasn't merged into mainline OpenSSL. Was this simply lost? Can I (or VIA) do anything to help this along? Thanks in advance, -- - Harald Welte [EMAIL PROTECTED] http://laforge.gnumonks.org/ Privacy in residential applications is a desirable marketing option. (ETSI EN 300 175-7 Ch. A6) __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH] PURIFY and valgrind
On Friday 18 July 2008 10:57:50 Bodo Moeller wrote: On Thu, Jul 17, 2008 at 7:07 PM, Frederic Heem [EMAIL PROTECTED] wrote: Please find attached a patch which makes valgrind and friends happy. Some changes had been done in md_rand.c which broke the purpose of PURIFY. Needless to say that the define PURIFY is *not* for production system... Defining PURIFY should never make the PRNG weak. If Valgrind finds data that is used uninitialized, then a PURIFY patch should only ensure that those exact bytes of data are initialized with some data. Never overwrite a byte that actually may have been initialized. Agreed, though where possible it's preferable for PURIFY-handling to simply not use the uninitialised data at all, rather than initialising it before use. (NB, I know this yields the same quality result, but appearances in the code are often as important as the outcome of the executable - particularly where package maintainers are concerned...) IIRC, it's complicated to remove the (uninitialised) loads here without also removing the desired stores, in which case I totally agree that memset(,0,) is a more obvious PURIFY(/valgrind) workaround. But where it's possible to trim our use of an input buffer so that we don't take some uninitialised trimmings, then the intent of the code would be much clearer. Moreover, it's more likely valgrind would then find bugs in the PRNG code (if ever we accidently debianise it ourselves) - whereas wholesale memset()s kind of limit the effectiveness of such debugging tools. Or put another way, -1 to the suggested patch. Also, -1 to the general statement that PURIFY is *not* for production system. That is not our call to make! If the production system is experiencing problems and is being diagnosed - it may very well make sense to run it under PURIFY/valgrind to help track down problems that have only been observed in production. (Anyone who has ever tried to gaze into a crystal ball and invent a test-case to reproduce sporadic problems from a production system will know what I'm talking about). Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: CRYPTO_THREADID_* code in 0.9.9: come gone again? [quick question]
On Wednesday 16 July 2008 16:58:11 Ger Hobbelt wrote: Hi, Sorry to bother, but just a quick question: those new threadid handling routines that got into 0.9.9 a while ago disappeared from the latest CVS (today) again. I assume this is intentional? Just a Yes/No 's all I crave. Thanks for answering and keep it up! Yep, I'll also be reverting Bodo's original idptr_callback commit too. It turns out easier to fix threadid relative to what was pre-0.9.9 than to hack it on top of existing stuff. So the first CRYPTO_THREADID attempt is gone, but I won't revert Bodo's existing changes until I'm ready to commit the second CRYPTO_THREADID attempt. Cheers, Geoff -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH] Windows builds revisited [fixes for #1335, #1704]
Hi Steph, Didn't see this show up on openssl-dev but that's probably because of the attachments. In any case, I'll top-post to leave your original email there (minus attachment) so that it's visible for whoever else wants to know. FWIW, I had your original post still flagged as TODO in my mail-client, it's just that like many other openssl hackers, windows is not my primary (or even secondary) environment. In fact, as it happens I haven't touched it at all in a couple of years but I need to reaquaint myself with it soon for other reasons, so had planned on taking a look at your stuff then (esp. as it deals with engine things). Please ping me personally in a while if you've still not heard anything, but FWIW this hadn't gone unnoticed, just unprocessed. OTOH if others get to this before I do then great, but I will try to take a peek some time soon. Cheers, Geoff On Saturday 05 July 2008 19:25:33 Steph Fox wrote: Ignore my patches from last week, please. I've now split them down now into smaller chunks, added some bits I missed the first time around and checked everything I've done against the OpenSSL bug tracker: - bug #1335 is fixed by the 'engine' patch (no-engine build fails) - bug #1704 is fixed by the 'build' patch (/Zi request) - bug #1598 should be closed as a duplicate of #153. The changed subject for #153 means the earlier report doesn't show up in bug tracker searches for ssl3_send_alert - that should probably be altered. The 'engine' patch also introduces three --enable-* options for items that are supposedly disabled by default, and disables them by default. Without these changes, no-engine is still broken in mk1mf builds. The 'build' patch also offers debug makefiles, adds library suffixes to differentiate between dynamic/static/release/debug builds, and fixes a handful of typos - some of which are *in variable names* - in the VC-32 makefile generation script. There are two smaller patches, one of which allows nasm or no-asm to be passed as an argument to ms/32all.bat in 0.9.8 or 0.9.9-dev, and the other of which fixes an MSVC compile warning (read: build failure) in 0.9.9-dev only. The patches were all created using diff -ur --binary, which forces UNIX line endings on all affected files. You may find that you need to run patch -p1 --binary when applying them. Note that batch files do NOT need CR/LF in order to run under NT-based systems. I'd like to start work on making the test suite functional for builds with configure lines other than 'perl Configure VC-WIN32', but it's a bit pointless if nobody on the OpenSSL dev team responds. I didn't even know the patches I sent last week wouldn't apply cleanly until I updated my openssl snaps to check for intervening changes - and following the instructions in the README turns all diff output to DOS, which I'm pretty sure isn't the way to go :) Thanks, - Steph ps This is the second time of sending. The openssl.org mail host doesn't like patches this week - hopefully it copes with tar.gz. -- Un terrien, c'est un singe avec des clefs de char... __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: SSL_shutdown nonblocking behavior
A quick skim of this patch seems to indicate that it makes sense, though the litmus test will be to get some kind of regression coverage. Eg. do connections get left dangling in any common scenarios? BTW, I note that the patch is against 0.9.8b, that's ... umm ... *old*. Could you please regen your patch against either 0.9.8h or a recent snapshot of the HEAD? Also, if you can let us know what test coverage you already have for this, that would help, and we could likewise ask here on the list for some others to verify that this doesn't tickle any other issues. (I'm thinking apache2 plus various https clients, stunnel, postfix-TLS, etc.) My vague concern here is that connection teardown is something browsers and bungled SSL/TLS implementations have managed to get wrong in various colourful ways in the past, so if your fix breaks interoperability in any of these scenarios, we'd need to rethink (or at least, we'd need to make the fixes conditional on the presence or absence of the appropriate run-time flags that turn on/off workarounds, etc). Thanks, Geoff On Tuesday 03 June 2008 09:01:34 Thor Lancelot Simon wrote: As has been discussed several times in (at least) the past two years, there's a serious problem with SSL_shutdown() for nonblocking connections: it discards the underlying BIO's WANT_READ/WANT_WRITE information so that the calling application cannot know in which direction it must wait for I/O; so the choices are to spin (by waiting for I/O in either direction), to use a timeout (not always practical in an event-driven application) and re-check, or to switch to blocking mode -- again not really a good option. Heuristics can reduce the chance of spinning or missing an I/O ready event to a very small chance, but they cannot eliminate it. This patch makes the problem go away. The attached patch was sent to openssl-dev by Darryl Miles two years ago. I strongly believe it implements correct behavior and is not really a change to the API -- it just uses WANT_READ/WANT_WRITE appropriately rather than returning SSL_ERROR_SYSCALL when it should not. This is *not* the patch offered by another frustrated nbio user last year which added a new error code to the library -- it uses the existing ones as it should. Would it be possible to get some feedback from the OpenSSL team? I feel quite strongly this patch is needed and am sorely tempted to just check it in to the copy of OpenSSL in NetBSD's tree, but would prefer to not do so if there's any chance I can get OpenSSL developer comment on the patch or the underlying problem, to avoid a local patch that might not then be needed. Darryl's description of the patch is at http://marc.info/?l=openssl-devm=115153998821797w=2 but one should note it addresses other problems with the API that may be more serious in practice than the one he originally observed (e.g. the blocking/ spinning described above). Thor __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [CVS] OpenSSL: openssl/crypto/x509/ x509_att.c
On Friday 30 May 2008 09:52:40 Ben Laurie wrote: Dr. Stephen Henson wrote: - if (len == -1) + if ((len == -1) !(attrtype MBSTRING_FLAG)) I do wish you wouldn't use these extra brackets around comparison operators. if (len == -1 !(attrtype MBSTRING_FLAG)) works just fine and is consistent with most of the rest of the code, and the rest of the world. I find Steve's version more readable. The object of the exercise is not to confront the reader with code that reminds them what the precedence of C operators are. Nor is it to require the reader to know those operators so well that they can spot code that looks right but is in fact wrong. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
On Monday 19 May 2008 15:27:24 dean gaudet wrote: Note that you should always build with no-asm if you're doing this kind of debug analysis. The assembly optimisations are likely to operate at granularities and in ways that valgrind could easily complain about. I don't know that this is the case, but it would certainly make sense to compare before posting a bug report. you know, this is sheer stupidity. you're suggesting that testing the no-asm code is a valid way of testing the assembly code? Whoa, you're getting a tad trigger happy there. I'm not suggesting that testing the no-asm code is a valid way of testing the assembly code, you're putting words in my mouth. I think my last sentence is pretty clear. Debugging and optimising the assembly code is a different ball-game than identifying C-level leaks or other bugs in openssl. That's why I suggested comparing with a no-asm build. If valgrind sees equivalent problems in both, fine. If it sees it only with the assembly optimisations, that's useful information too, right? additionally the suggestion of -DPURIFY as a way of testing the code is also completely broken software engineering practice. No, it's not. This is a PRNG, not a menu widget. You might also want to test with support for symbols and core-dumps, but run without either in production. Heisenburgs can also up when running in a debugger vs not running a debugger, using optimisation flags selectively, [...]. We have a PRNG, the accent is on the R, not the P. This is not valgrind's fault, just a one-off incompatibility with the notion that uninitialised==catastrophe. any special case changes for testing means you're not testing the REAL CODE. With respect, please check the though shalts at the door, thanks. for example if you build -DPURIFY then you also won't get notified of problems with other PRNG seeds which are supposed to be providing random *initialized* data. This is not true. -DPURIFY does *NOT* compile out the use of *initialised* data - that is the whole point. It compiles out any use of *uninitialized* data. (Or at least it's supposed to, Bodo thinks there's another case where an #ifdef would be needed in the corner case where there's a short width parameter - that's beside the point.) not to mention that a system compiled that way is insecure -- so you either have to link your binaries static (to avoid the danger of an insecure shared lib), or set up a chroot for testing. Also not true. in any event YOU'RE NOT TESTING THE REAL CODE. which is to say you're wasting your time if you test under any of these conditions. You haven't got your facts straight. In fact, I'm starting to suspect you've got them deliberately organised in a circle, for the sake of argument. Please make sure you're aware of *precisely* what's going on here, and *precisely* what -DPURIFY changes in openssl. Otherwise your second sentence is theologic, at best. openssl should not be relying on uninitialized data for anything. even if it doesn't matter from the point of view of the PRNG, it should be pretty damn clear it's horrible software engineering practice. More thou shalts. As for your other mail; On Monday 19 May 2008 23:48:14 dean gaudet wrote: On Thu, 15 May 2008, Bodo Moeller wrote: The use of unititialized data in this case is stupid because the entropy of this random data is close to zero. It may be zero, but it may be more, depending on what happened earlier in the program if the same memory locations have been in use before. This may very well include data that would be unpredictable to adversaries -- i.e., entropy; that's the point here. on the other hand it may be a known plaintext attack. For that sentence to be other than BS, it is a minimum requirement that the PRNG be fundamentally broken algorithmically. Please publish, or refrain from muddling the issue. what are you guys smoking? The charred remains of a debian package, it's very noxious. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
On Thursday 15 May 2008 16:51:55 John Parker wrote: I'm still seeing a lot of errors from valgrind, even with the latest snapshot. 19 15:12 tar xvfz ../openssl-0.9.8-stable-SNAP-20080515.tar.gz 20 15:12 cd openssl-0.9.8-stable-SNAP-20080515/ 21 15:12 ls 22 15:12 ./config no-asm -DPURIFY 23 15:12 make 24 15:14 valgrind ./apps/openssl genrsa 1024 Please let me know if I'm doing something wrong with this test sequence. The problems occur on Red Hat 5.1 server x86_64. For what it's worth, I don't get errors on (updated :) Ubuntu 7.10. I do get errors even with Bodo's addition to randfile.c. I'd be happy to post the valgrind output if that would be helpful. If this is environment/OS-specific, then it's probably indicative of a libc (or valgrind, or gcc) issue with the fixed-in-the-past versions of those packages on RH5.1-64bit. If compiling and running the same openssl source on a different system doesn't give you problems, that would seem to imply that the problem appears and disappears based on elements that vary rather than elements that are invariant. :-) OTOH, it might just be that the different OSes catch things differently. In any case, I don't have (or want) a RH5.1-64bit installation, so you might as well post some clippings from your output to give us an idea of what it's showing. (If possible, could you please use a snapshot from HEAD for this instead of 0.9.8-stable?) TIA. Cheers, Geoff PS: I wonder whether I should be so dismissive of RH5.1 - as far as I know, it always generated usable keys ... :-) __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
On Friday 16 May 2008 00:47:52 Thor Lancelot Simon wrote: On Thu, May 15, 2008 at 11:45:14PM +0200, Bodo Moeller wrote: It may be zero, but it may be more, depending on what happened earlier in the program if the same memory locations have been in use before. This may very well include data that would be unpredictable to adversaries -- i.e., entropy; that's the point here. Unfortunately, it may also very well include data that would be highly predictable to adversaries. If feeding predictable data into a PRNG that was already well seeded with unpredictable data produced a weaker PRNG, then you have found a security bug in the PRNG and I suggest you publish. (Or put it in a distribution-local patch and wait a couple of years...) After all, the PRNG itself is highly predictable to adversaries - it's open source - so the requirement of the PRNG is for it's stirring mechanism to be essentially mononotic w.r.t. entropy gained from the input. Any unpredictability in the data you feed it should make the PRNG output correspondingly unpredictable - and that should not become more predictable (ie. less unpredictable) because you supplement the existing unpredictable input with predictable input. That's the point - feed whatever you've got into it, and the resulting randomness is (roughly) as good as the total entropy of your input, no matter how sparse that input's entropy was. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
On Thursday 15 May 2008 11:52:08 John Parker wrote: It is already possible to use openssl and valgrind - just build OpenSSL with -DPURIFY, and it is quite clean. (we do it all the time here with WvStreams and Pathfinder, and it works like a charm). The problem is that this may reduce the keyspace so that keys are guessable. No it won't, it removes an entropy source whose quality is known to be unknown, ie. it may add nothing useful, it gets used just in case. Removing it does not reduce the keypsace at all. All you can say is that leaving it there *may* improve the PRNG depending on the user, the environment, the application, and quite probably, the alignment of the planets... The debian patch went further than -DPURIFY, as it removed more than just this unreliable source, it removed all use of reliable sources as well. http://blog.isotoma.com/2008/05/debians_openssl_disaster.html This blog does not suggest that building with -DPURIFY would a problem and nor should it. I think you may have misunderstood the details of this issue. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
On Thursday 15 May 2008 12:38:24 John Parker wrote: It is already possible to use openssl and valgrind - just build OpenSSL with -DPURIFY, and it is quite clean. Actually on my system, just -DPURIFY doesn't satisfy valgrind. What I'm asking for is something that both satisfies valgrind and doesn't reduce the keyspace. If you're using an up-to-date version of openssl when you see this (ie. a recent CVS snapshot from our website, even if it's from a stable branch for compatibility reasons), then please post details. -DPURIFY exists to facilitate debuggers that don't like reading uninitialised data, so if that's not the case then please provide details. Note however that there are a variety of gotchas that allow you to create little leaks if you're not careful, and valgrind could well be complaining about those instead. This blog does not suggest that building with -DPURIFY would a problem and nor should it. I think you may have misunderstood the details of this issue. I am clearly misunderstanding something. You seem to be saying that -DPURIFY satisfies valgrind but doesn't reduce the keyspace. I'm prepared to take it on faith that -DPURIFY doesn't reduce the keyspace. Well, more generally than some keyspace is the randomness of the PRNG itself. (Your keys are only random if the PRNG's output is random.) But yes, I'm saying that -DPURIFY does not diminish the quality of the PRNG, except *possibly* by some unquantifiable amount that you couldn't safely depend on anyway. As for your other mail; On Thursday 15 May 2008 12:09:46 John Parker wrote: All of this is independent of proper entropy seeding to the PRNG, which is what the debian patch crushed and which in turn led to the high seismic reading in the blogosphere. But it may help explain why I do *not* want us to unilaterally remove the use of uninitialised data in the PRNG. That seems to be motivated by a capitulation to the weight of users (or packagers) who don't know how to read the FAQ. Perhaps what we should do instead is I think we should be less worried how things seem and more worried about the practical consequences. That is more or less what I was doing. I hope that was clear. change -DPURIFY to -DNO_UNINIT_DATA or something else which has a clearer intention, so that debug packages (or even base packages that want to be valgrind-friendly) have a straightforward mechanism to apply. Well, a straightforward mechanism that doesn't kill the PRNG outright, I mean (otherwise there is already a highly-publicised patch we could apply...) What I was hoping for was a -DNO_UNINIT_DATA that wouldn't be the default, but wouldn't reduce the keyspace either. I believe this has been answered. For now, it's called -DPURIFY. Can someone provide a pointer to this highly-publicized patch? I'm afraid I'm dreadfully ignorant of the blogosphere. You started this mail thread, so you go and find it! :-) The patch I was referring to, tongue-in-cheek, is the debian patch that crippled the PRNG. As for the blogosphere, you aren't missing much, I'd recommend that continued ignorance would be far from dreadful - in fact I intend to join you in that respect, once this was-it-debian's-fault-or-openssl's-fault nonsense has died down a bit. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
I forgot to mention something; On Thursday 15 May 2008 12:38:24 John Parker wrote: It is already possible to use openssl and valgrind - just build OpenSSL with -DPURIFY, and it is quite clean. Actually on my system, just -DPURIFY doesn't satisfy valgrind. What I'm asking for is something that both satisfies valgrind and doesn't reduce the keyspace. If you're using an up-to-date version of openssl when you see this (ie. a recent CVS snapshot from our website, even if it's from a stable branch for compatibility reasons), then please post details. -DPURIFY exists to facilitate debuggers that don't like reading uninitialised data, so if that's not the case then please provide details. Note however that there are a variety of gotchas that allow you to create little leaks if you're not careful, and valgrind could well be complaining about those instead. Note that you should always build with no-asm if you're doing this kind of debug analysis. The assembly optimisations are likely to operate at granularities and in ways that valgrind could easily complain about. I don't know that this is the case, but it would certainly make sense to compare before posting a bug report. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
On Thursday 15 May 2008 16:56:17 Erik de Castro Lopo wrote: Patrick Patterson wrote: On May 15, 2008 10:58:07 am John Parker wrote: In the wake of the issues with Debian, is it possible to modify the source so that it is possible to use valgrind with openssl without reducing the key space? It is already possible to use openssl and valgrind - just build OpenSSL with -DPURIFY, and it is quite clean. A compile time option is not enough. I would like the be able to valgrind my program linked against the standard openssl library I get from my Linux distribution. Then tell your linux distribution to use -DPURIFY. Or order me an 8-core magic wand from Dell and I'll get right on it. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: valgrind and openssl
On Thursday 15 May 2008 17:31:45 Erik de Castro Lopo wrote: Geoff Thorpe wrote: Then tell your linux distribution to use -DPURIFY. Hangon, I've got a better idea. How about the OpenSSL develoeprs fix their library so that the standard version that they ship is valgrind clean. Then the distributions won't need to do anything other than compile it. What, you mean like how the standard version we ship has a good PRNG, so that the distributions don't need to do anything about that either? Funny, that doesn't work so well in the real world. Distributions always do something before compiling packages, as debian has so succinctly and spectacularly demonstrated. They pick target architecture settings for the distribution that are independent of the build host (eg. cross-compilation for non-x86 hosts, etc) and another other configuration options they think useful/necessary. Eg. static/dynamic, PIC or not, symbols or not, installation paths, optional features (and dependencies), documentation, [...]. Sometimes they even throw in patches, which is where they diverge dangerously from what we provide. If it is the distribution's preference to have openssl unnecessarily memset and/or unnecessarily restrict its seeding to pander to an unmodified and unconfigured valgrind, be that in the standard package or any other debug package, then that is their choice. We provide a (supported) method for doing this, it's called -DPURIFY. It doesn't require *any* source-code patching. ahem. Valgrind is a great tool and no doubt many non-noobs put it to good use on a daily basis. It helps find non-deterministic behaviour. That it finds non-deterministic behaviour in our PRNG should not be cause for basement-dwellers the world over to rise up against common-sense. We have a FAQ in case you get confused, and valgrind can also be taught to work around such cases as this. And again, you can build openssl to side-step the false-positives for a very small overhead. Just what level of base, ill-informed, and incompetent debugging help do we need to cater to? And to what non-technical extents are we prepared to go for it? Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: load_builtin_engines in Linux
Did you (re)try with a more recent snapshot? A fix to this area of the ENGINE logic was made recently in response to bugzilla ticket 1668; http://rt.openssl.org/Ticket/Display.html?id=1668 http://marc.info/?l=openssl-cvsm=120942258326907w=2 You indicate that you're using 0.9.8e which is a long way out of date (the latest 0.9.8 release is 0.9.8g, and even that doesn't have the fix - try a snapshot from; ftp://ftp.openssl.org/snapshot/ Use 0.9.8-stable-SNAP-*** if you need 0.9.8 compatibility. Cheers, Geoff On Monday 05 May 2008 05:45:34 Mathias Tausig wrote: Hello, sorry for bumping my question, but I was just wondering, wether might have posted it to the wrong list? cheers Mathias Hy! I want to use a Luna HSM with openssl, so I have downloaded the coresponding patch whic creates an engine and built it (version 0.9.8e). The problem is, that it is not working, because the engine is not a dynamic one but a builtin, and the ENGINE_load_lunaca3 function is nowhere executed. I du through the source code and found the load_builtin_engines function which should do the desired. It is executed via the apps_startup functions (defined in apps.h) in some envorinments but not in linux. Is there any particular reason why this is not done? Can I do any harm, if I just add this function to add_startup in a standard linux envornment? cheers Mathias __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
[openssl.org #1668] [PATCH] Fix for engine cache logic reversal
Nice analysis Ian and John, thanks for digging in to this. I agree with
what you've determined, though I think there was a missing 'uptodate'
line from the code too. I'm attaching a diff that matches yours but has
this extra line. Can you please confirm that this still gives you the
behaviour you're expecting? Thanks.
Index: crypto/engine/eng_table.c
===
RCS file: /v/openssl/cvs/openssl/crypto/engine/eng_table.c,v
retrieving revision 1.6.2.1
diff -u -r1.6.2.1 eng_table.c
--- crypto/engine/eng_table.c 6 Sep 2007 12:43:49 - 1.6.2.1
+++ crypto/engine/eng_table.c 27 Apr 2008 19:27:52 -
@@ -135,7 +135,7 @@
{
fnd = OPENSSL_malloc(sizeof(ENGINE_PILE));
if(!fnd) goto end;
- fnd-uptodate = 0;
+ fnd-uptodate = 1;
fnd-nid = *nids;
fnd-sk = sk_ENGINE_new_null();
if(!fnd-sk)
@@ -152,7 +152,7 @@
if(!sk_ENGINE_push(fnd-sk, e))
goto end;
/* touch this ENGINE_PILE */
- fnd-uptodate = 1;
+ fnd-uptodate = 0;
if(setdefault)
{
if(!engine_unlocked_init(e))
@@ -164,6 +164,7 @@
if(fnd-funct)
engine_unlocked_finish(fnd-funct, 0);
fnd-funct = e;
+ fnd-uptodate = 1;
}
nids++;
}
@@ -179,8 +180,7 @@
while((n = sk_ENGINE_find(pile-sk, e)) = 0)
{
(void)sk_ENGINE_delete(pile-sk, n);
- /* touch this ENGINE_CIPHER */
- pile-uptodate = 1;
+ pile-uptodate = 0;
}
if(pile-funct == e)
{
Re: Interesting logic in dso_lib.c (libcrypto)
Yeah I've raised this with Richard to get his sense of what he intended. Please raise a ticket on RT if you like and assign it to either of us. Cheers, Geoff On Wed, 2008-04-09 at 11:44 -0400, Brad House wrote: I'd have to look at the context of what is actually happening here but it looks like by intention, filespec1 should be allowed to be NULL as it can be retrieved from dso, but I see no additional sanity checks for filespec2, most likely, I would assume, the filespec1 reference on line 397 should _actually_ be filespec2, but again, that's just a cursory look (without evaluating how DSO_merge is actually called). -Brad Salivar.William wrote: That is what I got out of it. What is the process for getting code issues submitted and resolved for OpenSSL? *From:* [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] *On Behalf Of *Sendroiu Eugen *Sent:* Wednesday, April 09, 2008 4:41 AM *To:* openssl-dev@openssl.org *Subject:* Re: Interesting logic in dso_lib.c (libcrypto) If filespec1 is NULL, it doesn't matter what dso is, it will not pass the first if, so the latter checks for dso and filespec1 are useless. [snip] __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Nagios check_http plugin to do ssl
On Wed, 2008-03-26 at 17:07 -0400, Azam Syed wrote: I loaded openssl 0.9.8g and when I complie Nagios plugin it says yes next to openssl, but when I do the make I get the following. I complied Nagios plugin with [EMAIL PROTECTED] nagios-plugins-1.4.11]# ./configure --prefix=/usr/local/nagios/libexec --with-ssl-dir=/usr/local/ssl --with-libs=-ldl” .o sslutils.o netutils.o utils.o ../lib/libnagiosplug.a ../gl/libgnu.a -lnsl -lresolv -lssl -lcrypto gcc -g -O2 -o check_http check_http.o sslutils.o netutils.o utils.o -L/root/downloads/Nagios-3.0rc2/nagios-plugins-1.4.11/plugins -L/usr/local/ssl/lib ../lib/libnagiosplug.a ../gl/libgnu.a -lnsl -lresolv -lssl -lcrypto I haven't a clue what Nagios is, but it's clear from the output that the link step did not include the trailing -ldl that you need. One can only presume that --with-libs=-ldl isn't the configure switch you require. As for how to fix this, you'd need to take that up with Nagios, whoever and wherever they might be. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL and LSB
Theodore Tso wrote: I would suggest that the best way to do this is to *add* new mutator functions (and accessor functions, where necessary) which applications who care about ABI stability can use, and then document a set of interfaces for which ABI stability is guaranteed. That could be a relatively small set initially --- enough for applications that aren't doing anything strange, and then this list can gradually be expanded out, with some new ABI stable functions getting added as necessary. Possibly. There are (or have been) analogous developments that could contribute to your thinking of how to address LSB concerns. Eg. the FIPS work has certification, signature-checking, and other fixed interface considerations (I think, I haven't been involved in this at all). Also, OPENSSL_NO_DEPRECATED is used as a mechanism to nudge away from backwards-compatible interfaces to their preferred replacements (more of this coming to the cvs list in a day or three, BTW). If you define this symbol when building your code against openssl, deprecated interfaces will not be exposed from the headers. If you define this symbol when building openssl itself, deprecated interfaces will not be compiled (I know this doesn't address structures/macros/etc that are not compiled/linkable interfaces, but you get the general idea). There appear to be a few possible avenues to explore; * define LSB-specific wrapper libraries/interfaces to implement a stable ABI, * define only a small subset of functionality in LSB, with the intention of openssl keeping *those* interfaces fixed, * look for ways that a FIPS/NO_DEPRECATED-style pre-processor configuration could compile openssl in some LSB-friendly manner, * define LSB relative to an openssl stable branch. The last point should perhaps not be scoffed at. This is how compatibility is handled in the real world anyway - until openssl reaches some kind of v1.0 state, 0.9.7 is considered a different library to 0.9.8, such that fixes and maintenance for each branch *do* maintain ABI compatibility. Ie. distributions typically install distinct shared-libraries for each, and track them with distinct packages. Having said all that, I'll have to leave this issue to others who have the motivation and time for it. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Asynchronous ENGINE operation?
Hi Thor, On Sun, 2008-03-02 at 23:21 -0500, Thor Lancelot Simon wrote: FWIW, this is not really the right way of implementing async support in openssl. Completion events are not always going to be related to file descriptors in any clear way Certainly if the file in question is one for a Unix device driver, and that device driver is in use for asynchronous hardware-accellerated crypto operations, there will, in fact, be a perfectly clear relationship. Well, maybe in the cases you're interested in, but this is not an assumption I'm prepared to make w.r.t. abstractions. Even for crypto drivers that are fd-based, it would be lazy/inefficient for there to be a fd *per-operation* - ie. whilst a process's communication with a driver might very well be fd-based, the presence of data on that fd wouldn't necessarily indicate which asynchronous operation had completed. And even on unix, completions could occur via signals, IPC (of various forms), whatever. If openssl is to provide support for asynchronous completion, there needs to be a general interface for it. I don't care, call it an ENGINE-specified opaque token if you like; but you must be able to completion-wait on it without using threads and condition variables, or it won't be useful to many users of the existing non-blocking mode of the OpenSSL API. If you can't get the resource to wait on, or can't wait on it with select() or poll() -- then you cannot do asynchronous crypto. Sure, some kind of token representation will be required to allow the code up top to know there's something that needs completing before a given processing flow can continue, and that code would need to handle the cases of interest (ie. be it fd-based, signalled, a win32-message, [whatever]). My point is that the call the exact same API with the exact same parameters to continue semantic is unacceptable as a way of resuming. As I suggested earlier, even an fd-based mechanism could be transporting commands and completions for multiple crypto operations, so the token mechanism would need to include an ASYNC_TOKEN_is_complete() accessor - if that doesn't return TRUE, there's no point trying to resume the corresponding processing flow (a decent driver interface would allow you to associate opaque data with commands/completions that could point back to the token that can be resumed). , and besides which the SSL/TLS stack in openssl is not really one where you'd want to try and implement resume semantics. Ie. the call the exact same thing again approach sounds like the SSL/TLS objects get left in a limbo state after an asynchronous operation has started - I think I understand why you've done things this way We did it this way because it is *precisely what OpenSSL already requires for any asynchronous operation that returns before completion already*. There is no net change in semantics (these semantics are *already* required in the case where the underlying network transport is non-blocking; you must select for read or write per the WANT_READ or WANT_WRITE error return at the SSL layer, but the next SSL operation you must issue is required to be the exact same one you issued previously) merely an observation on my part that those semantics are...perhaps not what I myself would have designed. Nor I :-) Nonetheless, I've worked through one project already (which was one too many) where the suspend/resume mechanism consisted of calling back in to the API to resume. There are gargantuan problems associated with that. But the worst two; 1. exit paths on suspend - for every possible location where a function call is made that could in turn call other functions [and so on...] that result in an async crypto operation, there needs to be a *new* return value check for suspend that causes the function to exit without leaks. 2. wasted processing - when going back in, many functions that are involved in the call-stacks that lead to potentially-async operations include functions you do not want to repeat - eg. ASN [en|de]codings, memcpy()s, lookups, [etc]. Yes, the semantics are gory -- but they are already in place. Sorry if I caused confusion on this point; I meant only to add a new error code (WANT_CRYPTO in addition to the existing WANT_READ and WANT_WRITE) which would indicate the existing semantics re retrying operations are required (but to add an additional file descriptor to the select set), not change the whole mechanism. Do you already have a modified openssl tree to achieve what you're talking about? If so, I I presume you have this extra WANT_CRYPTO code too. If not, how is the extra return code on it's own going to help? Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Asynchronous ENGINE operation?
FWIW, I think we mostly agree but are attacking the general issue from different angles. On Wed, 2008-03-05 at 10:44 -0500, Thor Lancelot Simon wrote: Well, I did not suggest that one would ever use a file descriptor per operation. But, let me try again. Note first that that OpenSSL does not abstract I/O completion waiting (the API provides SSL_read(), SSL_write(), SSL_connect(), SSL_shutdown(), but *not* SSL_poll() or SSL_select). When I proposed correcting this, last year, I got a pretty strongly negative reception. I was not involved in the discussion so am not familiar with the arguments. However I agree that SSL_[poll|select]() would be unwise because it presumes to make the SSL/TLS stack transport-aware, whereas the BIO scheme is an honest (if sometimes inelegant) attempt to leave the stack transport-agnostic. SSL/TLS can be fed data on all sides purely from memory, and the real transport could come from lots of non-fd-based sources. Eg. if you somehow didn't trust multi-user windows (though we all know windows is 100% trustworthy so this is obviously an outlandish proposition ...) then you could theoretically use SSL/TLS to encapsulate message passing - ie. the ability to watch IPC would not reveal the contents of that IPC. Hmm, I can just about smell the presence of a numbskull who'll try to build a DRM scheme around this ... Just as the SSL user needs to know transport specifics to create BIOs in the first place for hooking up to the SSL object, they also need to know how (and on what) to wait when they see WANT_READ/WANT_WRITE codes and so-forth. I think the same would essentially be true for the use of asynchronous crypto. The point is to keep openssl agnostic, not the application. (Now I've probably offended a vast array of library fascists who will exclaim that libraries are supposed to hide the problem rather than hiding themselves from the problem - to which I can only say, (1) this is security, not ODBC-for-javascript, and (2) there is a world beyond POSIX, hard to imagine as that may be.) Note next that there are already a nontrivial number of nontrivial applications using the existing non-blocking select()/poll() driven API to OpenSSL, despite its warts. This API *already* requires, quite explicitly, extracting the file descriptor from each SSL session and adding those descriptors to a select() or poll() set. Applications are already written to this API and it does not abstract completion waiting as you suggest it should (I happen to agree with you, but neither you nor I can adjust all of everyone else's code out there in the world!). Actually I don't claim that it should abstract completion waiting, only that the SSL/TLS API shouldn't. For one thing, async crypto will be a libcrypto interface - not an SSL/TLS-specific interface. Just as BIOs are a libcrypto interface. This API also *already* requires any operations that would block to be reissued at the SSL layer by retrying exactly the same operation again after the file descriptor for that session comes up ready with select or poll. Sure, but that's because you are *not* suspended miles down a code path that you need to find your way back to. It means the state-machine has advanced as far as it can go and to advance further, you will need the presence of either input on your read BIO or some drainage on your write BIO. Those events are on the exterior of the SSL/TLS state-machine and represent a stable state, the code-paths exit cleanly in that state and when progress is possible, we will go down new code paths to do new work. This is not the same as trying to pick up where you left off way down a code-path that has not been designed with suspend-and-retry semantics in mind. Eg. if you were calling SSL_read() when you got a WANT_WRITE, but you then called SSL_write() - it should *not* crash. What you're describing, about being able to replay your way down a prior code-path, would be very hard to make robust in an equivalent way. NB, this would be different if the SSL/TLS code had, since the beginning, the idea of a third BIO-like dependency - ie. if the state-machine was designed to interact with a read BIO, a write BIO, and a crypto-interconnect. If that were the case, WANT_CRYPTO would already exist, just like WANT_READ and WANT_WRITE. But the code is not like that and making it like that would amount (IMHO) to a rewrite. I do not love this API, but it is what all the existing, non-threaded non-blocking OpenSSL API consumers are written to, and I think it would be a considerable mistake to change it now. I quite agree. So, given my note first and note second I believe I have established that, to do non-blocking I/O in a single-threaded application with OpenSSL, one must already: 1) Dip beneath the SSL abstraction layer to get the file descriptor for each session. No, this is one form it must be able to take, but this is not *the* general method for OpenSSL to support.
Re: Asynchronous ENGINE operation?
On Wed, 2008-03-05 at 13:54 -0500, Thor Lancelot Simon wrote: On Wed, Mar 05, 2008 at 12:10:04PM -0500, Geoff Thorpe wrote: I was not involved in the discussion so am not familiar with the arguments. However I agree that SSL_[poll|select]() would be unwise because it presumes to make the SSL/TLS stack transport-aware, whereas the BIO scheme is an honest (if sometimes inelegant) attempt to leave the stack transport-agnostic. SSL/TLS can be fed data on all sides But you can't stay transport-agnostic and actually use the stack in non-blocking mode now. You can't, but libssl sure can. That's why SSL_poll() is not a good idea IMHO. I've written SSL utils/tests that read and write memory on all 4 I/O points of the SSL state machine, and I'd argue that I'm using the stack in a *very* non-blocking mode in that case, what on earth can it block on? It's up to me to get new data to the two read inputs, and get data from the two send outputs, but *I'm* transport-aware, not the stack. SSL/TLS is a duplex data encapsulation protocol, it has *nothing* to do with TCP, unix domain sockets, stdio, or networking. Per se. It's assumptions involve pairs of reliable, unidirectional data-streams, that's all. Moreover, awareness of when data is available (read) and when output buffering is available (send, congestion, etc) depends on your SSL/TLS *use-case*. Ideally, there should be no network code anywhere in ./ssl/, and in fact there should only be network code in transport-specific files in ./crypto/bio/. So having an SSL_[poll| select]() interface in ssl.h is extremely counter-intuitive to me. You *must* extract the underlying file descriptor and sleep on it with select/poll, which effectively forecloses any option of using some other asynchronous event mechanism at the same time except heavyweight signals, under Unix at least (and using signals that way will butcher performance; I have benchmarked this). Sure, your circumstances may be like that. But I won't accept that openssl should wed itself to this model inextricably. If you already have proprietary code to this effect, then you don't need openssl.org to reproduce your solution for you. I don't mean to be argumentative, just saying that it's unacceptable to take a short-sighted approach to openssl.org's tree just to mimic what someone has already done when scratching their specific itch. Async crypto is a hard problem for openssl because of its historical baggage. We will crack it, but not in a way that solves the easiest use-case and makes all other use-cases harder than ever. If you're in an embedded/custom OS with a flat-memory model and no user-mode, and some opaque code provides your I/O and crypto via callbacks, please tell me how SSL_poll() is going to help? Sure, but that's because you are *not* suspended miles down a code path that you need to find your way back to. It means the state-machine has advanced as far as it can go and to advance further, you will need the presence of either input on your read BIO or some drainage on your write BIO. Those events are on the exterior of the SSL/TLS state-machine and represent a stable state, the code-paths exit cleanly in that state and when progress is possible, we will go down new code paths to do new work. This is not the same as trying to pick up where you left off way down a code-path that has not been designed with suspend-and-retry semantics in mind. Eg. if you were calling SSL_read() when you got a WANT_WRITE, but you then called SSL_write() - it should *not* crash. Should not, or does not? I think you're a bit optimistic about the current implementation, for what it's worth. Failure to retry the exact same operation, now, causes chaos, including data corruption. Ahh, well I didn't know about that. It shouldn't be the case (obviously). Do do you have any idea why? Fixes are welcome ;-) What you're describing, about being able to replay your way down a prior code-path, would be very hard to make robust in an equivalent way. For what it's worth, I and my coworkers have done this, to an originally openssl-derived code base, which still retains BIO, etc., and it was not too hard to make it work at least as well as the current read/write suspension does. I've also done it, and maybe I have an impatient nature, but I got royally hacked off and honestly wouldn't have trusted the result much (audit *that* you [EMAIL PROTECTED]). Bear in mind that if the stack is going to support async crypto, it ought to be ready for *any* crypto operation to go async. Did your changes only enable asynchronicity on specific calls/paths? I say any because the decision as to which operations should be async (and when) may be related to factors other than the particular code-path that got you there. Eg. resource availability, data-size, load, algorithm, key/iv-locality, time-slicing, [...]. No, this is one form it must be able to take
Re: Asynchronous ENGINE operation?
Hi Thor, I'm slowly coming back through a few tonnes of spooled email that has been accumulating for months (including everything openssl-related). That's just some context. I may have missed prior discussion about all sorts of things related to ENGINE and async requirements, if so my apologies. FWIW, this is not really the right way of implementing async support in openssl. Completion events are not always going to be related to file descriptors in any clear way, and besides which the SSL/TLS stack in openssl is not really one where you'd want to try and implement resume semantics. Ie. the call the exact same thing again approach sounds like the SSL/TLS objects get left in a limbo state after an asynchronous operation has started - I think I understand why you've done things this way (given a specific problem, you've probably come up with the easiest and most obvious specific solution that didn't involve the need for a nervous breakdown nor a bump in international coffee shipments...) FWIW, I've done something similar in the past with openssl for a custom job, and I neither could nor would have committed that code (or anything like it) into the public repository. BTW, this issue is not really related to ENGINE at all. That's just a bundling of RSA/DH/[...] vtable objects with some indexing and ref-counting. The ability for a modexp operation to go asynchronous would have little to do with whether it had been installed (and reference-counted) by an ENGINE. That's by the by, but just so we don't get confused - I really don't see this as an ENGINE issue so much as a call-stack issue, and in that regard, it has a lot more to do with the SSL/TLS implementation than anything else. I already have some ideas about this but will need to think some more and try to formulate something concrete. For one thing, the approach taken needs to be as portable as the rest of the openssl code base and needs to not be a singularly weird quirk in the SSL/TLS API. But if you have some code already that can serve as a point of discussion, please post a diff (or URL). I'll try to do the same as soon as practicable. Cheers, Geoff On Mon, 2008-02-18 at 09:09 -0500, Thor Lancelot Simon wrote: I have a set of local modifications to a proprietary SSL library which has an openssl-compatible API. They significantly increase performance for applications using non-blocking I/O by allowing the SSL I/O operations (SSL_read(), SSL_write(), etc.) to return a CRYPTO_PENDING error code if the hardware device actually performing the crypto is not yet ready. [snip] __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Random number generator, uninitialised data and valgrind.
Umm... On May 1, 2006 03:14 pm, Kurt Roeckx wrote: The code in question that has the problem are the following 2 pieces of code in crypto/rand/md_rand.c: 247: MD_Update(m,buf,j); 467: #ifndef PURIFY MD_Update(m,buf,j); /* purify complains */ #endif There's your first clue, build with -DPURIFY :-) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Self-interest and materialistic desire are parts of who we are, but not all. To base a social and economic system on these traits is dangerously fundamentalist. -- Joel Bakan __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Missing engines from crypto/engine when making a shared library
without needing to use any cryptographic hardware. The gmp engine seems to be disabled, yet it gets build as a shared module, but doesn't have any code in it. I don't think it's useful that a module gets created for that, since I can't do a thing with it. AFAICR, it should only be built if you explicitly configure it (it'll require GMP headers and libs anyway). It's not intended for production use and I would be quite surprised if any packagers shipped it. You are right however about the empty module - this is the case for any other engine that is disabled. The problem is that the build system isn't aware of what modules to build and not build so goes ahead and builds them all anyway - the preprocessor causes disabled modules to compile empty. I'm not sure what the best fix to this is, but it certainly involves make, probably perl, and quite a bit of digging. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Self-interest and materialistic desire are parts of who we are, but not all. To base a social and economic system on these traits is dangerously fundamentalist. -- Joel Bakan __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Problem using Broadcom uBSec engine in 0.9.8
Hi Martin, Comments inline. On October 28, 2005 09:54 am, Martin Del Vecchio wrote: Yes, there are two different libubsec.so libraries: 1) The one built in OpenSSL with ./config shared; this is the stub library that contains bind_engine, etc. 2) The one provided by the Broadcom SDK; this is the actual library that contains ubsec_bits_to_bytes, etc. If I put 1) in the right place (/usr/local/ssl/lib/engines), then ENGINE_by_id (ubsec) succeeds. If I remove it from that directory, then ENGINE_by_id (ubsec) fails. So I put 1) in the right place. Then I load the dynamic engine: Engine = ENGINE_by_id (dynamic); and give it the following commands: Return = ENGINE_ctrl_cmd_string (Engine, SO_PATH, ubsec, 0); Return = ENGINE_ctrl_cmd_string (Engine, LOAD, NULL, 0); These three steps are essentially what 'ENGINE_by_id(ubsec)' does. The result is that 'Engine' is now the ubsec engine and no longer the (or rather a copy of the) dynamic engine. Both succeed; it find the library in in /usr/local/ssl/lib/engines/, which is the OpenSSL stub. But when I run: Return = ENGINE_set_default (Engine, ENGINE_METHOD_ALL); It fails, because it can't find the ubsec_bits_to_bytes symbol. Yup, ENGINE_set_default() will try to initialise the engine - if it isn't already initialised - before setting it as a default implementation. Loading the broadcom library is delayed until the initialisation step to ensure you can always get a structural reference to an engine, whether or not you have the corresponding hardware, drivers, [etc]. Eg. it's what allows you see the ctrl-commands available for engines even if you can't initialise them as yet; 'apps/openssl engine -' So it smells like it wants the Broadcom library in the LOAD command, so I change that to: Return = ENGINE_ctrl_cmd_string (Engine, SO_PATH, /usr/lib/libubsec.so, 0); Which is where the Broadcom library is. Now the LOAD command fails, because it can't find the bind_engine symbol. So now it's looking for the OpenSSL stub library there! So then I renamed the Broadcom library to /usr/lib/bc.libubsec.so, and said: Return = ENGINE_ctrl_cmd_string (Engine, SO_PATH, /usr/lib/bc.libubsec.so, 0); And I got the same error. Did you do this as a replacement for the earlier SO_PATH step (ie. prior to the LOAD)? Until you run the LOAD command, 'Engine' is the dynamic engine. Once you've called LOAD, 'Engine' is the loaded ubsec engine. At this point, the SO_PATH ctrl-command is the one implemented by the ubsec engine and not that of the dynamic engine, ie. it refers to the broadcom library and not the engine library, but that's only *after* the LOAD. I think this is probably what you're missing - let me know if I've misunderstood. So, you probably want; #ifdef DO_IT_THE_LONG_WAY /* SO_PATH points the dynamic engine to the ubsec engine */ Engine = ENGINE_by_id (dynamic); Return = ENGINE_ctrl_cmd_string (Engine, SO_PATH, ubsec, 0); /* LOAD changes 'Engine' to be a ubsec engine handle */ Return = ENGINE_ctrl_cmd_string (Engine, LOAD, NULL, 0); #else /* This achieves the same thing as above, if the library has the * appropriate file-name and path. */ Engine = ENGINE_by_id (ubsec); #endif /* SO_PATH points the ubsec engine to the broadcom library */ Return = ENGINE_ctrl_cmd_string (Engine, SO_PATH, bc.libubsec.so, 0); /* This implicitly initialises the engine */ Return = ENGINE_set_default (Engine, ENGINE_METHOD_ALL); Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Self-interest and materialistic desire are parts of who we are, but not all. To base a social and economic system on these traits is dangerously fundamentalist. -- Joel Bakan __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Problem using Broadcom uBSec engine in 0.9.8
On October 27, 2005 02:47 pm, Martin Del Vecchio wrote: I get an error similar to what I was seeing before; it can't find the symbol 'ubsec_bytes_to_bits': error:2506406A:DSO support routines:DLFCN_BIND_FUNC:could not bind to the requested symbol name (dso_dlfcn.c:261) - symname(ubsec_bytes_to_bits): /usr/local/ssl/lib/engines/libubsec.so: undefined symbol: ubsec_bytes_to_bits OK, you need to know that the ubsec engine itself is a stub that uses a broadcom-provided library to do the actual ubsec operations, the engine merely slides functionality into openssl that is based on the the broadcom SDK. I think what's happening is that you've got the engine library loading *itself* rather than the broadcom support library. The error you see is just the first symbol that the ubsec engine tries to bind to - and of course none of the expected symbols exist because it's not the broadcom library, it's the engine library. I don't have broadcom hardware nor their SDK, libraries/drivers, etc, so you may need to dig around to get to the bottom of this. But first off, figure out which library is which :-) If the broadcom-provided lib is called libubsec.so, that'd be an unfortunate name-conflict but it shouldn't be catastrophic, because openssl doesn't look for engine libraries in arbitrary locations (doing so would have unquantifiable security consequences) but in an openssl-specific location. The ubsec engine, on the other hand, should pick up the broadcom library using standard library locations - and this is either missing or in a directory that isn't getting searched (LD_LIBRARY_PATH might help here) - it seems you've fixed this problem by moving/copying the engine library somewhere visible instead so that it loads itself instead of the broadcom lib. Hope that helps, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Self-interest and materialistic desire are parts of who we are, but not all. To base a social and economic system on these traits is dangerously fundamentalist. -- Joel Bakan __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Problem using Broadcom uBSec engine in 0.9.8
On October 25, 2005 11:06 am, Martin Del Vecchio wrote: I'm still looking for help with this problem. The uBSec support works in 0.9.7g with static linking. The uBSec support works in 0.9.7g with shared libraries. The uBSec support works in 0.9.8 with static linking. The uBSec support fails in 0.9.8 with shared libraries. [snip] Here is the command line I use to build the test program: gcc -o test test.c -I include ./libssl.a ./libcrypto.a -ldl It feels like this is such a major bug that it can't possibly be real, and I must be doing something wrong. The problem is that if you do a shared build, the engines are built as shared-libraries too - that is, this is now the case but wasn't under 0.9.7. Your code is calling ENGINE_load_ubsec() which doesn't exist in libcrypto, because when you call ENGINE_by_id(ubsec) it should fall back to automatically searching the default path to load the engine shared lib (libubsec.so). Let me know if this isn't the case, but from an initial glance that seems to be what you're hitting. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Self-interest and materialistic desire are parts of who we are, but not all. To base a social and economic system on these traits is dangerously fundamentalist. -- Joel Bakan __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Dynamic Engine problems with some 0.9.7x version
Hi, On October 24, 2005 11:50 am, Marco GRELLA wrote: I have developed an OpenSSL Engine for our hw crypto accelerator. I have no problems in using it as a dynamic engine with OpenSSL 0.9.8, neither I have problems in loading it with OpenSSL 0.9.7d. But I get problems with 0.9.7g/i (so, I guess 'h' too) versions. Are you rebuilding the dynamic engine (lib) for each version you try to load against or did you build against one fixed version and try to load it against others? If the latter, which version did you build against? Here I report the output I get: --- --- [EMAIL PROTECTED] hwa_engine]# openssl097i engine dynamic -c - -pre SO_PATH:/usr/local/ssl7i/lib/engines/libhwa.so -pre NO_VCHECK:1 -pre ID:hwa -pre LIST_ADD:1 -pre LOAD (dynamic) Dynamic engine loading support [Success]: SO_PATH:/usr/local/ssl7i/lib/engines/libhwa.so [Success]: NO_VCHECK:1 [Success]: ID:hwa [Success]: LIST_ADD:1 [Failure]: LOAD 4284:error:260B606D:engine routines:DYNAMIC_LOAD:init failed:eng_dyn.c:433: SO_PATH: Specifies the path to the new ENGINE shared library (input flags): STRING NO_VCHECK: Specifies to continue even if version checking fails (boolean) (input flags): NUMERIC ID: Specifies an ENGINE id name for loading (input flags): STRING LIST_ADD: Whether to add a loaded ENGINE to the internal list (0=no,1=yes,2=mandatory) (input flags): NUMERIC LOAD: Load up the ENGINE specified by other settings (input flags): NO_INPUT Why the NO_VCHECK? An engine lib built against 0.9.7x *should* load ok against any 0.9.7y (unless it doesn't, of course:-) but I think it's wise to assume this won't be the case between 0.9.7 and 0.9.8. If that's what you're trying, I wouldn't be surprised if you hit problems. Anyway, dig a little deeper if you can - if need be set a break point at or before eng_dyn.c:433 and go in a step it through. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Self-interest and materialistic desire are parts of who we are, but not all. To base a social and economic system on these traits is dangerously fundamentalist. -- Joel Bakan __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Dynamic Engine support on OS X
On October 4, 2005 04:08 am, Christopher P. Masone wrote: --- Frederic Donnat wrote: 26743:error:2507006C:DSO support routines:DSO_load:functionality not This says that you do not have the DSO support on! --- end of quote --- Yes, that error message led me to try passing both -DSO_DLFCN and -DDSO_DLFCN to the config script. But the first flag didn't change the behavior, and the second gave me a compile-time error. As far as I can tell, there is no dso config option...there's shared, which I was using. Is there a dso option for ./config? As Frederic observed, this is probably a case of DSO support missing from your build. Which target are you configuring the source for? (Try ./config -t) If you look in the TABLE file for that target, you should get a list of build settings. It may be that the default target from ./config isn't optimal, in which case run perl Configure to get a list of available targets and then run perl Configure target ... to specify it. Or it may be that the target for your system just doesn't have the appropriate DSO implementation enabled (or that it couldn't be enabled because of problems someone observed) - check the dso_scheme element for each entry in TABLE, for example. BTW, someone pointed out in a private mail that OSX does things differently, and that there are changes between 10.3 and 10.4 that affect this. Hmmm. Why aren't there any apple.com reps on this list? Or are there? (I'm sure Darwin would have encouraged a more active participation in the open source ecosystem...) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Même ceux qui se sentent pas des nôtres, ne nous voyant plus à genoux, seront, plus que jamais, chez eux chez nous. -- Loco Locass __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Dynamic Engine support on OS X
Hi, On October 3, 2005 01:15 pm, Christopher P. Masone wrote: I've been trying to get dynamic engine support working on OS X for about a week now, and no luck. I get a functionality not supported error when I try to load a dynamic engine. I'm wondering if dynamic engines are not supported on OS X, or if I'm just not building right. Is this the appropriate place to ask about this? If so, I can be more specific about the details of my problem... Sure, what kind of engine are you trying to build/load? I can't see any earthly reason why engines shouldn't work on OS-X, but I haven't tried this myself. Can you post a little more detail about the problem? Also, have you played with openssl engine - at all? It may help by generating an error-trace. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.openssl.org/ __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Considering SSL and Cryto libraries for LSB
On June 29, 2005 05:50 pm, Banginwar, Rajesh wrote: As part of LSB standardization process, we look at the interfaces and corresponding data types and make it part of the specification. If the data types are expected to change and the interfaces do not hide them, then that part of the library may not be a candidate for LSB. Well there are two things that spring to mind. 1. Most use of openssl, at least as might be considered useful from an LSB-perspective, is of the SSL layer itself (libssl). This requires the direct use of certain APIs in the underlying libcrypto layer too, but AFAICT these would be easier to standardise on than the full set of APIs and are probably more ABI-friendly (ie. loading and initialising RSA/X509 objects, etc). NB, that's a cursory thought, I've not attempted to dig terribly far on this. 2. This is a refined variation on the issues distributions already face when distributing openssl in shared-library form and maintaining classes of applications that have dependencies on these libs. The consensus seems to be to use versioning to manage this, where a system can have multiple openssl libraries installed of differing version levels. Eg. 0.9.6, 0.9.7, etc. Bugfix releases to individual versions are expected to maintain binary compatibility, or distributors at least Q/A this at their end and patch around any issues as/where necessary. You'd have to check with the distributions themselves to know more though, that's out of our hands. Mark, any comments from a Redhat perspective? The reasons Steve outlined cover why we can't provide assurances about full ABI compatibility (nor even API compatibility) in a long-term sense. To do so would take a lot of undesirable but legacy aspects of the toolkit and fix them in place. However, the unwritten rule is to ensure that each incremental(/bugfix) release of a given stable branch does preserve binary compatibility, and incorporation into LSB definitions would obviously go a long way turning this into a firmer commitment. Note also that major releases, which we freely admit can break ABI and/or API compatibility where required, are made far less frequently - as a perusal of release history will aptly demonstrate. I should also mention that we're not making vast overhauls of APIs on a regular basis, nor for the mere fun of it, so if adaptations are required long-term to allow future LSB revisions to incorporate newer release levels, this would hardly involve major surgery. Perhaps that's an avenue until such time as openssl does get to some kind of 1.0 plateau. Hope springs eternal. [ Or people wanting LSB compliancy can always statically link any version of openssl they like, but I guess that's not what you're after :-) ] Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.openssl.org/ __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Considering SSL and Cryto libraries for LSB
On June 29, 2005 08:44 pm, Banginwar, Rajesh wrote: So far from the preliminary analysis that we have done (by looking at some of the OSS applications) we see both libssl and libcrypto being used. E.g. from libcrypto I find functions in EVP, RSA, MD5 and DSA sets more commonly used than others. Unfortunately this is not an exhaustive study. Do you or anyone on this project have data suggesting which APIs are candidates for LSB inclusion both from demand and stability point of view? I'm not sure how you would harvest this kind of information. From a build perspective, it may be quite difficult to disentangle one API from the next. I guess someone truly masoc^H^H^H^H^Hdedicated could create an alternative openssl-lsb.h header to build against - one that declared only the subset of functionality that mattered from an LSB perspective and then see how external apps fare building against that. However there's a lot of macro-abuse in some areas of the code, and if this shimming got tangled up in trying to reproduce ASN1 definitions then I don't even want to contemplate where it might lead... Something to think about I guess. But it certainly seems hairy to attempt to standardise too much invariance across major releases that we expect will ... um ... vary. I just checked the release history for openssl and it is encouraging to see the major releases are years apart. I see that with some efforts it may be possible to hide some of the data structure dependencies (as Steve pointed out in other emails on this thread) and get some sets of APIs to a stable level and push for LSB inclusion. There was at least one comment from application developer encouraging this even though it will require major changes. It sounds like a fairly labourious process though (ie. I doubt we'll be dispersing the volunteers with tear-gas or anything). Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.openssl.org/ __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Problems with engines in 0.9.8-beta1
On May 26, 2005 03:06 am, Richard Levitte - VMS Whacker wrote: geoff Richard, any idea of how safe it would be to change the names geoff of the two shared librariesy at this stage of the 0.9.8 betas? geoff I'm reluctant to charge ahead for fear of breaking the strange geoff builds (win32, VMS, cygwin, ...) (I'll join Corinna in saying What? VMS a strange build? How dare you... ;-)) Eek, I've incurred the wrath of the Cult Of Strange Platforms In Denial ... :-) I have no problems with a name change of the sort. In the VMS case, I doubt anyone will mind :-). Actually, even in the unstrange build, this would be more intrusive than I anticipated - it would require renaming the C files too (ie. the makefile maps 'ncipher' to 'libncipher.so' and 'e_ncipher.c'). Thoughts? [snip] geoff duplicate symbol in libcrypto and so make install ended up [snip] Does that happen with a clean working directory? Oops, good call :-) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Greedy Genghis George, Guru of God and Guns. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Problems with engines in 0.9.8-beta1
On May 22, 2005 08:17 am, Corinna Vinschen wrote: now that I had first contact with engines, I thought it might be better to give them some testing. Yes, thanks for doing so :-) However, I found that lib4758_cca.so and libncipher.so don't load, because the engine id differs from the engine name. The engine id of lib4758_cca.so is 4758cca instead of 4758_cca, the id of libncipher.so is chil instead of ncipher. Yes, this is unfortunate. I've just committed a fix to the 0.9.8-stable and HEAD branches that will tolerate both names when binding as a dynamic engine. It'd still be preferable to change the names of the generated shared libraries too so that the default name with static and dynamic use is the same (ie. using 'chil' for a built-in engine and 'ncipher' for an external engine make much sense). Right now the existing change will just allow you to dynamically bind using 'ncipher'. Please try out the next nightly snapshot if you're able. Richard, any idea of how safe it would be to change the names of the two shared librariesy at this stage of the 0.9.8 betas? I'm reluctant to charge ahead for fear of breaking the strange builds (win32, VMS, cygwin, ...) Oh that reminds me too - the build I tried earlier got a duplicate symbol in libcrypto and so make install ended up without the shared-library version of libcrypto installed - everything seemed to work anyway (presumably everything linked to libcrypto.a instead) so this goes unnoticed quite easily. I'll try to dig up more info tomorrow when I get back to the machine I was on. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Greedy Genghis George, Guru of God and Guns. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
[openssl.org #956] engine code in ECDSA and ECDH
[EMAIL PROTECTED] - Thu Oct 21 08:19:52 2004]: please try a recent snapshot So it looks like this issue was resolved (I haven't checked, but Nils is a trustworthy sort :-). I'm closing the ticket. Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: [openssl.org #982] BN_add failuire in tests for openssl-0.9.7e on Sun 5.8
On December 11, 2004 08:41 am, Andy Polyakov via RT wrote: starting big number library test, could take a while... test BN_add Add test failed! I can't reproduce this problem with Sun C 5.2. There should be test/tmp.bntest file left over, can you submit it? Essentially this smells compiler bug and I should recommend to try another compiler [or apply vendor patches] or drop optimization level. We had problems with Sun C 5.0 (as you might notice in ./config), so there is a remote opportunity that 5.1 can be suffering from similar deficiency too... A. If it turns out not to be compiler bug, see if you can reproduce it with a CVS checkout or snapshot of the main branch. There's been an audit of the bignum code going on there, and all sorts of little conditions have been caught and corrected - it's quite possible you're seeing one of these conditions that remain in the 0.9.7-stable branch. (NB, we've avoided trying to backport these bignum changes to the stable branches because they're interdependent can't be moved piecemeal - and moving a large change-set back to a stable branch tends to make administrators and package maintainers very grumpy.) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Greedy Genghis George, Guru of God and Guns. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #697] bn-top related bug fixes
[EMAIL PROTECTED] - Thu Sep 25 11:37:20 2003]: as requested by Nil Larschs, i'm sending this diff to [EMAIL PROTECTED] For a discussion of these bugs and fixes, see the thread http://www.mail-archive.com/openssl-dev@openssl.org/msg16241.html Yup, in fact the bignum audit propsed in that mail thread has since begun and resolved a lot of stuff. I'll revisit the issues you raised and see how they look in the context of the new code - if the problems are still there, it'll be possible to fix them properly now. Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: Registering custom objects from dynamically-loaded engine
On December 3, 2004 07:30 pm, Dr. Stephen Henson wrote: On Fri, Dec 03, 2004, Dmitry Belyavsky wrote: On Fri, 3 Dec 2004, Dmitry Belyavsky wrote: I have custom dynamically-loaded engine implementing some non-standard algorithms. So I register NIDs using OBJ_create from engine bind function. Registering code looks like my_NID = OBJ_create(1.2.3.4.5,OID_example,Our example OID); On unload I get segfault in apps_shutdown. It happens in OBJ_NAME_cleanup. It looks like OBJ_NAME_cleanup tries to free memory belongs to unloaded library. ^^ ahh. What should I do to avoid this? I found out segfault happens because function registered as ENGINE_set_destroy_function is never called. When it is called (for example, when I use engine only to calc digest), there are some memory leaks looks like they are because of registering objects. I hacked up a quick dynamic engine that called OBJ_create() only and I couldn't reproduce this. I used the autoconfig mechanism mentioned in config(5) to load it. The object was added, it was recognized by asn1parse and there were no leaks on shutdown. Are you calling OBJ_cleanup() for example in the dynamic engine? This is probably another global state-type problem between the loader and loadee. See IMPLEMENT_DYNAMIC_BIND_FN() in engine.h for an peek into the horror. I'm guessing you guys are using different platforms, or at least different methods of compilation/linking? We need to ensure that the loaded engine use the loading app/lib's state - if necessary by redirecting its OBJ functionality to that of the loader. I'm out of here for the weekend, but if noone gets to the bottom of this before middle(ish) of next week, I'll take a nosey. Which version are you using BTW? Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ Greedy Genghis George, Guru of God and Guns. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL with Security Processor
On October 21, 2004 07:04 pm, Gurpreet Grewal wrote: I am facing a problem when I am trying to use the OpenSSL (0.9.7d to be precise) with the BraodCom BCM5823 card. I tried running the following two commands- openssl des3 -in inputfile -out outputfile -engine ubsec openssl des3 -in inputfile -out outputfile Ideally the first command should take less time/less main CPU coz it should be using the SSL processor. But they take almost the same amount of time/main CPU. That's because the 'ubsec' ENGINE doesn't support des3 or any other ciphers or digests. It only has support for RSA, DSA, and DH. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL with Security Processor
On October 21, 2004 07:55 pm, Gurpreet Grewal wrote: Does that mean for encrypt and decrypt with RSA? or is it true for private/public key generation too? I am asking this question coz I tried it with the private/public key generation commands but didnt see much a performance difference in key generation. No, the 'ubsec' engine is only used for accelerating key operations at run-time, it doesn't generate keys for you. NB, regarding the term accelerating, you may actually find the card performs few key-operations per second than a decent host machine - however the theory goes that this leaves your host machine's processor free of the overhead so that it can do other things. However, I won't be drawn into debating the merits one way or the other ... :-) Regards, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: compiling openssl-SNAP-20040910.tar.gz on OS X
On September 27, 2004 02:02 pm, Richard Levitte - VMS Whacker wrote: In message [EMAIL PROTECTED] on Mon, 27 Sep 2004 13:41:18 -0400, Geoff Thorpe [EMAIL PROTECTED] said: geoff Well eng_padlock.c is #ifdef'd out unless we are on a geoff supported platform, so I don't think portability of the geoff include is quite so difficult. It'd be safe to include alloca.h geoff I think. OTOH, we'd ideally move all includes inside the geoff preprocessor checks to avoid problems on platforms that don't geoff support this engine, but we need to include certain headers to geoff perform those preprocessor checks - so I'd put alloca.h and any geoff other header unrelated to the preprocessor checks down inside geoff the #ifdef-protected code. Something like the attached patch? Quick reaction without checking the code properly. No, I think the issue is that further down there is other preprocessor logic that checks whether the compiler is gcc, whether it's 386-only, [etc]. The inclusion of headers like alloca.h where pure portability can't be ensured (and isn't desired anyway) should probably be moved down into there. The OPENSSL_NO_*** checks you moved are related to whether the engine(s) are Configured out of the build rather than whether the code itself decides what (and what not) to compile. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: compiling openssl-SNAP-20040910.tar.gz on OS X
On September 27, 2004 03:20 pm, Richard Levitte - VMS Whacker wrote: In message [EMAIL PROTECTED] on Mon, 27 Sep 2004 14:33:32 -0400, Geoff Thorpe [EMAIL PROTECTED] said: geoff Quick reaction without checking the code properly. No, I think geoff the issue is that further down there is other preprocessor geoff logic that checks whether the compiler is gcc, whether it's geoff 386-only, [etc]. Ah, like this then... Note that I removed the VMS-specific part, as it's definitely not needed at that level. Cool, looks fine to me - though I can't really verify much of course (unless someone has spare VIA machines to throw around :-). I've CC'd the author so he can take a glance over this. BTW, on a related note - the logic that decides whether this code gets built or not (independently of whether it will operate at run-time due to autodetection) hinges off OPENSSL_NO_INLINE_ASM. So debug builds will typically bypass this engine, which means it doesn't get debugged ... (this is how I missed the basic need for an include of rand.h that was breaking compilation for others - though that in turn was probably only visible to pure and clean souls who set NO_DEPRECATED). Anyway, I don't know if anyone's got any bright ideas, but it seems like there should be a better way to handle this setting - eg. splitting ***_INLINE_ASM into stuff that is an optimisation of a C interface (so a debug build would choose not to have it) and stuff that is only available as assembly (such as padlock's autodetection). Thoughts? Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: compiling openssl-SNAP-20040910.tar.gz on OS X
On September 27, 2004 06:01 pm, Richard Levitte - VMS Whacker wrote: In message [EMAIL PROTECTED] on Mon, 27 Sep [...] geoff Anyway, I don't know if anyone's got any bright ideas, but it geoff seems like there should be a better way to handle this geoff setting - eg. splitting ***_INLINE_ASM into stuff that is an geoff optimisation of a C interface (so a debug build would choose geoff not to have it) and stuff that is only available as assembly geoff (such as padlock's autodetection). Thoughts? Hmm, you mean that we'd have more dummy code to satisfy the build? I'm not sure what you mean, but the intent of my comment had been that I use my debugging configuration target to improve the coverage of compile-time and run-time sanity-checks on this code (ie. at the expense of debugging optimised assembly). Until now NO_INLINE_ASM has been a way of using pure C implementations instead of assembly-optimised versions, but in the padlock case it causes all the code to be bypassed because a small element of it is necessarily in assembly. Hence I miss bugs, which is the exact opposite of my reason for using debugging options. Mind you, I'm not terribly fussed at this point, just thought I'd throw the question out there in case anyone had any ideas. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #926]
Sorry for the delay, this ticket slipped through the net, and Nils Larsch recently brought it to my attention. Vadim Fedukovich wrote: [snip] I'm particular interested exactly why it is believed that gcd(p-1,e)==1 and gcd(q-1,e)==1 (according to comments in the source). [snip] The gcd() check is redundant because p and q are already generated such that gcd([p|q]-1,e) is 1. As 'e' is prime, this amounts to stating ([p|q]-1) not divisible by 'e'. As neither is divisible by e, neither will (p-1)(q-1) be. Thanks for keeping an eye open, I'm cleaning out the commented code and the misleading commentry. -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: Any 386 users?
On July 28, 2004 06:24 pm, Andy Polyakov wrote: Now I use the same engine in both trees (only with different filenames) and would like to continue doing so. Or is the question if we can *keep* compatibility for all ethernity? Well, I can't make any promises and I can tell that engine interface is not exactly my domain/cup of tea. Probably Geoff can answer this question... A. Well you can probably do this using precompiler magic - there are engine versioning symbols you could steal for this purpose. Note however that once you and Andy are happy with the arch side of things and we've put this into CVS, there's no point maintaining the cvs version for old versions, nor would there be any point keeping the code on your web-site maintained for 0.9.7, so things could be simpler in both cases. (BTW the same sort of comment applies to the indenting style, for example.) I suggest that you and Andy get happy with the gritty stuff, and then I'll help with the rest. (We should probably at that point also put a contrib link to your site if it's not already there to cover people wanting a version backfitted to 0.9.7.) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: key compromise with memory debugger possilbe ?
On July 23, 2004 02:42 am, Oliver Welter wrote: We use openssl to en/decrypt data with 3des - is it possible to retrieve the used key while running a de/encryption via a memory debugger or something similar ? Are there any preventions against such attacks or has noone ever thought about such an attack ? I would appreciate any hints on related studys, documents, etc... They all say more or less the same thing, even in the ncipher inside(tm) research that was quoted somewhere else in the thread, which is - if your threat-model includes access to the memory of the running process, then you're essentially hosed. Playing hide and seek with your key material or cleartext inside memory is a losing battle. Those who can do as they wish with your process memory can presumably track your code too, and so any hide-and-seek logic can be laid bare given sufficient motivation on the part of the attacker. This is (fortunately:-) the inherent lost cause of secure media - if everything required to decrypt a secure data stream into something that will make sense to our ears is contained in a system under my control, I can sandbox it, capture it, watch it, scan it, hack it, change it, as well as any other verbs you can think of pertaining to reverse engineering. The solution (if it's a problem that needs one) is to ensure that this logic doesn't execute in an environment under my control, in which case complexity arguments about how hard it is to break are pointless. The problem reduces to a boolean. If your perceived threats include network attacks, rogue processes, or multi-user systems, then privilege separation on top of a solid (and maintained) OS is your best hope. Ie. if user beenhacked can't attach to a process running as hasntbeenhacked, then the memory scanning issue goes away. If he *can* attach to such a process, then hasntbeenhacked has been hacked after all. This property of privilege separation helps you build up diverse complex systems with security boundaries between the components (the same argument used to flog HKM hardware), and yet also explains why media/content distributors will continue to lose their rights jihad against system owners. IIRC, Bruce Schneier had another spiel on the technical futility of DRM in a recent cryptogram, which you might find useful w.r.t. technical arguments. The subject has been debated endlessly on numerous forums, you shouldn't have difficulty dredging up more commentary. But w.r.t. openssl, nothing out of the ordinary should be expected with respect to openssl's protection against these issues - it's not a question about correctness of the implementation (which *does* relate to openssl) but rather of the system and run-time environment (a probem it would be futile for openssl to address). It's a piece of rope, you can put it to any use you know a knot for ... Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Any 386 users?
Hi all, I'm looking for anyone who's using openssl on 386 (and perhaps 486) chip models that might be able to verify a patch before it gets applied? The RT ticket about this is at; http://www.aet.tu-cottbus.de/rt2/Ticket/Display.html?id=889 and the code it refers to is at; http://www.logix.cz/michal/devel/padlock/ To verify this adequately, please take a recent CVS snapshot (or checkout) of openssl (it. not any 0.9.* stable branches) and compile and run something first (eg. openssl speed). Then after applying the openssl-0.9.8cvs-padlock.diff patch from Michal's site, please compile and run again. It would also be useful to verify that the hook into Michal's detection code is being run, so please insert a printf into the padlock_insn_cpuid_available() function in crypto/engine/eng_padlock.c so you can see if it's being called when running programs. Note, this could should be bypassed except when gcc is being used, so VC++ is unlikely to be informative (though using cygwin tools might be interesting). Any feedback on this would be most useful, as this feature is only supported on certain x86 chips and so we can't commit unless the detection code is seen to be safe on the remaining x86 chips, particularly older ones without cpuid. Regards, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #889] [PATCH] Support for VIA PadLock ACE (fwd)
[EMAIL PROTECTED] - Sat Jul 10 01:26:23 2004]: AFAIK this way of checking for CPUID is 386-safe as well. Cool, thanks for the update. I've just sent a post to openssl-devel to ask for some assistance verifying the detection code under 386/486 systems. If I can get some confirmation back about this, I'll be able to proceed. Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #889] [PATCH] Support for VIA PadLock ACE (fwd)
[EMAIL PROTECTED] - Fri Jul 9 01:06:08 2004]: I finally decided to make the engine equal to other engines and build as a shared library. My next step will be extending the OpenSSL_config() (if necessary) to allow fine-grained loading of specified engines for apps that support it. But this will go to a new RT item. OK, but given that your engine can be made 486-proof, in principle I'd have no objection to placing the engine directly into the core code a la the BSD cryptodev implementation. Though perhaps starting off with it as a conventional plug-in would be best, then we could review bundling it into libcrypto later on (a la BSD's cryptodev). Well, on http://www.logix.cz/michal/devel/padlock/ you can find an updated version of the PadLock engine module for OpenSSL-CVS. It now contains all features as does the OpenSSL-0.9.7 version, i.e. AES in all keylengths and RNG. Cool. Incorporated is a check for CPUID instruction availability to be safe on 486 machines (does really somebody still use them?) We have to assume that it's possible, yes. People are using openssl in a variety of environments, and though 486s are rare on the desktop now they're far from extinct in the embedded world. There might even be one or two 386s still roaming wild ... (which poses the question; what about 386-safe?) and it only builds with GCC, otherwise an empty module with only a dummy ENGINE_load_padlock() is compiled. I don't see a reason to limit this module to Linux-only, IMHO bounding it to GCC-only should be enough. Well in theory you could allow it to be compiler-agnostic too, but the issue is how to ensure the code won't fail compilation or execution for any supported combination. If checking for gcc ensures the code won't burp on anything else, that's fine. Anyone with special needs can always look at extending (and testing) it for other compilers later. Would you accept it to the CVS in this form or should I change something? I'll try to find some time soon to comb through the code. If you haven't heard back from me in a week or two, send me a nag in private email. Thanks for carrying out the updates. Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #889] [PATCH] Support for VIA PadLock ACE (fwd)
Hi, Getting back to this as I just finished an updated version of the PadLock engine with support for AES192, AES256 and RNG. For now it is only for 0.9.7 (see http://www.logix.cz/michal/devel/padlock/#openssl) and before porting it to CVS I want to clarify how... Thanks for the update, this looks cool. Not necessarily - I am still unsure of whether we should attempt to automatically probe shared-lib engines during ENGINE_load_builtin_engines(). Right now, no shared-lib engines will be available unless they're loaded or specified by id, so you'd automatically fall back to the default software implementations. Hmm, that's unfortunate. May OPENSSL_config() have any influence on it? I haven't looked at the possibilities of configuring engines through openssl.cnf yet, but if something like... Yes indeed, this is exactly what that stuff is for. Note however that the application still needs to call that function, so you still have the problem that only enabled applications will give you any hook to ensure the engine is loaded. However it has the benefit of being a single hook that can allow a config file to control all potential functionalities, rather than apps having to add ENGINE_load_***(), (etc) calls on a per-case basis. On the other hand, it might be desirable to build openssl libs that perform this autoconfiguration every time they're used, rather than just for applications that see fit to call it. This came up in a chat with Steve some time back (he was the one that did the config-module work), though I don't know whether anything ended up in there to let you hard-code config-loading into the openssl libs. If not, it was probably just for lack of a conviction of whether it was worth doing, or for lack of a preferred mechanism/semantic for it. Steve, can you comment? BTW What's the reason for not loading engines by default but only explicitly through ENGINE_...() functions? The startup speed? Any other risks? Ah, well there's a couple of reasons. One was that, originally, it was desirable that the ENGINE code wasn't statically linked into any application that didn't make ENGINE API calls. Ie. not interferring in any legacy scenarios. The second is that the ENGINE code at the time would statically link in all the crypto code for *all* functionality supported by ENGINE. Ie. if you used ENGINE for RSA, you'd get all the DSA, RAND, (etc) baggage anyway. The second point is no longer an issue and the first is probably less relevant now that we're a couple of versions further on. However I'd probably need to take another cleanup-stick to the engine code before considering any auto-loading option there, and there also the issue of deciding how the auto-loading would happen: it's less than obvious in some cases (first-use-initialisation hacks can have some non-trivial drawbacks). But with the above notes aside, the configuration-file stuff is perhaps a better and more general nut to crack. There are also some similar issues, eg. if the config file support *could* cause a DSA key to be loaded if the config file specified to do so, then the library would necessarily link to the DSA functionality whether it was used in the config file or not (unless you were careful about using RTLD_LAZY, which still isn't a solution for some platforms). Input would no doubt be welcome on these types of issues, particular from Suse hackers or any other distribution gurus our there. I could go with compiling the padlock engine statically a la cryptodev iff we can ensure that it is only compiled on supported platforms. On non-i386 ENGINE_load_padlock() immediately returns and the rest is also #ifdef'ed and compiles only on i386. OK, one quick observation right off - you'd probably need to home this in on linux-x86, as non-linux platforms (and/or compilers) would probably croak on some or all of this. Particularly with the use of __func__. :-) OTOH the startup check for PadLock availability is really quick and simple - only two 'cpuid' instructions and parsing their output. No need to dlopen() other libraries, no need to wait for hardware initialization, etc. IMHO it could be safely compiled in... Can be this be made 486-safe? If so, and the code was made friendlier to non-linux x86 kernels (and non-gcc compilers), we'd be up and running. Is there anything else specific about padlock-capable systems that could be used to set specific targets for this stuff? Sorry, what do you mean by specific target? The Configure script (invoked by ./config if you choose to go that way) uses different targets for different compiler/OS/CPU combinations. And whatever else is required to build without using autoconf. Run; perl Configure -? Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED
Re: question on static/dynamic linking engines
On June 24, 2004 12:49 pm, Kevin Stefanik wrote: [snip] However I'm pretty confident the 0.9.7 use of ERR_get_implementation() is bogus. [snip] Linking the openssl engine to libcrypto.so shared library for 0.9.8 works fine as far as I've been able to test. Cool, so this is just an issue of finding a fix for 0.9.7 to avoid ERR_get_implementation. duck object=rancid tomatoesOr we could ignore this problem altogether./duck :-) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: question on static/dynamic linking engines
On June 16, 2004 12:46 pm, Richard Levitte - VMS Whacker wrote: kstef I think we can make do with a less involved fix, actually, by kstef just backing out the conditional if the engine still _requires_ kstef its own copy of the libcrypto code, or, preferably, just kstef linking to libcrypto dynamically if the earlier requirement no kstef longer holds. I just need an expert's best guess and I'll be kstef out of your hair. Backing out is not an option. The ENGINE_get_static_state() check is there for a very specific reason that hits most Unixly users, at least if the engine is linked with the exact dynamic library that loads it. Since that's a fairly plausible scenario, I think it would be a bad move not to make any checks of the sort. Indeed. However one problem with merging ENGINE_get_static_state() to 0.9.6-stable is that it requires a new exported API symbol in openssl. Specifically, building dynamic engines uses an engine.h macro that calls this (newly introduced) function, so we'd have to bump the magic version stuff to prevent weird linker freakouts and give more meaningful errors when the loading instance doesn't have that new function. I'm not against merging it, as such, but wanted to raise the issue. The alternative is to make the check draconian so at least the bug disappears - ie. if the error callbacks in the engine lib's context don't match the callbacks in the crypto lib's context, don't try to alter them (it'll fail anyway), just refuse the load. This would probably break dynamic engine capability on one or two strange platforms I guess, but then again, how could they have been working in this form anyway? If the engine lib resolves all libcrypto dependencies against the loading instance, things are fine, always. If the engine lib has its own copies of everything or resolves against a different instance(/version) than the loading context, it won't work. I'm not sure we can fix this in a stable-branch-friendly way, as it necessarily requires an API change, because it deals with libraries people build *using* the API (ie. they build engine libs), and that in turn opens the interop questions stable branches normally don't raise. Hmmm. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: question on static/dynamic linking engines
On June 16, 2004 07:48 pm, Richard Levitte - VMS Whacker wrote: geoff Indeed. However one problem with merging geoff ENGINE_get_static_state() to 0.9.6-stable is that it requires a geoff new exported API symbol in openssl. Well, I don't see that as a problem, since we don't have support for dynamic engines *at* *all* in 0.9.6... I assume you really meant to say 0.9.7-stable, and I see that as less of a problem on a general level, since there's no newer OpenSSL version than that. Well there will be, and I think the issue is that we have got to a point where we speak of openssl variants as being 0.9.6, 0.9.7, etc. Changes within a stable branch that affect interoperability, or worse still, configuration of dependant code, can be minefields. geoff Specifically, building dynamic engines uses an engine.h macro geoff that calls this (newly introduced) function, so we'd have to geoff bump the magic version stuff to prevent weird linker freakouts geoff and give more meaningful errors when the loading instance geoff doesn't have that new function. I'm not against merging it, as geoff such, but wanted to raise the issue. We *could* check if the magic version is the former level, and then not check at all (i.e. have some backward compatibility cruft), but if it's the newer level, we call ENGINE_get_static_state(). Of course, that means we need to keep two variants of struct st_dynamic_fns and do a bit of casting magic... I'm not sure what to say, I don't really have the variety of architectures to adequately test these combinations out. And given how complex this could get, it certainly will need decent testing to avoid creating a cure worse than the problem. I still wonder whether this can't be better settled in the version check itself? geoff The alternative is to make the check draconian so at least the geoff bug disappears - ie. if the error callbacks in the engine lib's geoff context don't match the callbacks in the crypto lib's context, geoff don't try to alter them (it'll fail anyway), just refuse the geoff load. This would probably break dynamic engine capability on geoff one or two strange platforms I guess, but then again, how could geoff they have been working in this form anyway? True, they probably worked better before I use ERR_get_implementation() to check against fns-err_fns. At the time I made that change, I didn't test on VMS for some reason, or I would have caught it at that time... Oh well, this sort of architecture-picky stuff is a nightmare. But just out of interest - what would happen if we ensured ERR_get_implementation() returned NULL if no callbacks were registered instead of automatically setting them to defaults? Of course the other ERR functionality would need to continue to have the set on first use behaviour, but the ERR_get_implementation() API itself - could that survive being altered to be read-only? If so, wouldn't that help? I know I know, questions questions questions. I've got a cold, that's all I can manage for now. :-) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: bug in stack.c - [SOLVED]
On June 15, 2004 08:16 am, Peter Poeml wrote: Kent Yoder, who maintains the IBMCA patch and whom I had contacted meanwhile, did find the reason: | Found a bug here in the IBMCA engine patch (apr29). If the device | driver cannot be loaded, ibmca_init()'s error patch calls | DSO_free(), but doesn't reset ibmca_dso=NULL. The next time openssl | call ibmca_init, ibmca_dso!=NULL, so we enter an error path, which | re-call's DSO_free on the already freed ibmca_dso variable. Ahh, cool. Glad you got to the bottom of it. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: question on static/dynamic linking engines
On June 14, 2004 10:20 am, Kevin Stefanik wrote:
[snip]
#define IMPLEMENT_DYNAMIC_BIND_FN(fn) \
int bind_engine(ENGINE *e, const char *id, const dynamic_fns *fns) { \
if (ERR_get_implementation() != fns-err_fns) \
{ \
if(!CRYPTO_set_mem_functions(fns-mem_fns.malloc_cb, \
fns-mem_fns.realloc_cb, fns-mem_fns.free_cb)) \
return 0; \
CRYPTO_set_locking_callback(fns-lock_fns.lock_locking_cb); \
CRYPTO_set_add_lock_callback(fns-lock_fns.lock_add_lock_cb); \
CRYPTO_set_dynlock_create_callback(fns-lock_fns.dynlock_create_cb);
\ CRYPTO_set_dynlock_lock_callback(fns-lock_fns.dynlock_lock_cb); \
CRYPTO_set_dynlock_destroy_callback(fns-lock_fns.dynlock_destroy_cb);
\ if(!CRYPTO_set_ex_data_implementation(fns-ex_data_fns)) \
return 0; \
if(!ERR_set_implementation(fns-err_fns)) return 0; \
} \
if(!fn(e,id)) return 0; \
return 1; }
If the library is statically compiled, then the ERR_get_implementation
will initialize the static err_fns to err_defaults. The
ERR_set_implementation will then report an error because that static
err_fns is not null.
But ERR_set_implementation() should not be called because
ERR_get_implementation() should equal fns-err_fns in the static case and
so the code path should skip straight to the call to fn(). However ...
I think this effectively makes statically linked engines impossible -
which I really don't mind. Can we just always use shared engines as of
0.9.7d?
Actually it seems to me to be the other way around. It appears that this
has probably broken the use of shared-lib engines on any platforms where
the loaded shared-lib has its own copy of libcrypto functions/state. What
exactly have you been seeing to make you believe it's the other way
round?
Thanks for the head's up, we'll need to look into this - but I'd also like
to know why you're seeing (or think you're seeing) errors in the
statically-compiled case? Can you alter the macro to display the values
of ERR_get_implementation() and fns-err_fns before that if(){} branch,
as those values should match in the static case.
Cheers,
Geoff
--
Geoff Thorpe
[EMAIL PROTECTED]
http://www.geoffthorpe.net/
__
OpenSSL Project http://www.openssl.org
Development Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
Re: question on static/dynamic linking engines
On June 14, 2004 12:00 pm, Kevin Stefanik wrote: I just realized that we may not have been discussing the same issue. When I was referring to dynamically or statically linked engines, I was referring to how the engines were linked to libcrypto. In all cases, we're discussing a dynamic engine contained in a shared library, so I think we agree on the problem. Ah, phew. Zoltan sent some stuff to me, but I'll need to wait a couple of days until I have time to sift through this. However I'm pretty confident the 0.9.7 use of ERR_get_implementation() is bogus. What results have you had (if any) against cvs snapshots? As you've observed, there is a better mechanism in place there and depending on what I find when I get a chance to dig into Zoltan's mail, I may elect to just merge that aspect of the engine code back to 0.9.7 to address the problem. À suivre ... Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #889] [PATCH] Support for VIA PadLock ACE (fwd)
OK, attached is a patch against CVS mainline. For me it works in both static and shared versions. Cool, I'll try to take a look in the next few days. I didn't get it, sorry. Should I make the Padlock support always static as is the cryptodev? The PadLock is available on new VIA Nehemiah CPUs and if present (and successfully detected by the Padlock engine initialization routine) it should be used, yes. But that's the same situation as with other engines, isn't it? Not necessarily - I am still unsure of whether we should attempt to automatically probe shared-lib engines during ENGINE_load_builtin_engines(). Right now, no shared-lib engines will be available unless they're loaded or specified by id, so you'd automatically fall back to the default software implementations. I could go with compiling the padlock engine statically a la cryptodev iff we can ensure that it is only compiled on supported platforms. The reason we can compile-in cryptodev is that it becomes a NOP except on openbsd and any other platform that supports cryptodev or is prepared to accept the additional overhead even if it isn't supported. I don't think a large class of x86 platform targets would accept such a situation with the padlock stuff given that the majority of run-time environments wouldn't be able to use it. Is there anything else specific about padlock-capable systems that could be used to set specific targets for this stuff? Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #889] [PATCH] Support for VIA PadLock ACE (fwd)
[EMAIL PROTECTED] - Fri Jun 4 11:13:39 2004]: I have written new engine module for VIA PadLock ACE. http://www.logix.cz/michal/devel/padlock/#openssl This is quite cool. However to proceed, you'd need to port the engine to the head of CVS, which has the engines relocated from ./crypto/engine/ into ./engines/, besides a few other changes too (eg. check out how the other engines have changed since 0.9.7 for an indication, particularly with respect to building shared-libs). That said, the OpenBSD-style cryptodev engine is statically built-in because it only applies for supported platforms on which it is (presumably) supposed to be the default. Perhaps the VIA chipset support is a similar thing? (Though whether openssl's Configure has, or can have, targets that would cover this remains to be seen.) Anyway, please take a look at a recent CVS snapshot from the website and resubmit your code for that. 0.9.7-stable is maintenance-only unless under exceptional circumstances, so 0.9.8-dev (HEAD) is where this needs to go. Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: [CVS] OpenSSL: openssl/ Configure Makefile.org openssl/crypto/engine/ ...
Yo, On June 1, 2004 02:34 am, Richard Levitte - VMS Whacker wrote: In message [EMAIL PROTECTED] on Tue, 1 Jun 2004 05:18:59 +0200 (CEST), Geoff Thorpe [EMAIL PROTECTED] said: geoff Log: geoff This fixes the installation target for dynamic engines, geoff which was trying to install to a different location than it geoff had created. (BTW, VMS will need a matching fix in geoff eng_list.c.) Note, these aren't ssl-specific, so I'm geoff putting engines/ into the libs directory rather than at geoff the --prefix level or inside ssl/. I have no problems with the change in itself, but I must protest against the reason you're stating. In that case, the stuff installed in bin/ should probably be moved off, since it isn't SSL-specific either. After all, the whole installation tree lives under ssl/, at least by default, and that just expresses what software package it comes from, nothing more (i.e. those are OpenSSL-specific files, rather than SSL-specific). Perhaps it wasn't expressed as it might have been. I was referring to the general --prefix=foo case where we get an install tree at foo containing the sub-directories; foo/bin/ foo/lib/ foo/engines/ ... foo/ssl/ This is the install tree that would be created, but the installation would then try to place engine libs in foo/ssl/engines/ (which doesn't exist). Hence my comment about ssl-specific. I organised it so that the engine libs go in foo/lib/engines/, and foo/engines is no longer created. The stuff in foo/ssl/ should, AFAICS, be ssl-specific admin and that doesn't include engine libraries. I'm guessing the confusion comes from the fact that if --prefix (and or --openssldir, modulo some combinatorial logic I'd need more coffee for) isn't specificied, then --prefix becomes /usr/local/ssl, so that we end up with everything under an ssl directory (where the ssl is now actually part of the prefix - presumably openssl is/was too long). This includes the /usr/local/ssl/bin example you mentioned. That wasn't what I meant by ssl-specific. In the default case, there is *no* ssl-specific sub-directory - it's contents move up one level (OPENSSLDIR==INSTALLTOP), so ssl is part of the installation path and contains everything rather than just ssl-specific files ... if you would like to join me in gibbering in a deranged fashion at this point, please feel free. Wibble. :-) Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: How BN_bin2bn works
Such a question will have a wider and more appropriate pool of responders on the openssl-users list than openssl-dev. Regards, Geoff On May 3, 2004 10:57 am, Carlos Cabañero wrote: I'm giving a look to the code of OpenSSL as I'm very interested in RSA encryptation / decryp. specially for a subject called Discrete Math. My [snip] -- Geoff Thorpe [EMAIL PROTECTED] http://www.openssl.org/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #874] [Fwd: Bug#243509: openssl: genrsa get crasy with small key size]
[I'd forgotten to send this to RT] OK, I took a look and the problem does not appear to be BN_generate_prime_ex() but the fact that, for small modulus sizes, the generated primes are always identical and the rsa keygen keeps looping in the hope that it eventually gets something unique. I've attached a patch that works for me, but it'd be good if you could verify you see the same behaviour. For example, I can generate RSA keys of 31 bits (meaning the primes are 15/16 bits each) but it fails for 30 bits. Do you see the same thing? Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #668] [PATCH] Fall back to software if nCipher hardware fails
Could you please adapt the patch for the head of CVS (nightly snapshots can be found on the ftp server) and resubmit? If this is not possible, let me know and perhaps I'll find a moment to have a poke at it. NB: in 0.9.8-dev, the engine implementations are to be found in ./engines/ rather than ./crypto/engine/. I doubt it would require much hacking to port this, but it will require testing the fallback scenarios with the hardware (which I can't do). Please also double check that the shared-lib engine build works too (./config shared). Colin, did you get a chance to update your patch? Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #695] [PATCH] DSO: dlfcn support for MacOS X
Antti, Any news on an update for your patch? Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #867] [PATCH] Bignum exponent in RSA_generate_key_ex (instead of long)
[geoff - Thu Apr 1 18:13:51 2004]: Can you please resubmit the patch as an attachment rather than inlining it? The patch gets word-wrapped otherwise and is unusable. Jelte, Any news on this? If you could update the request tracker ticket with a clean (non-wrapped) copy of your patch, I'd be able to take this further. Cheers, Geoff -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #867] [PATCH] Bignum exponent in RSA_generate_key_ex (instead of long)
Looks mostly ok. It's missing a RSA_METHOD change and has a redundant BN_new() (memory leak), but nothing major. I'll tidy it up and commit it in a sec, thanks. -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: [openssl.org #874] [Fwd: Bug#243509: openssl: genrsa get crasy with small key size]
On April 21, 2004 04:49 am, Richard Levitte via RT wrote: I'm a little hesitant to do this, however, as it has a strong smell of quick and dirty hack (which is about the same smell as I imagine a pair of socks you have worn for a week would while wearing sneackers 80% of the time...). Comments? How about wearing sandals? :-) We should find where/why things spin out of control and improve the handling to either work or bail out gracefully. I haven't the time to analyse anything for the next couple of days, but might be able to take a look later if someone else hasn't already nailed it by then. Quick question: is this occuring in the head of CVS or just release branches? Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: bug in stack.c
On April 21, 2004 08:26 am, Richard Koenning wrote: [EMAIL PROTECTED] wrote: I've found that in stack.c there is no check about index value. Results a crash if index=MIN_NODES index=st-num or unpredictable result if indexMIN_NODES index=st-num See the message with the subject Re: [PATCH] Missing loop end check in crypto/engine/eng_table.c from Geoff Thorpe. In short: the bug is already corrected in the CVS. Actually, only the sk_value() case was addressed, so the report is valid w.r.t. sk_set(). I'm just checking for other cases and will commit shortly. Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: [openssl.org #874] [Fwd: Bug#243509: openssl: genrsa get crasy with small key size]
On April 21, 2004 04:49 am, Richard Levitte via RT wrote:
[EMAIL PROTECTED] - Wed Apr 21 08:27:34 2004]:
Interesting bug. I guess that noone ever thought of testing for
ridiculously small moduli... A quick solution would be to have the
following at the beginning of BN_generate_prime_ex():
/* We know from experience that this algorithms barfs
endlessly at 30 or less bits */
if (bits 31) return 0;
OK, I took a look and the problem does not appear to be
BN_generate_prime_ex() but the fact that, for small modulus sizes, the
generated primes are always identical and the rsa keygen keeps looping in
the hope that it eventually gets something unique. I've attached a patch
that works for me, but it'd be good if you could verify you see the same
behaviour. For example, I can generate RSA keys of 31 bits (meaning the
primes are 15/16 bits each) but it fails for 30 bits. Do you see the same
thing?
Cheers,
Geoff
--
Geoff Thorpe
[EMAIL PROTECTED]
http://www.openssl.org/
Index: crypto/rsa/rsa_gen.c
===
RCS file: /e/openssl/cvs/openssl/crypto/rsa/rsa_gen.c,v
retrieving revision 1.12
diff -u -r1.12 rsa_gen.c
--- crypto/rsa/rsa_gen.c 29 Oct 2003 20:24:12 - 1.12
+++ crypto/rsa/rsa_gen.c 21 Apr 2004 16:06:54 -
@@ -140,11 +140,24 @@
goto err;
for (;;)
{
- if(!BN_generate_prime_ex(rsa-q, bitsq, 0, NULL, NULL, cb))
+ /* When generating ridiculously small keys, we can get stuck
+ * continually regenerating the same prime values. Check for
+ * this and bail if it happens 3 times. */
+ unsigned int degenerate = 0;
+ do
+ {
+ if(!BN_generate_prime_ex(rsa-q, bitsq, 0, NULL, NULL, cb))
+goto err;
+ } while((BN_cmp(rsa-p, rsa-q) == 0) (++degenerate 3));
+ if(degenerate == 3)
+ {
+ ok = 0; /* we set our own err */
+ RSAerr(RSA_F_RSA_GENERATE_KEY,RSA_R_KEY_SIZE_TOO_SMALL);
goto err;
+ }
if (!BN_sub(r2,rsa-q,BN_value_one())) goto err;
if (!BN_gcd(r1,r2,rsa-e,ctx)) goto err;
- if (BN_is_one(r1) (BN_cmp(rsa-p,rsa-q) != 0))
+ if (BN_is_one(r1))
break;
if(!BN_GENCB_call(cb, 2, n++))
goto err;
Re: [PATCH] Missing loop end check in crypto/engine/eng_table.c
On April 8, 2004 11:45 am, Richard Koenning wrote: In crypto/engine/eng_table.c the size of the accessed stack is not checked, so regularly non-existent stack elements are tried to access. Surprisingly often this does not crash (and all seems to function perfectly), but sometimes it does, as expected. My problems in connection with stunnel can be put down to this bug, and i think the problems observed by Robert Urban and Farkas Zsolt too. Not only is this true, but it was also recently fixed in CVS (stable branches included). The problem has been solved by making sk_value() return NULL for out-of-range indexes, making the stack API a little more robust and, by no coincidence, enforcing a behaviour I'd been relying on in eng_table.c that had never actually existed. gulp Cheers, Geoff -- Geoff Thorpe [EMAIL PROTECTED] http://www.geoffthorpe.net/ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
[openssl.org #867] [PATCH] Bignum exponent in RSA_generate_key_ex (instead of long)
Can you please resubmit the patch as an attachment rather than inlining it? The patch gets word-wrapped otherwise and is unusable. -- Geoff Thorpe, RT/openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
