Re: EVP macros for AES
On Wed, Mar 06, 2002 at 10:13:55AM +1300, Geoff Thorpe wrote: Well yes, but as Richard was explaining - static linking with a libcrypto.a full of stuff is not that different to linking with a custom-built ./crypto/des/-only libdes.a - your linked executable should just contain the DES bits either way. There's no particular win, that I can see anyway, to not *building* the rest of libcrypto ... any respectable linker will only drag in object files from the archive that are required. True, but I've had people tell me that this isn't as easy as it sounds in particular development environments, such as when building Windows device drivers. Apparently, you have to do a lot of work to convert standard make dependencies to whatever process they use, otherwise you end up recompiling OpenSSL in toto almost every time. I don't know enough about the circumstances (not really caring so much about Windows development). All I know is that it's been something I've had people ask about on several occasions. John __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
I'm not sure why you need to yank out source when it's just as easy to link with libcrypto and make sure you only use the specific algorithms. In such a case, one should avoid using things like EVP_get_cipherbyname() since that requires that all compiled algorithms be linked in. I was thinking more of environments where dynamic linking doesn't exist, or everything you need has to be stored in a limited amount of space like a floppy or some sort of solid-state memory device. This is somewhat common. A good modular library will only link in the .o's that are necessary to build a usable program. So it doesn't matter if you have libjpeg combined with libssl, it won't increase the size of your program any (except maybe for header info from the lib that could be stripped out of the binary). So if you cross-compile on your PC and upload the binary to the smaller device in question, there shouldn't be any problem. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Hi, On Tuesday 05 March 2002 08:12, John Viega wrote: On Mon, Mar 04, 2002 at 02:48:46PM +0100, Richard Levitte - VMS Whacker wrote: I'm not sure why you need to yank out source when it's just as easy to link with libcrypto and make sure you only use the specific algorithms. In such a case, one should avoid using things like EVP_get_cipherbyname() since that requires that all compiled algorithms be linked in. I was thinking more of environments where dynamic linking doesn't exist, or everything you need has to be stored in a limited amount of space like a floppy or some sort of solid-state memory device. This is somewhat common. Well yes, but as Richard was explaining - static linking with a libcrypto.a full of stuff is not that different to linking with a custom-built ./crypto/des/-only libdes.a - your linked executable should just contain the DES bits either way. There's no particular win, that I can see anyway, to not *building* the rest of libcrypto ... any respectable linker will only drag in object files from the archive that are required. Off the top of my head, I don't see why a well-organized library couldn't have well-compartmentalized algorithms that can be easily ripped out, yet are all accessible through a generic interface such as EVP. That is, I don't see any disadvantage to such an approach, Well, EVP *should* work like that now, though I fear that some C files in crypto/evp/ could have links across into other stuff (eg. perhaps RSA, DSA, etc). Without trying it I can't be sure. But if you don't call OpenSSL_add_all_[algorithms|ciphers|digests], and instead simply call EVP_add_[cipher|digest] for those algo's you want, then EVP should work as you suggest. especially if there's no fear of people breaking out little parts and maintaining them separately. This is what a linker (for static-linking anyhow) does ... it rips object files out of an archive on an as-needed basis ... why is it so important to actually build miniature archives? Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Sorry to be a couple of weeks behind on the discussion. Too much going on. On Fri, Feb 15, 2002 at 04:41:10PM -0600, Stephen Sprunk wrote: The current state is EAY legacy. His idea was that one should be able to pick out any of the algorithm directories and create a separate library for them (the old libdes is actually exactly the same as crypto/des/). First of all, do we still think that's necessary, given the wide acceptance of OpenSSL and libcrypto? I think it's a good idea to make algorithms easy to cut out. I very often run into people who want to yank out just the pieces of OpenSSL they need, particularly people who are working in embedded environments with memory constraints, or people working at a device driver level. John __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Thus spake Dr S N Henson: Maybe. It would be good to the the CFB and OFB modes working properly in general for other numbers of bits. The code for this is trivial; define me an API and I'll write the code underneath. I thought about moving the whole cipher mode handling to the EVP layer and trimming down the individual block ciphers to just encrypt/decrypt a single block at one point. However tests suggested that this would have a considerably impact on performance so we're probably stuck with the duplicate mode code for now. I'd like to see exactly where the performace problem was; I can't imagien it'd be statistically significant. There is some duplication in individual cipher modes and some macros might help though there is some variation where some cipher optimize the storing of things like the IV in the most effective internal format. All of these optimizations seem to revolve around endianness problems or char/uint64 representations. I wonder if we could do better by moving some of the mode handling to the assembly language routines and take advantage of some special cases to avoid function call overhead. I tried this for AES and got ~10% speed increase for CBC in about 5 minutes using some generic assembly. Another 17% by throwing some MMX stuff in over my lunch hour. And I haven't even touched the C algorithm itself :) Again, it's not worth doing this for each algorithm/mode, but if we had either a mode API or at least macro sets, it would be. S -- Stephen Sprunk So long as they don't get violent, I want to CCIE #3723 let everyone say what they wish, for I myself have K5SSSalways said exactly what pleased me. --Albert Einstein __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Stephen Sprunk wrote: Thus spake Dr S N Henson: Maybe. It would be good to the the CFB and OFB modes working properly in general for other numbers of bits. The code for this is trivial; define me an API and I'll write the code underneath. Well I was thinking of something almost identical to the variable key length code. Have a call EVP_CIPHER_CTX_set_feedback_bits() or whatever to allow it to be set to a non default value. I thought about moving the whole cipher mode handling to the EVP layer and trimming down the individual block ciphers to just encrypt/decrypt a single block at one point. However tests suggested that this would have a considerably impact on performance so we're probably stuck with the duplicate mode code for now. I'd like to see exactly where the performace problem was; I can't imagien it'd be statistically significant. Well you can get some idea of the overhead using 'openssl speed' and comparing the times for the low level routines and the EVP wrappers. You'd expect some kind of slowdown with fast ciphers like RC4 but the slowdown for things like DES is also significant. I wonder if we could do better by moving some of the mode handling to the assembly language routines and take advantage of some special cases to avoid function call overhead. I tried this for AES and got ~10% speed increase for CBC in about 5 minutes using some generic assembly. Another 17% by throwing some MMX stuff in over my lunch hour. And I haven't even touched the C algorithm itself :) Again, it's not worth doing this for each algorithm/mode, but if we had either a mode API or at least macro sets, it would be. Maybe could be done by adding some perl functions in the perl assembly language generator? Steve. -- Dr Stephen N. Henson. http://www.drh-consultancy.demon.co.uk/ Personal Email: [EMAIL PROTECTED] Senior crypto engineer, Gemplus: http://www.gemplus.com/ Core developer of the OpenSSL project: http://www.openssl.org/ Business Email: [EMAIL PROTECTED] PGP key: via homepage. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Thus spake Richard Levitte - VMS Whacker: From: Stephen Sprunk [EMAIL PROTECTED] stephen After implementing CBC, CFB, OFB, and CTR for AES (mostly plagiarized stephen from IDEA), I'm beginning to wonder why we have 40 different stephen implementations of the exact same algorithms in the first place. stephen Couldn't we have a general modes macro set, to which you pass the stephen ECB function of the cipher you're using? The current state is EAY legacy. His idea was that one should be able to pick out any of the algorithm directories and create a separate library for them (the old libdes is actually exactly the same as crypto/des/). First of all, do we still think that's necessary, given the wide acceptance of OpenSSL and libcrypto? In either case, it shouldn't be that difficult to develop a macro set that implements various modes. Pass in some function names, block size, types for casts, and let it generate the boilerplate. Worth doing? S -- Stephen Sprunk So long as they don't get violent, I want to CCIE #3723 let everyone say what they wish, for I myself have K5SSSalways said exactly what pleased me. --Albert Einstein __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Stephen Sprunk wrote: Thus spake Richard Levitte - VMS Whacker: The current state is EAY legacy. His idea was that one should be able to pick out any of the algorithm directories and create a separate library for them (the old libdes is actually exactly the same as crypto/des/). First of all, do we still think that's necessary, given the wide acceptance of OpenSSL and libcrypto? I'm not too bothered by that. We've been advising people to avoid the low level API where possible for a while now. In either case, it shouldn't be that difficult to develop a macro set that implements various modes. Pass in some function names, block size, types for casts, and let it generate the boilerplate. Worth doing? Maybe. It would be good to the the CFB and OFB modes working properly in general for other numbers of bits. I thought about moving the whole cipher mode handling to the EVP layer and trimming down the individual block ciphers to just encrypt/decrypt a single block at one point. However tests suggested that this would have a considerably impact on performance so we're probably stuck with the duplicate mode code for now. There is some duplication in individual cipher modes and some macros might help though there is some variation where some cipher optimize the storing of things like the IV in the most effective internal format. I wonder if we could do better by moving some of the mode handling to the assembly language routines and take advantage of some special cases to avoid function call overhead. Steve. -- Dr Stephen N. Henson. http://www.drh-consultancy.demon.co.uk/ Personal Email: [EMAIL PROTECTED] Senior crypto engineer, Gemplus: http://www.gemplus.com/ Core developer of the OpenSSL project: http://www.openssl.org/ Business Email: [EMAIL PROTECTED] PGP key: via homepage. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
From: Stephen Sprunk [EMAIL PROTECTED] stephen The current state is EAY legacy. His idea was that one should be able stephen to pick out any of the algorithm directories and create a separate stephen library for them (the old libdes is actually exactly the same as stephen crypto/des/). stephen stephen First of all, do we still think that's necessary, given the wide stephen acceptance of OpenSSL and libcrypto? Actually, with the dependence on opensslconf.h, we have already broken that pattern. stephen In either case, it shouldn't be that difficult to develop a macro set stephen that implements various modes. Pass in some function names, block stephen size, types for casts, and let it generate the boilerplate. Worth stephen doing? I've been playing with something like that for a few weeks now. I'll get back about that in a few days. -- Richard Levitte \ Spannvägen 38, II \ [EMAIL PROTECTED] Redakteur@Stacken \ S-168 35 BROMMA \ T: +46-8-26 52 47 \ SWEDEN \ or +46-733-72 88 11 Procurator Odiosus Ex Infernis-- [EMAIL PROTECTED] Member of the OpenSSL development team: http://www.openssl.org/ Software Engineer, GemPlus: http://www.gemplus.com/ Unsolicited commercial email is subject to an archival fee of $400. See http://www.stacken.kth.se/~levitte/mail/ for more info. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Thus spake Dr S N Henson: The reason for the macros was that when the EVP layer was being revised it was an absolute nightmare to change anything. IIRC at one point I had to add an additional field to the EVP_CIPHER structure. I then spent the next couple of hours modifying lots of almost identical EVP_CIPHER definitions spread over zillions of little files. So I rewrote the files to use macros so that any future changes could largely be handled by just changing the macros and the odd exception manually. That's roughly what I figured :) After implementing CBC, CFB, OFB, and CTR for AES (mostly plagiarized from IDEA), I'm beginning to wonder why we have 40 different implementations of the exact same algorithms in the first place. Couldn't we have a general modes macro set, to which you pass the ECB function of the cipher you're using? I've also been at a loss for how to use SSE/MMX registers to pass around keys and blocks, since doing so would grossly violate the current API. This would provide serious performance gains, however. S -- Stephen Sprunk So long as they don't get violent, I want to CCIE #3723 let everyone say what they wish, for I myself have K5SSSalways said exactly what pleased me. --Albert Einstein __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
Thus spake Richard Levitte - VMS Whacker:
Note that this puts a requirement on the algorithm functions to follow
a certain name standard. The expected frunctions are, for a certain
{prefix} (AES in the AES case, I assume :-)):
{prefix}_ecb_encrypt
{prefix}_cbc_encrypt
{prefix}_ofb64_encrypt
{prefix}_cfb64_encrypt
I called the AES versions ofb128 and cfb128 due to the block size. I
expect this can be handled somehow by #defines. I could create 64-bit
feedback variants, but I don't think this is what people expect by
default.
All the arguments etc. are like the other cfb/ofb functions.
I don't think CTR is considered at all in the EVP layer. In other
words, you may implement it, but it will only be available as a direct
algorithm function, not as part of the EVP layer.
I suppose that's fine for now. It appears that CTR is going to be in
the NIST FIPS for AES modes, so it'd be nice to get it into EVP someday.
In any case, I can most certainly help you out, as I've already
fiddled with the first patches you sent.
I was figuring you would :)
S
--
Stephen Sprunk So long as they don't get violent, I want to
CCIE #3723 let everyone say what they wish, for I myself have
K5SSSalways said exactly what pleased me. --Albert Einstein
aes.modes.diff.bz2
Description: Binary data
Re: EVP macros for AES
Stephen Sprunk wrote: Can someone help me implement the EVP macros for AES 128-bit CFB and OFB modes? It's too messy for me to figure out. I've got non-EVP versions written, but it appears the EVP macros do their own implementation of the various modes and only call the base ECB function. I've also got a trial implementation of CTR mode, but I don't see where that would fit into EVP or the ASN.1 stuff. The ASN.1 stuff you can largely ignore. Most of the existing CFB and OFB modes do and nothing really uses the ASN.1 stuff with those modes. The macros can also be ignored and you can populate the EVP_CIPHER structure manually if you wish. The reason for the macros was that when the EVP layer was being revised it was an absolute nightmare to change anything. IIRC at one point I had to add an additional field to the EVP_CIPHER structure. I then spent the next couple of hours modifying lots of almost identical EVP_CIPHER definitions spread over zillions of little files. So I rewrote the files to use macros so that any future changes could largely be handled by just changing the macros and the odd exception manually. Steve. -- Dr Stephen N. Henson. http://www.drh-consultancy.demon.co.uk/ Personal Email: [EMAIL PROTECTED] Senior crypto engineer, Gemplus: http://www.gemplus.com/ Core developer of the OpenSSL project: http://www.openssl.org/ Business Email: [EMAIL PROTECTED] PGP key: via homepage. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: EVP macros for AES
From: Stephen Sprunk [EMAIL PROTECTED]
stephen Can someone help me implement the EVP macros for AES 128-bit CFB and
stephen OFB modes? It's too messy for me to figure out.
stephen
stephen I've got non-EVP versions written, but it appears the EVP macros do
stephen their own implementation of the various modes and only call the base
stephen ECB function.
Nope, they don't do their own implementation, all they do is write EVP
wrappers that call the different functions of the algorithm itself.
Note that this puts a requirement on the algorithm functions to follow
a certain name standard. The expected frunctions are, for a certain
{prefix} (AES in the AES case, I assume :-)):
{prefix}_ecb_encrypt
{prefix}_cbc_encrypt
{prefix}_ofb64_encrypt
{prefix}_cfb64_encrypt
Furthermore, there are a few semantic things. All of those functions
have to use the exact same arguments for all algorithms, except for
the key schedule argument. Also, {prefix}_ecb_encrypt is expected to
encrypt only one block, while the others are expected to encrypt the
whole message that is sent to them.
stephen I've also got a trial implementation of CTR mode, but I don't
stephen see where that would fit into EVP or the ASN.1 stuff.
I don't think CTR is considered at all in the EVP layer. In other
words, you may implement it, but it will only be available as a direct
algorithm function, not as part of the EVP layer.
In any case, I can most certainly help you out, as I've already
fiddled with the first patches you sent.
--
Richard Levitte \ Spannvägen 38, II \ [EMAIL PROTECTED]
Redakteur@Stacken \ S-168 35 BROMMA \ T: +46-8-26 52 47
\ SWEDEN \ or +46-733-72 88 11
Procurator Odiosus Ex Infernis-- [EMAIL PROTECTED]
Member of the OpenSSL development team: http://www.openssl.org/
Software Engineer, GemPlus: http://www.gemplus.com/
Unsolicited commercial email is subject to an archival fee of $400.
See http://www.stacken.kth.se/~levitte/mail/ for more info.
__
OpenSSL Project http://www.openssl.org
Development Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
