So I came across an interesting issue with my GCM<AES> implementation
using Crypto++ statically linked (Win32), and google searches came up
pretty short. The issue seems to have been reported before
(specifically
http://www.mail-archive.com/[email protected]/msg06053.html),
but not fully understood. I think I can help move this along a little
bit, mostly I just want to help out the next guy that runs into the
same problem.
Here's the relevant test code....
------------------------------------
// this works fine...
CryptoPP::GCM<CryptoPP::AES>::Encryption *pGCM_AES = new
CryptoPP::GCM<CryptoPP::AES>::Encryption();
{
Encode(*pGCM_AES,symmetricKey,iv_buf,iv_buflen,buf,buflen,out_string);
delete pGCM_AES;
}
// this does not....
{
CryptoPP::GCM<CryptoPP::AES>::Encryption GCM_AES;
{
Encode(GCM_AES,symmetricKey,iv_buf,iv_buflen,buf,buflen,out_string);
}
}
// it hits the assert in secblock.h during the GCM_AES destructor
// Encode just wraps more code from the Crypto++ samples
void Encode(CryptoPP::GCM<CryptoPP::AES >::Encryption &GCM_AES, const
LPSymmetricKey &symmetricKey, const void *iv_buf, unsigned int
iv_buflen, const void *buf, unsigned int buflen, std::string
&out_string)
{
GCM_AES.SetKeyWithIV(symmetricKey.m_AES_Key.m_ptr,symmetricKey.m_AES_Key.size(),
(const byte *)iv_buf,iv_buflen);
CryptoPP::ArraySource( (const byte *)buf,buflen, true,
new CryptoPP::AuthenticatedEncryptionFilter( GCM_AES, new
CryptoPP::StringSink( out_string ), false, TAG_SIZE ) //
AuthenticatedEncryptionFilter
); // StringSource
}
---------------------------------------
If I comment out the Encode call, which pretty much just wraps code
from the Crypto++ GCM-AE samples, there is also no problem with the
assert -- meaning that it relies on USE of the cipher, and not just
allocating it.
The obvious difference here is that one GCM<AES> is allocated on the
heap, and one is on the stack. The not so obvious difference is that
the destructor for GCM<AES> is being treated differently by the
compiler: allowing GCM_AES to leave scope inlines the destructor (only
possible when linking statically), whereas delete pCGM_AES sets up a
call to the destructor via the vtable.
This seems to imply that either a buffer overrun or something along
those lines is occurring during Encode and corrupting the stack, or
alternatively that one or more registers after the Encode call do not
have the expected value but are properly repaired by the vtable call.
I'm pretty sure a buffer overrun occurring with one of the two methods
would also manifest itself with the other, so following the second
hunch I checked my Whole Program Optimization setting -- knowing full
well that this option will make the compiler use nonstandard calling
conventions and register usage. It was enabled, so after disabling
this option, lo and behold both methods work correctly.
So I googled for Crypto++ Whole Program Optimization, and there's not
much about it other than that it improves speed (obviously) and people
do indeed use this option with Crypto++. That doesn't particularly
surprise me, since WPO will give different results on different
programs and I believe Crypto++ also has to be statically linked for
it to matter here.
I'm no expert on the implementation details of Crypto++ but a quick
browse through the source turns up a bunch of inline asm. While I
can't point to anything specific, I believe that some of this inline
asm may *sometimes* be wrong, when Whole Program Optimization is
enabled.
At any rate, workarounds seem to be a) explicitly destroy the
encryptor object, or b) disable Whole Program Optimization (possibly
just on the lib?)
--
You received this message because you are subscribed to the "Crypto++ Users"
Google Group.
To unsubscribe, send an email to [email protected].
More information about Crypto++ and this group is available at
http://www.cryptopp.com.
