Now we are talking. HW and SW setup: AMD Athlon II X2 240 2.81 GHZ Windows XP Pro Version 2002 SP3 Visual Studio 2008 Pro
Crypto++ project optimization settings: C/C++ --> Optimization --> Maximize Speed, Inline Func: Any Suitable, Enable Intrinsic, Favor Fast Code, Omit Frame Pointers, Enable link-time code gen. C/C++ --> Code generation --> Multi-threaded DLL, Enable SSE2 Results ( I added AES/CFB 256-bit to the benchmarks class ): Crypto++ V5.5.2 - Release - Optimized Algorithm MiB/Second Microseconds to Setup Key and IV AES/CFB (128-bit key) 82 0.432 AES/CFB (256-bit key) 66 0.510 Crypto++ v5.6.0 - Release - Optimized Algorithm MiB/Second Microseconds to Setup Key and IV AES/CFB (128-bit key) 167 0.532 AES/CFB (256-bit key) 130 0.594 I also ran tests with v5.6.0 in Release mode but without any of the optimizations detailed above. Results: Crypto++ v5.6.0 - Release - Not optimized Algorithm MiB/Second Microseconds to Setup Key and IV AES/CFB (128-bit key) 157 0.603 AES/CFB (256-bit key) 130 0.679 So Wei, you were absolutely right that these optimizations don't make a big difference at all. Finally, I ran the benchmarks in Debug mode. Results: Crypto++ v5.6.0 - Debug Algorithm MiB/Second Microseconds to Setup Key and IV AES/CFB (128-bit key) 161 1.647 AES/CFB (256-bit key) 126 1.750 These are not bad at all, although there's an important increment in time needed to set up the key and iv. So I must have been doing something wrong when I got the first set of results for VC9 + v5.6.0, which were really bad. Version 5.6.0 does improve AES performance _considerably_, although this does not show up if you use VC6 + SP6. Thanks a lot Chris and Wei for your help. Greatly appreciated. Walt. On Mon, Feb 1, 2010 at 4:58 PM, Wei Dai <[email protected]> wrote: > You don't need to set any optimizations, just build the > "release" (instead of "debug") version of cryptest. If it's still slower, > trying setting a debug breakpoint inside the inline assembly code in > rijndael.cpp to see if it's being used (although I can't think of any reason > why it wouldn't be). > > If it's still slower, send me a copy of the complete benchmarks output. I > want to see if everything is slower, or just AES. > > *From:* Walter Villalba <[email protected]> > *Sent:* Monday, February 01, 2010 3:44 PM > *To:* Chris Morgan <[email protected]> > *Cc:* Wei Dai <[email protected]> ; Crypto++ > Users<[email protected]> > *Subject:* Re: 5.5.2 vs 5.6.0 AES performance > > Official benchmark, using the cryptest app. > > On Mon, Feb 1, 2010 at 3:36 PM, Chris Morgan <[email protected]> wrote: > >> Were these results from the official benchmark or your test app? >> >> Chris >> >> >> >> On Feb 1, 2010, at 5:57 PM, Walter Villalba <[email protected]> wrote: >> >> Thank you both for your feedback. I've just installed VS 2008 and ran >> the benchmarks. I set some optimizations for the cryptest project, but I'm >> not sure which ones are required. >> The results I had got for v5.5.2 are still better than what I just got for >> v5.6.0 + VS 2008. I will try different optimization values, but it would >> be great to know exactly which ones are required to get the best performance >> in v5.6.0. >> >> Thanks ! >> Walt. >> >> >> On Wed, Jan 27, 2010 at 12:45 PM, Chris Morgan < <[email protected]> >> [email protected]> wrote: >> >>> Cool. Walter, were you able to get the processor pack installed or >>> move to a newer compiler? I tried to run the benchmarks but with 5.5.2 >>> I'm seeing a seg fault in the tests for , or immediately after, >>> salsa20, before it gets to the ones you wanted to compare so I was >>> hoping that my test results wouldn't be needed now that Wei has >>> responded. >>> >>> Chris >>> >>> >>> >>> On Tuesday, January 26, 2010, Wei Dai < <[email protected]> >>> [email protected]> wrote: >>> > >>> > >>> > >>> > >>> > >>> > MS VC6 SP 6 is the problem. It doesn't support the inline >>> > assembly in the AES code. >>> > >>> > Try upgrading to any version of VC after that, or VC6 SP5 with >>> > Processor Pack if you really can't upgrade. I'm not sure if you can >>> still >>> > download the Processor Pack anywhere though. >>> > >>> > >>> > From: Walter Villalba < <[email protected]>[email protected]> >>> > Sent: Tuesday, January 26, 2010 10:37 AM >>> > To: Chris Morgan < <[email protected]>[email protected]> >>> > Cc: Crypto++ Users < <[email protected]> >>> [email protected]> >>> > Subject: Re: 5.5.2 vs 5.6.0 AES performance >>> > >>> > These are the results I got: >>> > >>> > >>> > >>> > Version 5.5.2 >>> > Algorithm >>> > MiB/Second >>> > Microseconds to Setup Key and IV >>> > AES/CFB (128-bit key) >>> > 82 >>> > 0.451 >>> > AES/CFB (256-bit key) >>> > 66 >>> > 0.528 >>> > >>> > >>> > >>> > >>> > Version 5.6.0 >>> > Algorithm >>> > MiB/Second >>> > Microseconds to Setup Key and IV >>> > AES/CFB (128-bit key) >>> > 73 >>> > 0.667 >>> > AES/CFB (256-bit key) >>> > 61 >>> > 0.760 >>> > >>> > >>> > Version 5.6.0 without dynamic_cast checking (lib modified) >>> > Algorithm >>> > MiB/Second >>> > Microseconds to Setup Key and IV >>> > AES/CFB (128-bit key) >>> > 73 >>> > 0.661 >>> > AES/CFB (256-bit key) >>> > 61 >>> > 0.716 >>> > >>> > >>> > >>> > >>> > My setup: >>> > Windows XP Professional Version 2002 SP 3 >>> > AMD Athlon II X2 240 2.81GHZ 3.25 GB of RAM >>> > (used by the OS) >>> > MS VC6 SP 6 >>> > >>> > >>> > All tests were run under similar conditions. These results match >>> > the ones I got using my test app; version 5.5.2 still seems to >>> perform better >>> > than 5.6.0. Removing the dynamic_cast checking doesn't seem to make >>> a >>> > difference. >>> > >>> > >>> > Any other ideas ? >>> > >>> > >>> > Thanks, >>> > Walt. >>> > >>> > >>> > On Tue, Jan 26, 2010 at 7:55 AM, Chris Morgan < <[email protected]> >>> [email protected]> wrote: >>> > The benchmarks page says they are a part of the source >>> > code package. >>> > You should run these and see how your numbers compare so >>> > there is a >>> > equivalence baseline. If the numbers match then looking at >>> > the >>> > particulars of your test vs the one in the benchmark seems the >>> > next >>> > logical thing to do. If they don't match then we should look >>> > at >>> > comparing CPU specs, compiler versions, OSes etc. >>> > >>> > Chris >>> > >>> > >>> > On Tuesday, January 26, 2010, Walter Villalba < <[email protected]> >>> [email protected]> wrote: >>> >> >>> > I'm not sure how the official benchmarks were generated, but Wei >>> Dai sent >>> > them to me a while ago. Full benchmarks for version 5.6.0: >>> > <http://www.cryptopp.com/benchmarks.html> >>> http://www.cryptopp.com/benchmarks.html >>> >> >>> >> >>> > < <http://www.cryptopp.com/benchmarks.html> >>> http://www.cryptopp.com/benchmarks.html>The results I >>> > got using my little test app were run under similar OS conditions. >>> The >>> > app basically encrypts and decrypts 1.5mb of text ( actually it was >>> 2016 >>> > bytes by the time I got these results, but then I changed it to >>> 1.5mb >>> > ), 200000 times, and measures the time it takes to do that ( this >>> time >>> > does _not_ include set up time ). I compiled it against both >>> > versions, 5.5.2 and 5.6.0, and then ran it on my system. >>> >> >>> >> I >>> > just removed the dynamic_cast check, compiled against the library >>> and then >>> > my test app, and got better results, but still not as good as the >>> ones I get >>> > with version 5.5.2. >>> >> >>> >> v5.6.0 without dynamic_cast (lib >>> > modified)128-bit key: 11, 12, 11, 11, 11 secs256-bit key: 13, 13, >>> 13, 12, 13 >>> > secs >>> >> v5.5.2128-bit key: 11, 10, 10, 11, 10 secs >>> > >>> > >>> > > 256-bit key: 12, 12, 12, 12, 12 secs >>> >> As you can >>> > see, version 5.5.2 still performs better. >>> >> Walt. >>> >> >>> >> On >>> > Tue, Jan 26, 2010 at 7:25 AM, Chris Morgan < <[email protected]> >>> [email protected]> wrote: >>> >> >>> > Ahh, it wasn't obvious that your results differed, I thought you >>> generated >>> > the chart. >>> >> >>> >> How were the original numbers generated? If they >>> > come from an app inside of cryptopp then are the input parameters >>> close to >>> > the same? Same input buffer sizes etc? What happens if you run on >>> your >>> > system the app that generated the claimed results? >>> >> >>> >> >>> >> >>> > You w-- >>> > You received this message because you are subscribed to the >>> > "Crypto++ Users" Google Group. >>> > To unsubscribe, send an email to >>> > <[email protected]> >>> [email protected]. >>> > More information about >>> > Crypto++ and this group is available at <http://www.cryptopp.com> >>> http://www.cryptopp.com. >>> > >>> > >>> > >>> > -- >>> > You received this message because you are subscribed to the "Crypto++ >>> Users" Google Group. >>> > To unsubscribe, send an email to >>> <[email protected]> >>> [email protected]. >>> > More information about Crypto++ and this group is available at >>> <http://www.cryptopp.com>http://www.cryptopp.com. >>> >> >> > -- You received this message because you are subscribed to the "Crypto++ Users" Google Group. To unsubscribe, send an email to [email protected]. More information about Crypto++ and this group is available at http://www.cryptopp.com.
