Hi Walter. I looked at the cryptopp site but can't find the page that contains the benchmark comparison between the two versions, the one that you posted. Where did you get those results from? A URL to would be best.
Tomorrow I'll run the tests on my local machine to see of they match up with what you are seeing. It's great news that the cryptopp benchmark matches your benchmark. It would have been a lot more difficult if it hadn't. As far as ideas. I might suspect sse2 or other CPU differences affecting the results. We would need to know the system the tests were run on. If it's the core2 machine that is listed on the benchmarks page then it may be CPU differences that explain the change. The next step imo would be to run a profiler during the affected test on the old and new version. If I see slowness tomorrow I can run the test under valgrind(callgrind) in Linux and get a method listing. That should make it pretty easy to see which methods are taking more time and make it easier to look at the svn changes to the files. If the results don't reproduce for me, ie 5.6.0 is faster, then you'll have to run the profiler. If you have Linux on the same machine, even through something like virtualbox, it should be possible to get reasonable results that should match the delta in performance you see in windows and should produce a useful graph of methods and their execution times. Chris On Tuesday, January 26, 2010, Walter Villalba <[email protected]> wrote: > These are the results I got: > Version 5.5.2Algorithm MiB/Second Microseconds to Setup > Key and IV > AES/CFB (128-bit key) 82 0.451AES/CFB (256-bit key) 66 > 0.528 > > > Version 5.6.0Algorithm MiB/Second Microseconds to Setup > Key and IV > AES/CFB (128-bit key) 73 0.667AES/CFB (256-bit key) 61 > 0.760 > > Version 5.6.0 without dynamic_cast checking (lib modified)Algorithm > MiB/Second Microseconds to Setup Key and IV > AES/CFB (128-bit key) 73 0.661AES/CFB (256-bit key) 61 > 0.716 > > > My setup:Windows XP Professional Version 2002 SP 3AMD Athlon II X2 240 > 2.81GHZ 3.25 GB of RAM (used by the OS)MS VC6 SP 6 > All tests were run under similar conditions. These results match the ones I > got using my test app; version 5.5.2 still seems to perform better than > 5.6.0. Removing the dynamic_cast checking doesn't seem to make a difference. > > Any other ideas ? > Thanks,Walt. > > On Tue, Jan 26, 2010 at 7:55 AM, Chris Morgan <[email protected]> wrote: > The benchmarks page says they are a part of the source code package. > You should run these and see how your numbers compare so there is a > equivalence baseline. If the numbers match then looking at the > particulars of your test vs the one in the benchmark seems the next > logical thing to do. If they don't match then we should look at > comparing CPU specs, compiler versions, OSes etc. > > Chris > > > On Tuesday, January 26, 2010, Walter Villalba <[email protected]> wrote: >> I'm not sure how the official benchmarks were generated, but Wei Dai sent >> them to me a while ago. Full benchmarks for version 5.6.0: >> http://www.cryptopp.com/benchmarks.html >> >> <http://www.cryptopp.com/benchmarks.html>The results I got using my little >> test app were run under similar OS conditions. The app basically encrypts >> and decrypts 1.5mb of text ( actually it was 2016 bytes by the time I got >> these results, but then I changed it to 1.5mb ), 200000 times, and measures >> the time it takes to do that ( this time does _not_ include set up time ). >> I compiled it against both versions, 5.5.2 and 5.6.0, and then ran it on my >> system. >> >> I just removed the dynamic_cast check, compiled against the library and then >> my test app, and got better results, but still not as good as the ones I get >> with version 5.5.2. >> >> v5.6.0 without dynamic_cast (lib modified)128-bit key: 11, 12, 11, 11, 11 >> secs256-bit key: 13, 13, 13, 12, 13 secs >> v5.5.2128-bit key: 11, 10, 10, 11, 10 secs >> 256-bit key: 12, 12, 12, 12, 12 secs >> As you can see, version 5.5.2 still performs better. >> Walt. >> >> On Tue, Jan 26, 2010 at 7:25 AM, Chris Morgan <[email protected]> wrote: >> Ahh, it wasn't obvious that your results differed, I thought you generated >> the chart. >> >> How were the original numbers generated? If they come from an app inside of >> cryptopp then are the input parameters close to the same? Same input buffer >> sizes etc? What happens if you run on your system the app that generated the >> claimed results? >> >> >> You would have to look to see if that dynamic cast could be removed. Adding >> a print message there, or a breakpoint under a debugger, would let you see >> if that code path was taken. I don't know if rtti is necessary for other c++ >> operations other than the dynamic cast. >> >> >> Chris >> >> >> >> On Jan 26, 2010, at 10:11 AM, Walter Villalba <[email protected]> wrote: >> >> >> Hi Chris, >> Thanks a lot for your response. Whether or not RTTI can be disabled is one >> of the questions. There's only one usage of dynamic_cast in version 5.6.0 >> of the library. I am not able to run my little test app if I don't enable >> RTTI for it ( regardless of enabling RTTI or not when I compile the library. >> Weird. ) >> >> >> >> The results don't look good at all, that's why I'm concerned. The image I >> sent are the official benchmarks, and they show AES performance was improved >> in version 5.6.0, but the results I'm getting do _not_ show this. Quote >> from my previous email: >> >> >> >> >> >> 128-bit key >> Release-v5.5.2-PPro-NoRTTI >> 16 / 16 / 16 / 16 / 16 >> Release-v5.5.2-PPro-RTTI >> 16 / 16 / 16 / 16 / 15 >> Release-v5.6.0-PPro-NoRTTI >> "Access violation - no RTTI data!" error >> Release-v5.6.0-PPro-RTTI >> 18 / 18 / 18 / 18 / 19 >> >> 256-bit key >> Release-v5.5.2-PPro-NoRTTI >> 17 / 18 / 17 / 17 / 18 secs >> Release-v5.5.2-PPro-RTTI >> 18 / 18 / 18 / 18 / 17 secs >> Release-v5.6.0-PPro-NoRTTI >> "Access violation - no RTTI data!" error >> Release-v5.6.0-PPro-RTTI >> 21 / 21 / 20 / 21 / 21 secsFor instance, AES 128-bit using version 5.5.2 >> takes 16 secs, and the same app compiled using version 5.6.0 takes 18. The >> source code was included in my first email. >> >> >> >> So AES seems to perform better in version 5.5.2 than in version 5.6.0, >> exactly the op > -- You received this message because you are subscribed to the "Crypto++ Users" Google Group. To unsubscribe, send an email to [email protected]. More information about Crypto++ and this group is available at http://www.cryptopp.com.
