Argh crossed emails there. Yes we noticed this in the office, the amount of help you give in cryptopp and then the total lack of feedback from boost on some pretty decent bug reports. We have found some issues as well but tamed a lot of them using condition variables to get over some race conditions and the likes. Often some weird errors though. I would hope some of the boost guys take a look at your detailed explanations here and maybe up the game a wee bit in replying.
I am not sure if you have ever tried OpenMP, but for the self encrypting file stuff I did use this and really enjoyed the threading experience (yes you are reading correctly). Whilst admittedly it may not be the most efficient, when you can remove dependencies from loops then it's almost a simple one line pre processor statement and immediate responses, you know sometimes what should go faster slows down, but other times the difference is amazing. What is cool is the speed at which you can try a thread model with it. It's not multi purpose though and not as efficient as had crafting a thread model, butt it's very efficient in it's own right and does back off a thread to let the computer still appear responsive whilst the rest of the cores are hammering away. Really recommend a play around with it. Best Regards David Irvine --------------------------------------------------------------------- Be independent of the good opinions of other people and have a healthy disregard for the impossible! ----------------------------------------------------------------- Einstein was right when he said "Great spirits have always encountered violent opposition from mediocre minds." ------------------------------------------------------------ maidsafe.net Ltd is a limited liability company incorporated in Scotland with number SC297540. Registered Office; 72 Templehill, Troon; KA10 6BE. VAT Registered 889 0608 77. Telephone Scotland: +44 1292 750020 On Sun, Oct 2, 2011 at 9:22 PM, Jeffrey Walton <[email protected]> wrote: > > > On Oct 2, 3:47 pm, David Irvine <[email protected]> wrote: > > On Sun, Oct 2, 2011 at 8:42 PM, Jeffrey Walton <[email protected]> > wrote: > > > > > On Oct 2, 3:30 pm, David Irvine <[email protected]> wrote: > > > > On Sun, Oct 2, 2011 at 7:39 PM, Jeffrey Walton <[email protected]> > > > wrote: > > > > > > > On Sep 27, 4:41 pm, David Irvine <[email protected]> > wrote: > > > > > > Hi > > > > > > > > I am testing gcc 4.6 and found what appears to be a bug. If I use > > > > > > CryptoPP::AutoSeededX917RNG< CryptoPP::AES > > > > > > > g_srandom_number_generator; > > > > > > (any cypher fails) > > > > > > > > Then it's a segfault (see below) > > > > > > > > However switching to (which I do not want to do, or more likely > > > > > > cannot) > > > > > > CryptoPP::AutoSeededRandomPool g_srandom_number_generator; > > > > > > > > Is fine, if not as secure. > > > > > > ######################################################################## > > > > > > System is Linux > > > > > > Linux di-ubuntu-64 3.0.0-11-generic #18-Ubuntu SMP Tue Sep 13 > > > 23:38:01 > > > > > > UTC 2011 x86_64 x86_64 x86_64 GNU/Linux > > > > > > Ubuntu oneiric (development branch) > > > > > > > > I am not wishing to make too much of this as this is a beta > release > > > > > > OS, but it seems more of a cryptopp issue I think. Is anyone else > > > > > > using ? > > > > > > gcc (Ubuntu/Linaro 4.6.1-9ubuntu3) 4.6.1 > > > > > > Copyright (C) 2011 Free Software Foundation, Inc. > > > > > > This is free software; see the source for copying conditions. > There > > > > > > is NO > > > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A > PARTICULAR > > > > > > PURPOSE > > > > > > > > Segfault below > > > > > > > > #0 0x000000000071c3ad in > > > > > > CryptoPP::BufferedTransformation::ChannelPut2(std::string const&, > > > > > > unsigned char const*, unsigned long, int, bool) > > > > > > () > > > > > > #1 0x00000000007b543e in > > > > > > CryptoPP::X917RNG::GenerateIntoBufferedTransformation(CryptoPP::BufferedTransformation&, > > > > > > std::string const&, unsigned long long) () > > > > > > #2 0x000000000071c88e in > > > > > > CryptoPP::RandomNumberGenerator::GenerateBlock(unsigned char*, > > > > > > unsigned long) () > > > > > > #3 0x00000000007b576b in > > > > > > CryptoPP::X917RNG::X917RNG(CryptoPP::BlockTransformation*, > unsigned > > > > > > char const*, unsigned char const*) () > > > > > > #4 0x0000000000743cc5 in > > > > > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(unsigned > char > > > > > > const*, unsigned long, unsigned char const*, unsigned char > const*) () > > > > > > #5 0x0000000000743ed8 in > > > > > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::Reseed(bool, > > > unsigned > > > > > > char const*, unsigned long) () > > > > > > #6 0x0000000000744000 in > > > > > > CryptoPP::AutoSeededX917RNG<CryptoPP::Rijndael>::AutoSeededX917RNG(bool, > > > > > > bool) () > > > > > > #7 0x00000000006eddc0 in > __static_initialization_and_destruction_0 > > > > > > (__initialize_p=1, __priority=65535) > > > > > > at > /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/ > > > > > > maidsafe/common/crypto.cc:61 > > > > > > #8 0x00000000006ede24 in _GLOBAL__sub_I_crypto.cc(void) () > > > > > > at > /home/dirvine/Devel/MaidSafe-Common/maidsafe_common_lib/src/ > > > > > > maidsafe/common/crypto.cc:299AES/CTR IV custom increment > > > > > > #9 0x00000000007e9ded in __libc_csu_init () > > > > > > #10 0x00007ffff70a02a0 in __libc_start_main (main=0x5f3114 > <main(int, > > > > > > char**)>, argc=1, ubp_av=0x7fffffffe118, > > > > > > init=0x7e9d90 <__libc_csu_init>, fini=<optimized out>, > > > > > > rtld_fini=<optimized out>, stack_end=0x7fffffffe108) at > libc-start.c: > > > > > > 185 > > > > > > #11 0x00000000005f305d in _start () > > > > > > (gdb) quit > > > > > I would guess that g_srandom_number_generator is in one compilation > > > > > unit, and the code using it is in another. Ie, > > > > > g_srandom_number_generator is in my_rng.cpp, and the code using it > is > > > > > in my_key_gen.cpp. In this case, you would be violating C++'s > static > > > > > non-local rules. > > > > > > > The fix is relatively easy. Offer a static/global function to serve > up > > > > > a static local: > > > > > > > RandomNumberGenerator& GetRandomNumberGenerator() > > > > > { > > > > > static AutoSeededX917RNG< CryptoPP::AES > s_prng; > > > > > return s_prng; > > > > > } > > > > > > > Jeff > > > > > > Thanks Jeff. It's possibly the case,, although this was initialised > in > > > the > > > > same file it was used, however what I noticed was that > > > > merely initialising the rng and never calling it anywhere would cause > > > this > > > > issue. > > > > > > To be a bit more exact this was in a my_rng.cc which was compiled > into a > > > lib > > > > that was then linked to another file.cc (or object, but you know what > I > > > > mean). The file.oo did have some cryptopp code in there (AES, HASH > etc. > > > but > > > > no explicit RNG calls). This is why it took a wee while to figure it > out. > > > It > > > > looks like there may have been a static somewhere I had not seen > (inside > > > > cryptopp) or something very similar. > > > I believe Crypto++ is fairly safe in its use of globals. They are > > > used, and they are easy to audit: searh for "g_*" in the source files. > > > > > > I have tried your suggestion and looks a good, although it does still > > > > segfault in some circumstances (I am tracking that down just now). I > > > think > > > > it may be a threading issue though, but will post again if it turns > out > > > to > > > > be more (which I doubt). > > > Crypto++ objects are thread safe, meaning they do not depend on global > > > data which might become corrupted across calls in different threads. > > > You still have to lock on concurrent access to an object. > > > > > For what it worth, Wei uses a different technique for initializing a > > > singleton (his method results in a memory leak under 'worst case', if > > > I recall correctly). > > > > > If you suspect you might have threading issues, Helgrind would > > > probably be very useful. But you need a good test program to get the > > > most out of it. > > > valgrind --tool=helgrind my_test_program > > > > > Jeff > > > > > Doing that right now Jeff, thanks > > > > For anyone who is interested the (offending) code is located here > https://github.com/maidsafe/MaidSafe-Common > At the risk of a flame war from the Boost fan boi club..... Don't use > Boost::Threads or Boost::Threading. From a recent audit, I believe the > subproject is not production quality. For example: > * Ignoring return values from locking functions > (ie, pthread_mutex_lock and WaitForSingleObject) > * Race conditions > * Resource leaks on negative code paths > > Most frustrating is the 20 or so bugs filed against Boost are still > sitting in their bug tracker unacknowledged (I believe two were > acknowledged and closed to date). > > Jeff > > -- > You received this message because you are subscribed to the "Crypto++ > Users" Google Group. > To unsubscribe, send an email to > [email protected]. > More information about Crypto++ and this group is available at > http://www.cryptopp.com. > -- You received this message because you are subscribed to the "Crypto++ Users" Google Group. To unsubscribe, send an email to [email protected]. More information about Crypto++ and this group is available at http://www.cryptopp.com.
