On Tue, Sep 7, 2010 at 12:01 AM, Dan McGee <[email protected]> wrote: > On Mon, Sep 6, 2010 at 10:45 AM, Allan McRae <[email protected]> wrote: >> On 07/09/10 01:15, Dan McGee wrote: >>> >>> On Mon, Sep 6, 2010 at 10:04 AM, Allan McRae<[email protected]> wrote: >>>> >>>> On 07/09/10 00:25, Bryce Gibson wrote: >>>>> >>>>> ----- Original message ----- >>>>>> >>>>>> On 04/09/10 19:16, Jürgen Hötzel wrote: >>>>>>> >>>>>>> Hi Dan, >>>>>>> >>>>>>> 2010/9/2 Dan McGee<[email protected]>: >>>>>>>> >>>>>>>> This does not remove the MD5 code from our codebase, but it does >>>>>>>> enable linking against OpenSSL to get their much faster >>>>>>>> implementation if it is available on whatever platform you are >>>>>>>> using. At configure-time, we will default to using it if it is >>>>>>>> available, but this can be easily changed by using the >>>>>>>> `--with-openssl` or `--without-openssl` arguments to configure. >>>>>>> >>>>>>> What about just replacing the current MD5 implementation with the >>>>>>> OpenSSL implementation? >>>>>>> >>>>>>> This would prevent conditional compilation and a direct OpenSSL >>>>>>> dependency in libalpm. >>>>>> >>>>>> Can we do that? Openssl is BSD code. >>>>>> >>>>>> >>>>>> Anyway, I have concerns... Think of an openssl upgrade. pacman is >>>>>> in >>>>>> SyncFirst and it pulls in all its deps. If that pulls in openssl with >>>>>> a >>>>>> soname bump, things may get interesting. I have not check, but I do >>>>>> not >>>>>> think --as-needed saves us there. >>>>>> >>>>>> Allan >>>>>> >>>>> The licenses that openssl is released under aren't compatible with the >>>>> gpl >>>>> (according to the fsf) because of the advertising clause... So openssl >>>>> code >>>>> can't really be used like that... (Unfortunately) >>>> >>>> How does the coreutils md5sum speed compare? That is GPL code so could >>>> be >>>> included directly. Though I guess it is now GPL3 being a GNU project... >>> >>> Quite a bit slower. Test results (for one platform, please test on >>> whatever you are running on, it is very machine-dependent) and the >>> test harness included below, run using ./md5-speed.sh<bigfile> >> >> >>> . md5-speed.sh /home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz >> path: /home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz >> md5(/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz) = >> 40a30098649adf29ba79fcd6699d5f67 >> Version: i686 >> real 0m28.441s >> user 0m25.692s >> sys 0m2.643s >> >> path: /home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz >> md5(/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz) = >> 40a30098649adf29ba79fcd6699d5f67 >> Version: native >> real 0m28.527s >> user 0m25.455s >> sys 0m2.970s >> >> 40a30098649adf29ba79fcd6699d5f67 >> /home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz >> Program: md5sum >> real 0m22.553s >> user 0m20.552s >> sys 0m1.923s >> >> MD5(/home/arch/pkgcache/i686/nexuiz-data-2.5.2-1-any.pkg.tar.gz)= >> 40a30098649adf29ba79fcd6699d5f67 >> Program: openssl dgst -md5 >> real 0m22.436s >> user 0m19.745s >> sys 0m2.650s >> >> So not much difference here. > > Interesting, good to have someone else try it at least. I tried it on > three machines and two of the three machines (P4 and Atom, both i686) > were definitely slower with the coreutils stuff than the OpenSSL > version.
Thought it might be interesting to compare a 64bit atom (Atom 330, dual-core 1.6 GHz). Quite a big improvement with both md5sum and openssl. path: ../foo.tar md5(../foo.tar) = 49c19f21d5719a2508066780a41eee4a Version: x86-64 real 0m12.196s user 0m11.449s sys 0m0.733s path: ../foo.tar md5(../foo.tar) = 49c19f21d5719a2508066780a41eee4a Version: atom real 0m9.938s user 0m9.216s sys 0m0.703s path: ../foo.tar md5(../foo.tar) = 49c19f21d5719a2508066780a41eee4a Version: native real 0m10.128s user 0m9.279s sys 0m0.827s 49c19f21d5719a2508066780a41eee4a ../foo.tar Program: md5sum real 0m5.934s user 0m4.883s sys 0m0.640s MD5(../foo.tar)= 49c19f21d5719a2508066780a41eee4a Program: openssl dgst -md5 real 0m5.134s user 0m4.360s sys 0m0.753s
