Trying out gold (no LTO) on GCC's cc1, a log shows that the single biggest part (20%) of user time for such a medium sized program may be spent in SHA1 computation for the build id. I think the SHA1 code in libiberty would benefit from an SSE implementation, such as the one found in git or described on <http://en.wikipedia.org/wiki/SHA-1>.
Adding bug-gnulib in Cc: because that's where the libiberty SHA1 implementation comes from. Cheers, Ralf
