I've investigated this on my powerpc system. libmhash2 Debian version 0.9.6-1,
recompiled with -O0 and installed unstripped, produces this stack trace:
#0 0x0de3a170 in mutils_memcpy (dest=0x14, src=0x7fb956d4, n=20)
at stdfns.c:145
#1 0x0de3bf60 in _mhash_gen_key_s2k_simple (algorithm=MHASH_RIPEMD160,
keyword=0x103010a8, key_size=24, password=0x102ed068 "zzzz", plen=4)
at keygen_s2k.c:69
#2 0x0de3bb54 in mhash_keygen (algorithm=KEYGEN_S2K_SIMPLE,
opt_algorithm=MHASH_RIPEMD160, count=0, keyword=0x103010a8, keysize=24,
salt=0x0, saltsize=0, password=0x102ed068 "zzzz", passwordlen=4)
at keygen.c:147
#3 0x0e2c03f0 in grg_key_gen () from /usr/lib/libgringotts.so.2
#4 0x10012b2c in ?? ()
#5 0x10009c88 in ?? ()
#6 0x0e409c64 in g_cclosure_marshal_VOID__VOID ()
[... Gtk stuff from here downwards ...]
[Steps used to reproduce: Install gringotts version 1.2.8+1.2.9pre1-13.
gdb gringotts, run, File->New, enter password "zzzz" twice, OK.]
As previously noted in the bug report, the logic setting up times and
total in _mhash_gen_key_s2k_simple appears to be wrong. As times is
uninitialized when it is used to calculate total, and presumably happens to
be set to a large number, mhash tries to allocate an unreasonable amount of
memory and malloc() returns NULL.
In the "for" loop, the first time mutils_memcpy() is called, it
detects the NULL argument and silently returns. The second time it is
called, the dest argument is (NULL + block_size), i.e. 0x14, and it
attempts the copy and segfaults.
Reversing the order of the times and total assignments as suggested in
the Debian bug appears to overrun the end of the key buffer and causes stack
corruption, in my testing with gringotts (looking at the mhash source, it
seems gringotts must be using key_size not a multiple of block_size).
I think the correct order is as in the attached diff:
times = [...], if ([...]) times++, total = [...].
diff -u mhash-0.9.6/debian/changelog mhash-0.9.6/debian/changelog
--- mhash-0.9.6.orig/lib/keygen_s2k.c
+++ mhash-0.9.6/lib/keygen_s2k.c
@@ -39,12 +39,12 @@
MHASH td;
mutils_word32 block_size = mhash_get_block_size(algorithm);
- total = times * block_size;
-
times = key_size / block_size;
if (key_size % block_size != 0) times++;
+ total = times * block_size;
+
key = mutils_malloc(total);
#if defined(MHASH_ROBUST)
