Your message dated Tue, 16 Sep 2014 09:32:31 +0000
with message-id <[email protected]>
and subject line Bug#760817: fixed in ssdeep 2.11-1
has caused the Debian Bug report #760817,
regarding ssdeep: wrong scoring on two fuzzy hashes with same block sizes
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
760817: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=760817
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: ssdeep
Version: 2.7-2
Severity: important
Tags: patch
Dear Maintainer,
ssdeep (and libfuzzy2 Debian package) before version 2.10 has a bug
which may make wrong score on two fuzzy hashes with same block sizes.
This will make clustering/comparing files unreliable.
This bug was fixed in 2.10 by Jesse Kornblum
<[email protected]> but still not fixed in Debian versions
(sid, unstable and stable).
I encountered this bug while clustering about 10M files based on ssdeep
hashes and I had to recluster all the files.
Sorry that I have no `natural' examples to reproduce (because I slightly
changed the parameter after building patched versions of
ssdeep/libfuzzy2 2.7-2 and it will take about 2 months * 20 CPU cores to
compare clusters) but we can generate `artificial' example by truncating
second chunk of fuzzy hashes.
[PROMPT_EXAMPLE_BEGIN]
$ # Generate artificial test cases
$ cat >test <<_END
ssdeep,1.1--blocksize:hash:hash,filename
24:5nmkHuww9FXe0ZpPKoVH7bK3KT1Odk8gKgNWvoqzDVEatXSHlY31x:E4uV9FX,"1"
24:5nmkHuww9FXe0ZpPKoVH7bK3KT1Odk8gKgNWvoqzDVENXSCYA1x:E4uV9FX,"2"
_END
$ # This is the expected result.
$ $SSDEEP_FIXED/ssdeep -k test -x test
test:1 matches test:2 (100)
test:1 matches test:2 (100)
test:2 matches test:1 (100)
test:2 matches test:1 (100)
test:1 matches test:2 (100)
test:1 matches test:2 (100)
test:2 matches test:1 (100)
test:2 matches test:1 (100)
$ # This is the result from Debian versions of ssdeep.
$ ssdeep -k test -x test
test:1 matches test:2 (94)
test:1 matches test:2 (94)
test:2 matches test:1 (94)
test:2 matches test:1 (94)
test:1 matches test:2 (94)
test:1 matches test:2 (94)
test:2 matches test:1 (94)
test:2 matches test:1 (94)
$
[PROMPT_EXAMPLE_END]
As you can see, buggy ssdeep/libfuzzy2 returns score of 94 but fixed
versions of ssdeep/libfuzzy2 returns score of 100 for cases:
* file 1 and file 2
* file 1 and file 1 (matching itself)
* file 2 and file 2 (matching itself)
Attached patch is excerpt from actual Jesse Kornblum's patch (applied in
ssdeep 2.10) formatted for Debian version of 2.7-2.
By the way, I recommend UPGRADING THE UPSTREAM VERSION TO 2.10 on
`unstable' and `sid' instead of applying the patch because ssdeep
version 2.10 fixes some other bugs (I didn't encountered but someone
other may).
Thanks and I hope this will be fixed before `Jessie' is frozen.
Tsukasa OI
http://a4lg.com/
-- System Information:
Debian Release: 7.6
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 3.2.0-4-amd64 (SMP w/40 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages ssdeep depends on:
ii libc6 2.13-38+deb7u4
ssdeep recommends no packages.
ssdeep suggests no packages.
-- no debconf information
diff --git a/fuzzy.c b/fuzzy.c
index a9b771c..bcdef56 100644
--- a/fuzzy.c
+++ b/fuzzy.c
@@ -584,7 +584,7 @@ int fuzzy_compare(const char *str1, const char *str2)
if (block_size1 == block_size2) {
uint32_t score1, score2;
score1 = score_strings(s1_1, s2_1, block_size1);
- score2 = score_strings(s1_2, s2_2, block_size2);
+ score2 = score_strings(s1_2, s2_2, block_size1*2);
// s->block_size = block_size1;
--- End Message ---
--- Begin Message ---
Source: ssdeep
Source-Version: 2.11-1
We believe that the bug you reported is fixed in the latest version of
ssdeep, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to [email protected],
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Helmut Grohne <[email protected]> (supplier of updated ssdeep package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing [email protected])
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Format: 1.8
Date: Tue, 16 Sep 2014 09:55:46 +0200
Source: ssdeep
Binary: ssdeep libfuzzy2 libfuzzy2-dbg libfuzzy-dev
Architecture: source
Version: 2.11-1
Distribution: unstable
Urgency: medium
Maintainer: Debian Forensics <[email protected]>
Changed-By: Helmut Grohne <[email protected]>
Description:
libfuzzy-dev - Recursive piecewise hashing tool (development headers)
libfuzzy2 - Recursive piecewise hashing tool (library)
libfuzzy2-dbg - Recursive piecewise hashing tool (debugging symbols)
ssdeep - Recursive piecewise hashing tool
Closes: 702551 721217 734912 741431 760817
Changes:
ssdeep (2.11-1) unstable; urgency=medium
.
* Team upload.
* Imported upstream version 2.11 (Closes: #741431)
+ New thread-safe libfuzzy API functions (Closes: #721217)
Update debian/libfuzzy2.symbols
+ No longer uses PATH_MAX
+ Fixes hash scoring (Closes: #760817)
+ Fixes memory leak (CloseS: #702551)
* Add patch to build shared library again
* Declare compliance with policy version 3.9.5: no changes needed
* Switch from autotools-dev to dh-autoreconf to support new architectures
(Closes: #734912)
* Fix changelog syntax for 2.7-2: missing clonon after "Closes".
* Bump to debhelper compat level 9.
+ Hardening just works
+ Multi-Arch paths (update debian/*.install)
* Convert to Multi-Arch
* Update debian/copyright.
Checksums-Sha1:
f1f6eb3fa33085bb38573d84925fb3b0608e200f 2036 ssdeep_2.11-1.dsc
f44db91fda437f36626eefd8e649ed8d5aea0e1b 376529 ssdeep_2.11.orig.tar.gz
75346dc939d47f60bc5f68ae8e81353e24381ed6 4208 ssdeep_2.11-1.debian.tar.xz
Checksums-Sha256:
db7f97fd3e04c697040773a03b0e9e5ecb9e6abbf21b6d707045cf56ebe83c02 2036
ssdeep_2.11-1.dsc
82cc0e06f44127fc5c9c507881951714981da6187cdcfed0158c9167f39effc7 376529
ssdeep_2.11.orig.tar.gz
5a350309205ff6db3a3395779b461b3d096d78f5cb59860dbab82debc9659572 4208
ssdeep_2.11-1.debian.tar.xz
Files:
737279885b3b001ed1fee646ebdde37f 2036 admin optional ssdeep_2.11-1.dsc
fb733169f8c7f210421805b1534b37e7 376529 admin optional ssdeep_2.11.orig.tar.gz
f5e2bcd6afbb8ccd956832ef84bb3a4a 4208 admin optional
ssdeep_2.11-1.debian.tar.xz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBCgAGBQJUF+3pAAoJEC0aqs8kRERCO+YQAJjjvyXocxQ/4LW5W8YHiPxC
5ttCMxW+UxYDaoxoekXHk47TKkmlsyh7UGtNx1BkM9kV+ntHIfIgjCGvUdqzkdFy
sG8O5H0FK6ECGQIZXZf4gzGM02vVYEwjCC6oxMD7EvvxThQRiWQzh94mbUinG/bh
mzRWa22XQzTh5Eq1wTSOl02TQ8nVAk/vz6YEOR1GNC2WaE9T1iN9w3UcWznzN8JC
nZ2SD0g4ibBNCX8egLQ3Nni3vn1MTFp9NT0bL4DgA0Dm5KJK4OO+awB7gQPM2MOP
YVFNeWDLrzMb51f++bbeXpVQW/50+CbAqedxodN1n7JxfPPpL3FDwpi5RSovLffH
z39FeWHb5c2rIwgHZULca0lraSnqi1ZDq7sGAtTWvZkVJ4UCw5Wk4U6SCC8FOdeB
J7fvz72RbhKI/ts6TLrdZ8wnf1M5Mc5h4whz3NDzpLLznQ1SAIZfsNK8ol3M12Ar
3uPAE95jibFqEC32K23tEbUaVrdDpJmA0PeuowiISK1VT1ju7Xw5R4H11YBuMLg2
JbSYjd8wFyU57B6yuwMAnpiEyp4Db8KB+bgzDJnHaHkj7grbvJwetinFvxWU98ye
+pC6Mw6Xbnp4Y20Xw7b32zdyDNvW5sEH+806trnnmeV/uuSUrCjfXQPUbwMN/tzM
fmwyGNIzC8l26Ct7Q0st
=vB4I
-----END PGP SIGNATURE-----
--- End Message ---
_______________________________________________
forensics-devel mailing list
[email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/forensics-devel
