Bug Tracker item #2815880, was opened at 2009-07-02 17:11
Message generated for change (Comment added) made by csmr
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=2815880&group_id=250683
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: daemon
Group: v3.9.0
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: Carlo Rodrigues (csmr)
Assigned to: Stevan Bajic (sbajic)
Summary: Another email that crashes dspam 3.9.0 git
Initial Comment:
dspam crashes when classifying this email. I noticed it while running
dspam_train, but it also crashes by invoking dspam, isolated, or in
client/server mode.
dspam --class=spam --source=corpus --user ad...@net4b.pt <
Spam/1222473209.13708_2.relay-2.lx.esp\,S\=2939\:2\,S
*** glibc detected *** dspam: munmap_chunk(): invalid pointer:
0x00000000036cfb60 ***
*** glibc detected *** dspam: malloc(): memory corruption: 0x00000000036cf880
***
NOTE: 3.8.0 does not crash, but 3.9.0 does.
I'm using the latest git code, after commit
9d7c29ac1ced95157a814754595b2b9d7a14f4cd .
I'm uploading the offending message.
----------------------------------------------------------------------
>Comment By: Carlo Rodrigues (csmr)
Date: 2009-07-10 12:01
Message:
Another one. This one has been trained before with noissues, but this time
crashed. It has a slightly different backtrace:
[test: spam ] Spam//1240591678.20856_49.relay- result: FAIL (Virus)***
glibc detected *** /usr/bin/dspam: double free or corruption (out):
0x00007fffa9a7b090 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3fd1c71ce2]
/lib64/libc.so.6(cfree+0x8c) [0x3fd1c7590c]
/usr/lib64/libdspam.so.7(_ds_operate+0x45d)[0x2b8200c70c5d]
/usr/lib64/libdspam.so.7(dspam_process+0x25c)[0x2b8200c7184c]
/usr/bin/dspam(retrain_message+0x17a)[0x40681a]
/usr/bin/dspam(process_message+0x48c)[0x40a72c]
/usr/bin/dspam(process_users+0x7f9)[0x40bc99]
/usr/bin/dspam(main+0x3c8)[0x40c778]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3fd1c1d974]
/usr/bin/dspam[0x403ef9]
----------------------------------------------------------------------
Comment By: Stevan Bajic (sbajic)
Date: 2009-07-07 15:33
Message:
Hello Carlo,
with your messages I don't get any crash but I do get the same error as
you on other messages. I really need to track that thing down. I just had
not enough time to spend on DSPAM in the last days and weeks. However...
for me the double free is a show stopper. As long as that thing is there I
will not let DSPAM 3.9.0 go productive. I will probably not have much time
to spend hunting down that problem till after the start of next week. So
don't expect any news from me in the next days. But I am not forgetting the
problem. If any one else has time and finds and corrects the problem that's
fine with me too :)
Kind Regards,
Stevan Bajic
----------------------------------------------------------------------
Comment By: Carlo Rodrigues (csmr)
Date: 2009-07-07 12:15
Message:
Sorry for taking this long to answer, but I was tracking the development of
this 'artifact' on the Dspam-devel mailing list, and I did not receive
further info regarding this thread since my previous post In fact, I did
not receive any email for the mailing list since July 3 until last dawn.
Today I decided to login to sourceforge and it was only then that I saw
that you had been working on it.
I'm attaching the output of valgrind -v --show-reachable and of gdb.
I'm no guru using these tools, so if you want to run any of them with some
other flags, just ask.
Carlo Rodrigues
----------------------------------------------------------------------
Comment By: Stevan Bajic (sbajic)
Date: 2009-07-03 13:55
Message:
Learning works too:
-----
theia vuadmin # dspam --class=spam --source=corpus --user mergedglobal
--stdout --deliver=summary <
1222473209.13708_2.relay-2.lx.esp\,S\=2939\:2\,S
X-DSPAM-Result: mergedglobal; result="Spam"; class="Spam";
probability=1.0000; confidence=1.00; signature=N/A
theia vuadmin #
-----
Anyway... I am taking your issue serious. That it works for me is all fine
but under no circumstances should we get a double free in DSPAM.
----------------------------------------------------------------------
Comment By: Stevan Bajic (sbajic)
Date: 2009-07-03 13:48
Message:
This here is god info for me:
*** glibc detected *** dspam: double free or corruption (!prev):
0x0953fa68 ***
======= Backtrace: =========
/lib/libc.so.6[0x3cd0f1]
/lib/libc.so.6(cfree+0x90)[0x3d0bc0]
/usr/lib/libdspam.so.7(_ds_degenerate_message+0x1ce)[0xcdf52e]
/usr/lib/libdspam.so.7(dspam_process+0x1f5)[0xcd70e5]
dspam(process_message+0xc6a)[0x805284a]
dspam(process_users+0x972)[0x8053972]
dspam(main+0x440)[0x8054750]
/lib/libc.so.6(__libc_start_main+0xdc)[0x379e8c]
dspam[0x804adf1]
_ds_degenerate_message has an issue I am currently looking at. I am since
+/- one week hunting the issue and I know it is there but I still have not
managed to iron out the issue. I know what the issue is but I have jet not
managed to trap why the issue is happening? The internal code in DSPAM is
going wild and I have not managed to understand why. I see it happen on my
setup and I know why later DSPAM is crashing but I have not found the
reason or a logical explanation why it is producing the result that leads
to a crash. The bad thing is that I can not enclose the bad result in a
condition and work around the problem since programamatically I can not
trap the problem inside _ds_degenerate_message(). The error is outside the
function and is happening in the node tree DSPAM uses. So I must fix the
problem there where it is really happening and not try to work around the
issue in _ds_degenerate_message().
The reason that DSPAM < 3.9.0 is not crashing is that it does copy memory
around while processing the exactly same message as 3.9.0. And that memory
copy stuff does not crash DSPAM. In 3.9.0 we have replaced that memory
copying code with a more robust and efficient code but that leads to
problems. Now one can argue that the old code was more robust and we should
bring back that code to DSPAM 3.9.0. That is right. It did not crashed
DSPAM but it copied just some memory around which had nothing to do with
the message it was processing. So DSPAM < 3.9.0 was just doing some useless
work and we fixed that in 3.9.0 but now the fix in one module does show us
that there is another problem that we need to fix as well. And that second
problem is deep, deep down in the DSPAM node tree code. I need more time to
hunt that issue then I have expected :(
Kind Regards,
Stevan Bajic
----------------------------------------------------------------------
Comment By: Stevan Bajic (sbajic)
Date: 2009-07-03 13:36
Message:
Hallo Carlo
Bad news: It works for me
-------
theia vuadmin # dspam --user mergedglobal --stdout --deliver=summary
--classify < 1222473209.13708_2.relay-2.lx.esp\,S\=2939\:2\,S
X-DSPAM-Result: mergedglobal; result="Innocent"; class="Innocent";
probability=0.0000; confidence=0.55; signature=N/A
theia vuadmin #
-------
Doing a process on my user account works too:
-------
theia vuadmin # dspam --user xx...@xxxxx.xx --stdout --deliver=summary
--process < 1222473209.13708_2.relay-2.lx.esp\,S\=2939\:2\,S
X-DSPAM-Result: xx...@xxxxx.xx; result="Spam"; class="Spam";
probability=1.0000; confidence=0.84; signature=4,4a4df96d94451923619268
Return-Path: <a.craws...@acentech.com>
Delivered-To: c...@net4b.pt
Received: (qmail 24492 invoked by uid 1012); 26 Sep 2008 21:42:49 +0100
Delivered-To: n...@net4b.pt
Received: (qmail 24490 invoked from network); 26 Sep 2008 21:42:49 +0100
Received: from unknown (HELO smtp.net4b.pt) ([10.0.17.2]) (envelope-sender
<a.craws...@acentech.com>)
by relay-1.lx.esp (qmail-ldap-1.03) with SMTP
for <n...@net4b.pt>; 26 Sep 2008 21:42:49 +0100
Received: (qmail 10074 invoked from network); 26 Sep 2008 21:42:48 +0100
Received: (simscan 1.4.1 ppid 9971 pid 10021 t 1.1310s)
(scanners: regex: 1.4.1 spam: 3.2.4 attach: 1.4.1 clamav:
0.94/m:/d:8344); 26 Sep 2008 20:42:47 -0000
X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on smtp-2.lx.esp
X-Spam-Level: ********
X-Spam-Status: No, score=8.2 required=10.0
tests=BAYES_00,FH_HELO_EQ_D_D_D_D,
HELO_DYNAMIC_IPADDR,HTML_MESSAGE,HTML_TAG_BALANCE_BODY,RCVD_IN_PBL,
RDNS_DYNAMIC,URIBL_RHS_DOB,URIBL_SBL autolearn=no version=3.2.4
Received: from ppp91-78-189-215.pppoe.mtu-net.ru ([91.78.189.215])
(envelope-sender <a.craws...@acentech.com>)
by smtp.net4b.pt (qmail-ldap-1.03) with SMTP
for <n...@net4b.pt>; 26 Sep 2008 21:42:47 +0100
Date: Fri, 26 Sep 2008 18:55:22 +0000
Message-ID: <70883....@goliath>
From: "darian cliff" <a.craws...@acentech.com>
To: <n...@net4b.pt>
Subject: [SPAM] Em Portugal, pela primeira vez, foi inaugurado o site
gratuito com video erotico.
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="=_peDNzZVWVjaW3a"
X-DSPAM-Result: Spam
X-DSPAM-Processed: Fri Jul 3 14:28:29 2009
X-DSPAM-Confidence: 0.8443
X-DSPAM-Improbability: 1 in 543 chance of being ham
X-DSPAM-Probability: 1.0000
X-DSPAM-Signature: 4,4a4df96d94451923619268
This is a multi-part message in MIME format.
--=_peDNzZVWVjaW3a
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Bom dia, estimados clientes.
Alegra-nos informar que em Portugal, pela primeira vez, inaugura-se =
oficialmente o Big Porno Tube.
Só nós temos filmes gratuitos pornográficos mais quentes =
e perversos, assim como videos de curta metragem.=20
Pornografia e erótica de primeira a qualquer gosto.
Primeiro portal pornográfico legal em Portugal.
http://pornoportugal.net
--=_peDNzZVWVjaW3a
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Diso-8859-1">
</HEAD>
<BODY bgColor=3D#f6FFb9>
Bom dia, estimados clientes.<br>
<br>
Alegra-nos informar que em Portugal, pela primeira vez, inaugura-se =
oficialmente o Big Porno Tube.<br>
Só nós temos filmes gratuitos pornográficos mais quentes =
e perversos, assim como videos de curta metragem. <br>
Pornografia e erótica de primeira a qualquer gosto.<br>
Primeiro portal pornográfico legal em Portugal.<br>
<br>
<a href=3D"http://pornoportugal.net>Visita-nos e ficas =
contente>></a><br>
<br>
Tudo de bem! Nadia Bruno.<br>
</BODY>
</HTML>
--=_peDNzZVWVjaW3a--
theia vuadmin #
-------
That is btw a 32 bit Linux system. I need some time to look at your
dspam.conf and other things to see what the issue is. If you could run the
command in valgrind and/or GDB that would help me. Let me know if you need
help/instructions on how to run in GDB/Valgrind.
Kind Regards,
Stevan Bajic
----------------------------------------------------------------------
Comment By: Carlo Rodrigues (csmr)
Date: 2009-07-03 12:21
Message:
Sorry. In 32bit is also crashes, when I test it as the root user.
It crashes even if I change --user to anything else, and with a empty db.
In 64bit it just crashed if --user was ad...@net4b.pt.
# dspam --client --class=spam --source=corpus --user c...@net4b.pt <
/mnt/home/csmr/1222473209.13708_2.relay-2.lx.esp\,S\=2939\:2\,S
*** glibc detected *** dspam: double free or corruption (!prev):
0x0953fa68 ***
======= Backtrace: =========
/lib/libc.so.6[0x3cd0f1]
/lib/libc.so.6(cfree+0x90)[0x3d0bc0]
/usr/lib/libdspam.so.7(_ds_degenerate_message+0x1ce)[0xcdf52e]
/usr/lib/libdspam.so.7(dspam_process+0x1f5)[0xcd70e5]
dspam(process_message+0xc6a)[0x805284a]
dspam(process_users+0x972)[0x8053972]
dspam(main+0x440)[0x8054750]
/lib/libc.so.6(__libc_start_main+0xdc)[0x379e8c]
dspam[0x804adf1]
======= Memory map: ========
00110000-00119000 r-xp 00000000 fd:00 393428 /lib/libnss_files-2.5.so
00119000-0011a000 r-xp 00008000 fd:00 393428 /lib/libnss_files-2.5.so
0011a000-0011b000 rwxp 00009000 fd:00 393428 /lib/libnss_files-2.5.so
0011b000-0011f000 r-xp 00000000 fd:00 393253 /lib/libnss_dns-2.5.so
0011f000-00120000 r-xp 00003000 fd:00 393253 /lib/libnss_dns-2.5.so
00120000-00121000 rwxp 00004000 fd:00 393253 /lib/libnss_dns-2.5.so
001ff000-00207000 r-xp 00000000 fd:00 1353304
/usr/lib/libkrb5support.so.0.1
00207000-00208000 rwxp 00007000 fd:00 1353304
/usr/lib/libkrb5support.so.0.1
0020a000-00237000 r-xp 00000000 fd:00 1349443
/usr/lib/libgssapi_krb5.so.2.2
00237000-00238000 rwxp 0002d000 fd:00 1349443
/usr/lib/libgssapi_krb5.so.2.2
00291000-00292000 r-xp 00291000 00:00 0 [vdso]
00346000-00360000 r-xp 00000000 fd:00 393218 /lib/ld-2.5.so
00360000-00361000 r-xp 00019000 fd:00 393218 /lib/ld-2.5.so
00361000-00362000 rwxp 0001a000 fd:00 393218 /lib/ld-2.5.so
00364000-004a2000 r-xp 00000000 fd:00 393234 /lib/libc-2.5.so
004a2000-004a4000 r-xp 0013e000 fd:00 393234 /lib/libc-2.5.so
004a4000-004a5000 rwxp 00140000 fd:00 393234 /lib/libc-2.5.so
004a5000-004a8000 rwxp 004a5000 00:00 0
004aa000-004ac000 r-xp 00000000 fd:00 393249 /lib/libdl-2.5.so
004ac000-004ad000 r-xp 00001000 fd:00 393249 /lib/libdl-2.5.so
004ad000-004ae000 rwxp 00002000 fd:00 393249 /lib/libdl-2.5.so
004b0000-004d5000 r-xp 00000000 fd:00 393245 /lib/libm-2.5.so
004d5000-004d6000 r-xp 00024000 fd:00 393245 /lib/libm-2.5.so
004d6000-004d7000 rwxp 00025000 fd:00 393245 /lib/libm-2.5.so
004d9000-004ec000 r-xp 00000000 fd:00 393247 /lib/libpthread-2.5.so
004ec000-004ed000 r-xp 00012000 fd:00 393247 /lib/libpthread-2.5.so
004ed000-004ee000 rwxp 00013000 fd:00 393247 /lib/libpthread-2.5.so
004ee000-004f0000 rwxp 004ee000 00:00 0
004f2000-0052d000 r-xp 00000000 fd:00 393255 /lib/libsepol.so.1
0052d000-0052e000 rwxp 0003a000 fd:00 393255 /lib/libsepol.so.1
0052e000-00538000 rwxp 0052e000 00:00 0
0053a000-00550000 r-xp 00000000 fd:00 393256 /lib/libselinux.so.1
00550000-00552000 rwxp 00015000 fd:00 393256 /lib/libselinux.so.1
0055f000-00571000 r-xp 00000000 fd:00 1344678 /usr/lib/libz.so.1.2.3
00571000-00572000 rwxp 00011000 fd:00 1344678 /usr/lib/libz.so.1.2.3
00574000-0057f000 r-xp 00000000 fd:00 393254
/lib/libgcc_s-4.1.2-20080825.so.1
0057f000-00580000 rwxp 0000a000 fd:00 393254
/lib/libgcc_s-4.1.2-20080825.so.1
00655000-00668000 r-xp 00000000 fd:00 393413 /lib/libnsl-2.5.so
00668000-00669000 r-xp 00012000 fd:00 393413 /lib/libnsl-2.5.so
00669000-0066a000 rwxp 00013000 fd:00 393413 /lib/libnsl-2.5.so
0066a000-0066c000 rwxp 0066a000 00:00 0
0066e000-00677000 r-xp 00000000 fd:00 393291 /lib/libcrypt-2.5.so
00677000-00678000 r-xp 00008000 fd:00 393291 /lib/libcrypt-2.5.so
00678000-00679000 rwxp 00009000 fd:00 393291 /lib/libcrypt-2.5.so
00679000-006a0000 rwxp 00679000 00:00 0
006c4000-006c6000 r-xp 00000000 fd:00 393433 /lib/libcom_err.so.2.1
006c6000-006c7000 rwxp 00001000 fd:00 393433 /lib/libcom_err.so.2.1
006e8000-006f7000 r-xp 00000000 fd:00 393431 /lib/libresolv-2.5.so
006f7000-006f8000 r-xp 0000e000 fd:00 393431 /lib/libresolv-2.5.so
006f8000-006f9000 rwxp 0000f000 fd:00 393431 /lib/libresolv-2.5.so
006f9000-006fb000 rwxp 006f9000 00:00 0
006fd000-006ff000 r-xp 00000000 fd:00 393429 /lib/libkeyutils-1.2.so
006ff000-00700000 rwxp 00001000 fd:00 393429
/lib/libkeyutils-1.2Aborted
If I can be of any further assistance, just ask.
Carlo Rodrigues
----------------------------------------------------------------------
Comment By: Carlo Rodrigues (csmr)
Date: 2009-07-03 12:06
Message:
I'm testing it against CentOS 5.3 32bit / 64 bit.
In the 32bit environment it does not crash.
----------------------------------------------------------------------
Comment By: Carlo Rodrigues (csmr)
Date: 2009-07-03 11:46
Message:
# cat /var/dspam/group
default:shared,managed:*
ad...@net4b.pt:shared,managed:*...@net4b.pt
Just tested it against another user (from the same group) and it did not
complain this time.
r...@ravaso-1 dspam]# dspam --class=spam --source=corpus --user
u...@net4b.pt <
/mnt/home/csmr/1222473209.13708_2.relay-2.lx.esp\,S\=2939\:2\,S
[r...@ravaso-1 dspam]# dspam --class=spam --source=corpus --user
ad...@net4b.pt <
/mnt/home/csmr/1222473209.13708_2.relay-2.lx.esp\,S\=2939\:2\,S
*** glibc detected *** dspam: munmap_chunk(): invalid pointer:
0x000000000edf5b60 ***
*** glibc detected *** dspam: malloc(): memory corruption:
0x000000000edf5880 ***
I'm going to erase all data on mysql db and /var/dspam/ and I'll do the
corpus training again, to check if the error appears again.
----------------------------------------------------------------------
Comment By: Carlo Rodrigues (csmr)
Date: 2009-07-03 11:39
Message:
It's a text / html message.
See the attached dspam.conf.
This is a x86_64 system. I haven't tested it in on 32 bits, though. I'll
do that as well.
The output of 'dspam --version';
# dspam --version
DSPAM Anti-Spam Suite GIT (agent/library)
Copyright (c) 2002-2009 DSPAM Project
http://dspam.sourceforge.net.
DSPAM may be copied only under the terms of the GNU General Public
License,
a copy of which can be found with the DSPAM distribution kit.
Configuration parameters: '--prefix=/usr' '--with-logdir=/var/log/dspam'
'--with-dspam-home=/var/dspam' '--with-dspam-home-mode=2775'
'--with-dspam-home-owner=dspam' '--with-dspam-home-group=dspam'
'--with-dspam-mode=0755' '--with-dspam-owner=dspam'
'--with-dspam-group=dspam' '--sysconfdir=/etc' '--enable-daemon'
'--enable-clamav' '--enable-virtual-users' '--enable-preferences-extension'
'--with-storage-driver=mysql_drv'
'--with-delivery-agent=/usr/sbin/sendmail'
'--with-mysql-libraries=/usr/lib64/mysql'
'--with-mysql-includes=/usr/include/mysql' '--enable-debug'
dspam.debug contents:
5593: [07/03/2009 11:36:17] DSPAM Instance Startup
5593: [07/03/2009 11:36:17] input args: dspam --class=spam --source=corpus
--user ad...@net4b.pt
5593: [07/03/2009 11:36:17] pass-thru args:
5593: [07/03/2009 11:36:17] processing user ad...@net4b.pt
5593: [07/03/2009 11:36:17] uid = 0, euid = 0, gid = 0, egid = 0
5593: [07/03/2009 11:36:17] loading preferences for user ad...@net4b.pt
5593: [07/03/2009 11:36:17] _mysql_drv_getpwnam: successful returning
struct for name: ad...@net4b.pt
5593: [07/03/2009 11:36:17] Loading preferences for uid 3
5593: [07/03/2009 11:36:17] Loading preferences for uid 0
5593: [07/03/2009 11:36:17] loaded default preferences externally
5593: [07/03/2009 11:36:17] using /var/dspam/opt-in/ad...@net4b.pt.dspam
as path
5593: [07/03/2009 11:36:17] using
/var/dspam/opt-out/ad...@net4b.pt.nodspam as path
5593: [07/03/2009 11:36:17] assigning user ad...@net4b.pt to group
default
5593: [07/03/2009 11:36:17] assigning user ad...@net4b.pt to group
ad...@net4b.pt
5593: [07/03/2009 11:36:17] sedation level set to: 5
5593: [07/03/2009 11:36:17] _mysql_drv_getpwnam: successful returning
struct for name: ad...@net4b.pt
5593: [07/03/2009 11:36:17] Connecting to 127.0.0.1:3310 for virus check
5593: [07/03/2009 11:36:17] Connecting to 127.0.0.1:1060 for virus stream
transmission
5593: [07/03/2009 11:36:17] message is signed. retaining original text
for reassembly
5593: [07/03/2009 11:36:17] message is signed. retaining original text
for reassembly
----------------------------------------------------------------------
Comment By: Stevan Bajic (sbajic)
Date: 2009-07-02 22:18
Message:
Hallo Carlo
Damn! That should not happen. Lt me guess: The mail is not pure text but
is a HTML mail. Right?
Could you post your dspam.conf and any other additional information you
have?
Kind Regards,
Stevan Bajic
----------------------------------------------------------------------
Comment By: Carlo Rodrigues (csmr)
Date: 2009-07-02 17:18
Message:
By the way, I'm using mysql backend and osb tokenizer.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=2815880&group_id=250683
------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Dspam-devel mailing list
Dspam-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspam-devel
