-------- Original-Nachricht -------- > Datum: Fri, 13 Nov 2009 18:00:57 -0200 > Von: Felipe Szczesny Rout <[email protected]> > An: [email protected] > CC: [email protected] > Betreff: [Fwd: Re: [Dspam-user] can\'t remove address from whitelist]
Hallo Felipe, > > > > -------- Original-Nachricht -------- > > > > Datum: Fri, 13 Nov 2009 15:38:53 -0200 > > > > Von: Felipe Szczesny Rout <[email protected]> > > > > An: Steve <[email protected]> > > > > CC: [email protected] > > > > Betreff: Re: [Dspam-user] can\'t remove address from whitelist > > > > > > > Em Sex, 2009-11-13 às 17:56 +0100, Steve escreveu: > > > > > -------- Original-Nachricht -------- > > > > > > Datum: Fri, 13 Nov 2009 13:58:00 -0200 > > > > > > Von: Felipe Szczesny Rout <[email protected]> > > > > > > An: Steve <[email protected]> > > > > > > CC: [email protected] > > > > > > Betreff: Re: [Dspam-user] can\'t remove address from whitelist > > > > > > > > > > > > > > > > > > > > > > > > > > > This all looks okay to me. I need to and dive into the source > code > > > > of > > > > > > DSPAM and look what the hell is happening. > > > > > > > > > > > > > > Can you once try to retrain using the Web UI? > > > > > > > > > > > > > > > > > > > > // Steve > > > > > > > > > > > > Hello Steve, > > > > > > > > > > > > I tried using Web UI and didn't work, so I tried command line: > dspam > > > > > > --source=error --class=spam > --signature=24,4af4bdf7303331677219375 > > > > > > --user [email protected] --debug > > > > > > > > > > > > Unfortunately it didn't work too: > > > > > > > > > > > You run that with --debug turned on. Could you post the debug log > for > > > > that command? > > > > > > > > > > > > 6023: [11/13/2009 15:19:42] No QuarantineAgent option found. Using > > > > standard quarantine. > > > > 6023: [11/13/2009 15:19:42] DSPAM Instance > > > > Startup > > > > 6023: [11/13/2009 15:19:42] input args: dspam --source=error > > > > --class=spam --signature=24,4af4bdf7303331677219375 --user > > > > [email protected] --debug > > > > 6023: [11/13/2009 15:19:42] pass-thru > > > > args: > > > > > > > > > 6023: [11/13/2009 15:19:42] processing user > > > > [email protected] > > > > > > > > > 6023: [11/13/2009 15:19:42] uid = 0, euid = 0, gid = 0, egid = > > > > 0 > > > > > > > > > 6023: [11/13/2009 15:19:42] loading preferences for user > > > > [email protected] > > > > > > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] Loading preferences for uid > > > > 24 > > > > > > > > > 6023: [11/13/2009 15:19:42] Loading preferences for uid > > > > 0 > > > > > > > > > 6023: [11/13/2009 15:19:42] loaded default preferences > > > > externally > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference > > > > 'dailyQuarantineSummary' => > > > > 'off' > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'enableBNR' => > > > > 'on' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'enableWhitelist' > => > > > > 'on' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'showFactors' => > > > > 'on' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference > 'signatureLocation' => > > > > 'message' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'spamAction' => > > > > 'quarantine' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'spamSubject' => > > > > '***DSPAM***' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference > 'statisticalSedation' > > > > => > > > > '5' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'trainingMode' => > > > > 'TUM' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'optIn' => > > > > 'off' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'optOut' => > > > > 'off' > > > > > > > > > 6023: [11/13/2009 15:19:42] > > > > using /var/spool/dspam/opt-in/al.rs.gov.br/webmaster.dspam as > > > > path > > > > 6023: [11/13/2009 15:19:42] > > > > using /var/spool/dspam/opt-out/al.rs.gov.br/webmaster.nodspam as > > > > path > > > > 6023: [11/13/2009 15:19:42] adding user to merged group > > > > [email protected] > > > > > > > > > 6023: [11/13/2009 15:19:42] sedation level set to: > > > > 5 > > > > > > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam returning cached > name > > > > [email protected]. > > > > > > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] loading preferences for user > > > > [email protected] > > > > > > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] Loading preferences for uid > > > > 24 > > > > > > > > > 6023: [11/13/2009 15:19:42] Loading preferences for uid > > > > 0 > > > > > > > > > 6023: [11/13/2009 15:19:42] loaded default preferences > > > > externally > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference > > > > 'dailyQuarantineSummary' => > > > > 'off' > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'enableBNR' => > > > > 'on' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'enableWhitelist' > => > > > > 'on' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'showFactors' => > > > > 'on' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference > 'signatureLocation' => > > > > 'message' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'spamAction' => > > > > 'quarantine' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'spamSubject' => > > > > '***DSPAM***' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference > 'statisticalSedation' > > > > => > > > > '5' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'trainingMode' => > > > > 'TUM' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'optIn' => > > > > 'off' > > > > > > > > > 6023: [11/13/2009 15:19:42] aggregated preference 'optOut' => > > > > 'off' > > > > > > > > > 6023: [11/13/2009 15:19:42] processing signature. length: > > > > 32767 > > > > > > > > That's +/- 32KB of data. Somehow I can't believe that. I believe that > the number in the field "length" is that high but I doubt that the whole > signature data is exactly the size of a signed small integer. > > Ok. > > > > > > > > > > > > > > > > > 6023: [11/13/2009 15:19:42] reversing 2047 > > > > tokens > > > > > > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] Control: [10 10] [11 9] Delta: [1 > > > > -1] > > > > > > > > > 6023: [11/13/2009 15:19:42] reclassifying iteration 1 result: > > > > 0 > > > > > > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam successful; > returning > > > > struct for name: > > > > [email protected] > > > > 6023: [11/13/2009 15:19:42] _mysql_drv_getpwnam returning cached > name > > > > [email protected]. > > > > > > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 6620605024458477568 > > > > (1frq, 9s, > > > > 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6620605024458477568 > > > > (1frq, 9s, > > > > 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 6599595429959041024 > > > > (1frq, 7s, > > > > 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6599595429959041024 > > > > (1frq, 7s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 8714215901233383021 > > > > (1frq, 7s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 8714215901233383021 > > > > (1frq, 7s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 6168168858540393442 > > > > (1frq, 8s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6168168858540393442 > > > > (1frq, 8s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 6524638155333945577 > > > > (2frq, 8s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6524638155333945577 > > > > (2frq, 8s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6524638155333945577 > > > > (2frq, 8s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 6620594680415387648 > > > > (1frq, 7s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6620594680415387648 > > > > (1frq, 7s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 5844884572724838400 > > > > (1frq, 10s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 5844884572724838400 > > > > (1frq, 10s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 12895274306055942099 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 12895274306055942099 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 382622526891763602 > > > > (3frq, 9s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 382622526891763602 > > > > (3frq, 9s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 382622526891763602 > > > > (3frq, 9s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 14120625105485046823 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 14120625105485046823 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 11816723597185145088 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 11816723597185145088 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 6721905138393415680 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6721905138393415680 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 17799590094035219423 > > > > (2frq, 27s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 17799590094035219423 > > > > (2frq, 27s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 17799590094035219423 > > > > (2frq, 27s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 6176834397643031522 > > > > (1frq, 14s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6176834397643031522 > > > > (1frq, 14s, 0i) > > > > 6023: [11/13/2009 15:19:43] [graham] [0.990000] E: > 14654105889212800539 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 14654105889212800539 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 9710725860101162536 > > > > (1frq, 12s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 6620882308173922304 > > > > (1frq, 5s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 17452455216015254526 > > > > (2frq, 6s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 17452455216015254526 > > > > (2frq, 6s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 8041186030285825251 > > > > (1frq, 7s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 7847970995338872181 > > > > (1frq, 6s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 4830379897255074049 > > > > (1frq, 11s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 2380572828823977984 > > > > (1frq, 5s, 0i) > > > > 6023: [11/13/2009 15:19:43] [burton] [0.990000] E: > 13484623132244863244 > > > > (1frq, 45s, 0i) > > > > 6023: [11/13/2009 15:19:43] Graham-Bayesian Probability: 1.000000 > > > > Samples: 15 > > > > 6023: [11/13/2009 15:19:43] Burton-Bayesian Probability: 1.000000 > > > > Samples: 27 > > > > 6023: [11/13/2009 15:19:43] using Graham factors > > > > 6023: [11/13/2009 15:19:43] Result Confidence: 0.99 > > > > 6023: [11/13/2009 15:19:43] total processing time: 0.07539s > > > > 6023: [11/13/2009 15:19:43] libdspam returned probability of > 1.000000 > > > > 6023: [11/13/2009 15:19:43] message result: SPAM > > > > 6023: [11/13/2009 15:19:43] appending header X-DSPAM-Reclassified: > Spam > > > > 6023: [11/13/2009 15:19:43] _mysql_drv_getpwnam returning cached > name > > > > [email protected]. > > > > 6023: [11/13/2009 15:19:43] DSPAM Instance Shutdown. Exit Code: 0 > > > > > > > > > > > > > > > > > > > > > > Could you run the same command slightly changed?: > > > > > dspam --source=error --class=spam > --signature=24,4af4bdf7303331677219375 > > > > --user [email protected] --debug ; echo ${?} > > > > > > > > > > What is the output of the above command? > > > > > > > > It returns 0. > > > > > > > > I'm using version 3.8.1 > > > > > > > 3.8.1? Never heard of that version before. Anyway... everything below > 3.9.0 has issues with large data. If you have read my comment above then > you understand what I mean. If I look at the definition of the table for < > 3.9.0 then I see this here: > > > create table dspam_signature_data ( > > I'm using dspam about 1 year, when I began study it Debian version was > (and I think still is) 3.6.8. I know there was version 3.8.0, searching > in google I found this page > https://juxtaposition.axley.net/2007/11/postfix-dspam-3-1.html with > instructions to create a new Debian package using version 3.8.0 and I > did it. I applied some diff files as this was a step for creating the > package. After all process I had a package with this name: > dspam_3.8.1~cvs20080106-1~etch2_amd64.deb > > This is the version I'm using. > > > > > > uid smallint unsigned not null, > > > signature char(32) not null, > > > data blob not null, > > > length smallint not null, > > > created_on date not null > > > ) type=MyISAM max_rows=2500000 avg_row_length=8096; > > > > > > Or: > > > create table dspam_signature_data ( > > > uid smallint unsigned not null, > > > signature varchar(32) not null, > > > data blob not null, > > > length smallint not null, > > > created_on date not null > > > ) type=MyISAM; > > > > > > Or: > > > create table dspam_signature_data ( > > > uid smallint unsigned not null, > > > signature char(32) not null, > > > data blob not null, > > > length smallint not null, > > > created_on date not null > > > ) type=MyISAM; > > > > > > See that field definition for "length"? That is the problem. The data > in the blob might go theoretically up to 2GB but the field "length" is > limiting DSPAM from reading the whole content. > > > > > > DSPAM 3.9.0 is lifting that restriction. In MySQL the definition is: > > > length int unsigned not null, > > > > > > And inside the totally reworked MySQL driver the limits where lifted > as well. So if you want your DSPAM to work properly on messages bigger then > 32KB that have been saved in the dspam_signature_data table then you must > switch to 3.9.0 which allows you to go up to 4GB per message (probably MySQL > will first bark before you can reach that limit but from the code > viewpoint that sits in DSPAM the limit is 4'294'967'295). > > > > > > I know that 32KB should be enough to have the sender parsed (since the > from line mostly sits on top of the message) but the way DSPAM saves the > degenerated message in binary format is not following the logical > composition of a mail. > > > > > > I can assure you that I was bitten by that bug long, long ago. That > was one of the reasons I reworked the MySQL driver. I did that very early and > ported it from release to release and finally included it this year into > 3.9.0. > > > > > > If you can then give 3.9.0 GIT a try. You will be positively surprised > by it's speed, stability and some functions that should have been working > since ages. > > I will take a look in new version, the update chapter mainly since I can't > drop my tables and restart from zero. > You don't need to drop the tables. Not at all. You only would need to do some small upgrade script on the MySQL table. That's all. Please have a look at the packages from Julian Valroff: http://packages.kirya.net/ He is actively upgrading DSPAM in his personal repository. A lot of people using Debian here on the list use Julian's packages. > We are planning to extend use for > all users here in my work (about 2.000) and stability is really important. > I can't force you to use a development version of DSPAM. But judging from the huge amount of fixes we have done for 3.9.0 I really ask myself if 3.8.0 is not the bigger beta/alpha/whatever then 3.9.0 BETA/GIT. > Thanks for your help Steve, maybe I'll ask you some help again with update > to new version !! :-) > No problem. The list is open 24 * 7. We have a lot of knowledge here bundled together on the list. Just ask. We are in general very friendly :) > > > > > > > > > > > The output of dspam --version: > > > > > > > > DSPAM Anti-Spam Suite CVS (agent/library) > > > > > > > > Copyright (c) 2002-2006 Jonathan A. Zdziarski > > > > http://dspam.nuclearelephant.com > > > > > > > > DSPAM may be copied only under the terms of the GNU General Public > > > > License, > > > > a copy of which can be found with the DSPAM distribution kit. > > > > > > > > Configuration parameters: '--host=x86_64-linux-gnu' > > > > '--build=x86_64-linux-gnu' '--prefix=/usr' > '--includedir=/usr/include' > > > > '--mandir=/usr/share/man' '--infodir=/usr/share/info' > > > > '--with-logdir=/var/log/dspam/' '--localstatedir=/var' > > > > '--libexecdir=/usr/lib/dspam' '--srcdir=..' > > > > '--with-dspam-home=/var/spool/dspam' '--sysconfdir=/etc/dspam' > > > > '--enable-domain-scale' '--enable-signature-headers' > > > > '--with-delivery-agent=/usr/bin/procmail' '--enable-daemon' > > > > '--with-mysql-includes=/usr/include/mysql' > > > > '--with-pgsql-includes=/usr/include/postgresql' > > > > > '--with-storage-driver=libdb4_drv,mysql_drv,pgsql_drv,sqlite3_drv,hash_drv' > > > > '--enable-ldap' '--enable-verbose-debug' '--enable-virtual-users' > > > > '--enable-preferences-extension' '--enable-clamav' > 'build_alias=x86_64-linux-gnu' > > > > 'host_alias=x86_64-linux-gnu' 'CFLAGS=-Wall -g -O2' 'CXXFLAGS=' > > > > > > > What distro is that? We could maybe help with a binary DSPAM 3.9.0 > package for that distro. > > It's a Debian-made-in-home package ;-) > > > > > > > > > > > > > > > > > > > > dspam_dump [email protected] From*"Carta IEDI > > > > <[email protected]>" > > > > > > 6322142184428839436 S: 00000 I: 00029 P: 0.0009 > > > > > > > > > > > > I belive that the command line above exclude MTA configuration > > > > > > problems... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > // Steve // Steve -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Dspam-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dspam-user
