> Steve,
>
Hello Ibrahim,
> Here is the latest log before dspam crash.
>
Ahh.. Great. That is the crash you had on FreeBSD. Right?
> It seems that binary file instead of email file trained mistakenly?
>
Looks like but I can not tell. However... if I classify a binary on my setup,
then DSPAM does not crash:
theia dspam # dspam --user mergedglobal --classify --deliver=summary --stdout
--mode=notrain < /usr/bin/dspam
X-DSPAM-Result: mergedglobal; result="Innocent"; class="Innocent";
probability=0.0023; confidence=1.00; signature=N/A
theia dspam #
Let me try something stupid and process the binary. No it works:
theia dspam # dspam --user mergedglobal --process --deliver=summary --stdout <
/usr/bin/dspam
X-DSPAM-Result: mergedglobal; result="Innocent"; class="Innocent";
probability=0.0023; confidence=1.00; signature=1,4a42605b98216007443871
theia dspam #
> 1327: [06/15/2009 23:29:18] decode.c:365: unexpected data: header
> string
> '^C"\xdea\x9b\x8c\xcc\xb...@\xfa\xdfm\xcb\xd5\xd8\xbb\xd4\xc2\xbb\xf6
> +C\xf6\x98\xa7\xbed^R\x82\x8b\xa9\xab!\xddM\xe15\xcb\xed\x9e\xb1\xc5c\xc8^L{zr\xd0\x99\x99\x84G\xa4\xdeq\xaf0$\xcf0f^^\x9d%F\x99'
> doesn't cont
> ains `:' character
> 1327: [06/15/2009 23:29:18] decode.c:365: unexpected data: header
> string
> '..\x96/*?^?[]\x8b_\xd8\xa3^MQ\xe4`]^\\xaa\x97^S\xdb^B\xad\xdb%\x88\x
> 90^X\xfb%z\x90]\xc5\x90\xd3=^A\x90KV\x83q\x8a^Q\xe5W\xb3\xf1d3W^[\xdet\xe8\xf7\xac^U\xe7z\xe4\xbe\xbf\xd0^X\xb7\xed"^\\xe2$^B\xe6l\xad\xa2\x93
> \xe4\xa0\xa8\x86\xeeu\xad(\xaf^N\xfdCs~^GZ\xf7!\x89^R\xed6\x8eU\xbe\x89^N\xdf\xaf\x9a\xa7\xba\xb4\x8a
> L\xf8^\^]g\xd5.\xaf\x86ee\x88H\x9b%{\x8c
> y\xefv,\x96]^H\xf3[q\xd8NO\xd3\xf4\xbd^F\x83h\xae}C^P\xc0^C\xbc|4u\xa7\xa8"I\xbc\xe9l{\xe6*\xd5g\xf4^X^]f\xec[zM\xdc\xd6f\xfc^X\xf1M3y\xef~\xe
> 8\x87Y\xcd\xe5x>\x8c^T\\x9en\xd6nf\xb9P\xd4j\xe8\xca#^F\xaf\x93^S\xe90\xe3#\x96\xf2^ST\xb20\xc7\xc4i\xcer3\x94\xc9\xe6\xd4O\x97\xf7,\xf9\xa9\x
> d5nh^D^_\xab\xa0EM\xeckcp\xf8\xd4\xd6833D\xad\xd1%&x\xf4">b>\xbb\xfc\xba{\x8b\xfd"\xdc^W\xe6\xa1(\xce\xe3\x9f\xdf\xd2\xa6\xdfg_e\xb3j^?\xf3\xf
> 6\xd0]%'\xf7j\x91^ZG\xaaS\xf5\x86\xcd\xc6\xa1\x97M^[\xe8^TzU^\~^G}m\xe4D^[=`'
> doesn't contains `:' character
> 1327: [06/15/2009 23:29:18] decode.c:365: unexpected data: header
> string
> 'mj\xa5\x92cWqF\xb3Y^U\xf4\xad\xb3\xa1^R\x86Q^Q"%\x9b\x95\xe7\xe4^V\x
> f5UF\xf4^H,g\xb0I\xdd\xc6 w\xf7\xe7d\xc2\x88\xdaH\x95*T}^Mk&O
> ^R,7!\xc0\x99'\xe3\xce'^l\x...@\xa9v\x8e\x96\xfelo\xf8\x9a^\6@\xca~\xd4\x8
> 3^^.\x9d\x8b\xec\xb2\xb2q0\x80\xe1^]\x97^F\xdb\xe1\x9e1p^H\xf5ikF#\xa4R^S\xee\xe2[\xd9\x88{\xcf\\xbb9P\xb0\xc2?\xbck'
> doesn't contains `:' cha
> racter
> 1327: [06/15/2009 23:29:18] decode.c:365: unexpected data: header
> string '\xbc!\x9c\xb0D\xbb\xea\x93!\xd2\xb2
> \xd1^X\xba\xa0+CH\xb4^?\xea\x9
> b/\xd1(^D\x85^B\xedl\x9fi\xad1O^^>y\x90v\xb3\xbe\xc7\xfe^E\x90\xcb\xf3\x8d^T\xeeA\xc8|\xa0M^[^Xx\xb9Cdf?\x8a\xe0D\xe8\xf3^RX\xaa\xee^]\x8a?\xb
> c\xcf\xe7\xf1^S\xd40\xaf\x9f^Z\xa0\xae;=\xfd\xed' doesn't contains `:'
> character
> 1327: [06/15/2009 23:29:18] decode.c:365: unexpected data: header
> string
> '\xf9a^C\xb6\x98^B\x84\x98b\xc0$R\x96\xbb\xb1&\xb0Z,s7^QlV\xa0\xf1\x8
> 9^U~^U\xd8^B\x80E\xfcg]G^^\xef\x94b
> \x95\xe6\x97\x83\xfa^P\xcb\xb1\x94*\xb5\x90\xf6\xf0Z^C\xbe\xfd\xfb^B7\xdf\xfa\xbb~2\xd25^Q\xd0\x87\xbc\xbd
> \xe3\xb2Fo(\xe8\xf4"\xf4"c\xe8{\xd6\x87M\xd4\x92\xc8\xc6\xa5\xa9\xa7
> \xeeU\xc5J\xeb\xed\x97A6a\xa7\xe9\xbd^FX^T4\xc2]\xff3\xe2X\xb7\xac\xc8
> >L\x80\x9b\x83K\xd2W\xed3n\x9f\xed^D\x9d\xb6' doesn't contains `:'
> character
> 1327: [06/15/2009 23:29:18] decode.c:365: unexpected data: header
> string
> '\xafH\x93t}\xa9\xb6b8S%~G;\xae\xed\xdcvX\xd6\xb1\x84\xbe^[\xa4\xcd\x
> aa\xf21r\xd9j^]B\xbb\xe6\xa4\x8e]"G^L&,_\xec\xa0"^T\xdc\xb0\x88t6\xe7iH\xcdO\xc4)\xde<~\x93^B\xbb$^G^A4^H\xc9\x87\x80\xfeex\xe0^Os\xfeD/\xc6\x
> aah\xdf^D\xfc\x94\xbf\xa4\xc20\xa0\xa7^ER\xf2^Vzs\xfd^[\xf1\xf2T^^E$\xfb+\x95\xd3{^F\xfd\xb0\x8c\xca'
> doesn't contains `:' character
> 1327: [06/15/2009 23:29:18] decode.c:365: unexpected data: header
> string
> '\xe6?\xd3\x85'\xdf\x8e\x8f\x98\xf8\x96"\xab\x8e^P\xa5\xb6\xfb^O\xb5\
> xbeE>N]FP=\x86UP\xe9SOM\xee\xf5\x93\xf5\xdf\x84\xda~\x86
> \^E^U^W\xac^Z\xda\x8fg\x96^D^Q^V\xc9O^N\xcfc\xb7\x83\x8e\x97\xba^K}0\x80\xab\x
> d4^_h\xa4^_,%lJ\xea\xcb\xd5\xad\xc7\xc6
> \xcc^Bsi^A8\xdd\xdb\xef\xd5'\x8b%\xf5T\x97s\x87\xfbn\x86\xa0tm\xf0\xc4r\xa96Z\xa7\xb6\xf9\xdfl\x8ao\x9
> 9P\x88<\xe2\xfe\...@\xf5\xa3us7\xfa\x8b\xe8\xa8
> \xf8X\xde\xba>~\xd2\xc0\x89\x80F\xf9^O\x91HP%H\xa4\xf5\xe2^Q\x9c^U\xcd\xb1\x88\x98\x8b\xc3\x9a
> j$^M^_\xca4\x91' doesn't contains `:' character
>
Something ultra strange is happening here. Do you have by accident the above
message? Could you send it to me? Is this with the PostgreSQL driver?
> 1327: [06/15/2009 23:31:01] DSPAM Instance Shutdown. Exit Code: 0
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back
> the current transaction and exit, because another server process
> exited
> abnormally and possibly corrupted shared memory.
>
Shared memory? What shared memory? This looks to me coming from PostgreSQL but
not from DSPAM.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
>
That looks like a message from your RDBMS but not from DSPAM. Could that be?
> 1327: [06/15/2009 23:33:59] _ds_getall_spamrecords() failed
>
This line is sure from DSPAM. But from an older release of the DSPAM PostgreSQL
driver. I changed all the output to be in the format "<function name>: <error
message>".
Can you reproduce that error? I would kill for getting that bug ironed out of
3.9.0.
Steve
> On Tue, Jun 16, 2009 at 2:31 PM, Steve<[email protected]> wrote:
> >> Hello,
> >>
> > Hello
> >
> >
> >> I am running dspam git version on FreeBSD 6.3 64 bit platfrom (6GB RAM
> >> , 4GB swap)
> >>
> >> I installed dspam last week but getting out of swap space recently.
> >>
> >> pid 1327 (dspam), uid 0, was killed: out of swap space
> >> swap_pager_getswapspace(4): failed
> >> swap_pager_getswapspace(3): failed
> >> swap_pager_getswapspace(3): failed
> >> swap_pager_getswapspace(4): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(4): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(2): failed
> >> swap_pager_getswapspace(2): f
> >>
> > That's not good. However... that does not mean that DSPAM is the one to
> blame. It just means that you run out of resources and that DSPAM is/was
> the first one to be killed.
> >
> > Anyway... I would be interested to hear more about the case. Did you had
> the issue running out of resources with the old DSPAM too? How fast do you
> run out of resources? Would it be possible to run without DSPAM and see if
> you still run out of resources?
> >
> >
> >> Here is the last entry in Changelog(in order to find the version in git
> >> repo)
> >> # more CHANGELOG
> >> Version 3.9.0
> >> -------------
> >> [20090606:1043] sbajic: Fixing build problems when building against
> >> SQLite3 3.6 series
> >> * Fixed issues with m4 macro when building against SQLite 3.6.x [Bug
> >> ID #2774657]
> >>
> >>
> >> # dspam --version
> >>
> >> DSPAM Anti-Spam Suite GIT (agent/library)
> >>
> >> Copyright (c) 2002-2009 DSPAM Project
> >> http://dspam.sourceforge.net.
> >>
> >> I am using pgsql (8.3.0) storage (vacuuming every night)
> >>
> >> and running dspam with --daemon --debug parameter and training with
> >> dspam --client --user spammaster --class=spam --source=corpus
> >>
> >> database entries:
> >>
> >> dspam=# SELECT count(*) from dspam_signature_data ;
> >> count
> >> -------
> >> 1037
> >> (1 row)
> >>
> > That should not bring DSPAM down.
> >
> >
> >> dspam=# SELECT count(*) from dspam_token_data ;
> >> count
> >> ---------
> >> 6333844
> >> (1 row)
> >>
> > This too is not high enough to bring DSPAM down.
> >
> >
> >> # dspam_stats -H
> >> spammanager:
> >> TP True Positives:
> 754
> >> TN True Negatives:
> 283
> >> FP False Positives:
> 0
> >> FN False Negatives:
> 0
> >> SC Spam Corpusfed:
> 3979
> >> NC Nonspam Corpusfed: 3059
> >> TL Training Left:
> 0
> >> SHR Spam Hit Rate
> 100.00%
> >> HSR Ham Strike Rate: 0.00%
> >> PPV Positive predictive value: 100.00%
> >> OCA Overall Accuracy: 100.00%
> >>
> >>
> >> Here is dspam.conf
> >>
> >> Home /usr/local/var/dspam
> >> StorageDriver /usr/local/lib/dspam/libpgsql_drv.so
> >>
> > Okay. PostgreSQL driver....
> >
> >
> >> OnFail error
> >> Trust root
> >> Trust dspam
> >> Trust apache
> >> Trust mail
> >> Trust mailnull
> >> Trust smmsp
> >> Trust daemon
> >>
> >> TrainingMode toe
> >> TestConditionalTraining on
> >> Feature noise
> >> Feature whitelist
> >> Algorithm graham burton
> >> Tokenizer chain
> >> PValue bcr
> >> WebStats off
> >>
> >>
> >> Preference "showFactors=off"
> >> Preference "spamAction=deliver"
> >> Preference "whitelistThreshold=20"
> >> Preference "spamSubject=[SPAM]"
> >>
> >> AllowOverride trainingMode
> >> AllowOverride spamAction spamSubject
> >> AllowOverride statisticalSedation
> >> AllowOverride enableBNR
> >> AllowOverride enableWhitelist
> >> AllowOverride signatureLocation
> >> AllowOverride showFactors
> >> AllowOverride optIn optOut
> >> AllowOverride whitelistThreshold
> >>
> >>
> >> PgSQLServer /tmp
> >> PgSQLPort 5432
> >> PgSQLUser dspamuser
> >> PgSQLPass dspampass
> >> PgSQLDb dspam
> >>
> >> PgSQLConnectionCache 30
> >>
> > 30 connections to PostgreSQL. Do you really need that much?
> >
> >
> >> PgSQLUIDInSignature on
> >> HashRecMax 98317
> >> HashAutoExtend on
> >> HashMaxExtents 0
> >> HashExtentSize 49157
> >> HashPctIncrease 10
> >> HashMaxSeek 10
> >> HashConnectionCache 30
> >>
> >>
> >
> > Would it be possible for you to run DSPAM over valgrind and look if you
> have any memory leaks with it? I think most memory leaks are fixed by now
> but still... if you find one, then please post it here so we can track it
> down and fix it.
> >
> > Thank you for testing 3.9.0 and reporting here.
> >
> >
> > // Steve
> > --
> > GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
> > Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
> >
> >
> ------------------------------------------------------------------------------
> > Crystal Reports - New Free Runtime and 30 Day Trial
> > Check out the new simplified licensing option that enables unlimited
> > royalty-free distribution of the report engine for externally facing
> > server and web deployment.
> > http://p.sf.net/sfu/businessobjects
> > _______________________________________________
> > Dspam-user mailing list
> > [email protected]
> > https://lists.sourceforge.net/lists/listinfo/dspam-user
> >
--
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
------------------------------------------------------------------------------
_______________________________________________
Dspam-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspam-user
