Bugs item #793830, was opened at 2003-08-24 05:41 Message generated for change (Settings changed) made by anadelonbrin You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=793830&group_id=61702
Category: Outlook Group: Outlook 008 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Juergen Kremp (kremp) Assigned to: Mark Hammond (mhammond) Summary: Last message arrived on Server not filtered when starting Initial Comment: Hi, I use Outlook Plugin Version 007. It is used in an Environment of a central Microsoft Exchange 2000 Server to which I connect with an Outlook 2000 client. I observe that when starting the client, all messages in the Inbox that arrived during the time period where the Client was unavailable are filtered by SpamBayes, ***except*** the last one that arrived. It is unfiltered and shown with 0% rate. When manually rating (Show spam clues...), the rate indicates that it is clearly Spam. Also performing a complete rating/filtering (Filter now) rates the message correctly and moves it accordingly. If Outlook is permamently running, I do not observe such effect. It seems to be related to the code initially examining the Inbox when starting Outlook. Juergen Kremp ---------------------------------------------------------------------- Comment By: Juergen Kremp (kremp) Date: 2004-07-20 01:40 Message: Logged In: YES user_id=850764 I got new hardware with changed operating system (Microsoft Windows XP Professional) and installed 1.0rc1. In this configuration, the problem does not occur. ---------------------------------------------------------------------- Comment By: Jim Brown (brown2611) Date: 2004-02-10 03:31 Message: Logged In: YES user_id=963745 I believe this is the same as bug #876281. As far as I can tell, having only started looking at Python and the SpamBayes Windows code yesterday, here is why this happens: 1. When SB starts, it (eventually) invokes BayesManager.EnsureOutlookFieldsForFolder() for each folder. This method finds the first MalItem in the specified folder and looks for the "Spam" UserProperty attached to that MailItem. If the MailItem does not have the "Spam" UserProperty, SB adds the UserProperty to the MailItem, causing the MailItem to have a Spam score of 0. 2. In trying to process missed messages, SB invokes MAPIMsgStoreFolder.GetNewUnscoredMessageGenerator(), which excludes any MailItem that already has the "Spam" UserProperty from processing. Voila. The item most recently received in a folder while SB was not running is automatically assigned a score of 0 and not filtered. Apparently, there isn't a clean way to detect if a folder has a particular user defined field. So, the only way to check for the presence of the user defined field in the folder is to check the items in the folder. SB (correctly) assumes that adding the UserProperty to the first MailItem will force the creation of the user defined field in the folder. However, this is not a benign act. Possible solutions: * Don't force the creation of the user defined field until SB has an actual score to store. However, I fear there may be a great number of places in the code that assume the Spam field already exists. For example, GetNewUnscoredMessageGenerator() certainly makes this assumption. The error resulting from the lack of the Spam field could be trapped, but I don't know the code well enough to find all the places where the absence of the field might be a problem. Nonetheless, this seems to be the correct solution to me. * EnsureOutlookFieldsForFolder() could check more than just the first MailItem in the folder. However, this doesn't avoid the problem if every message in the folder is a missed message. * If SB is going to force the creation of the Spam field, go ahead and filter the message. Aside from not being terribly clean, its not clear to me that enough of the code has been initialized by this point to filter a message. * If SB is going to force the creation of the Spam field, initialize it to a value that is easily detected as unscored, for example -1. However, I'm not confident that the code doesn't depend on 0 <= Spam <= 100. In addition, I don't know how many places in the code would have to be changed to recognize this value. * Change the minimum score to 0.0001, or the like, and detect a score of 0 as an unscored message. Kludgey and one would still have to find all of the places where something with a ham-like score (or any score) is excluded from further filtering (e.g., addin.ProcessMessage()). * Since OutlookAddin.ProcessMissedMessages() only _seems_ effectively to be invoked at startup, it could be modified to always process the first MailItem if the item is unread and has a score of exactly zero. Again, kludgey, but at least the kludge is confined to a single place. On the other hand, it looks to me like SB has already hooked into the folders it is watching at this point. if SB is filtering in the background, I suspect that any mail received before the "processing start delay" expires would bump the improperly flagged message out of the first position. Hmmm. * One could treat anything with a score of 0 as an unscored message, but it isn't really desirable to rescore all of those messages, since the majority of them have presumably already been filtered correctly. Any feedback from someone who actually knows the code? ---Jim ---------------------------------------------------------------------- Comment By: Jim Brown (brown2611) Date: 2004-01-30 18:00 Message: Logged In: YES user_id=963745 I can confirm this problem with Plugin 0.81 on Outlook 2002 SP-2 and XP Pro with Exchange 2000. I'm running SpamBayes in background mode with a 1.0 sec start delay and a 0.5 sec delay between messages. It appears that SpamBayes fails to find and filter the most recently received email message when Outlook is started. Example: I started Outlook at about 11:40 PM. The most recent email in my inbox was received at 10:57 PM, when Outlook was not running. Here's the log from SpamBayes with verbosity 1. It clearly shows that SpamBayes did not find the message upon startup: --- System verbosity set to 1 No old pickle file to migrate Loaded bayes database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_bayes_database.db' Loaded message database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_message_database.db' Bayes database initialized with 25710 spam and 22311 good messages Loaded databases in 11.4339ms SpamBayes Outlook Addin, Binary version 0.81 (September 9, 2003) starting (with engine SpamBayes Beta2, version 0.2 (July 2003)) on Windows 5.1.2600 (Service Pack 1) using Python 2.3+ (#46, Aug 6 2003, 16:39:24) [MSC v.1200 32 bit (Intel)] SpamBayes: Watching for new messages in folder Inbox SpamBayes: Watching for new messages in folder Junk (pending) Processing missed spam in folder 'Inbox' by starting a timer New message timer started - id=25680, delay=1000 FAILED to add the toolbar item 'SpamBayesCommand.Manager' - (- 2147352567, 'Exception occurred.', (0, None, None, None, 0, - 2147467259), None) Deleted the dead popup control - re-creating The timer with id=25680 fired The new message timer found no new items, so is stopping --- ---------------------------------------------------------------------- Comment By: Jim Brown (brown2611) Date: 2004-01-30 17:57 Message: Logged In: YES user_id=963745 I can confirm this problem with Plugin 0.81 on Outlook 2002 SP-2 and XP Pro with Exchange 2000. I'm running SpamBayes in background mode with a 1.0 sec start delay and a 0.5 sec delay between messages. It appears that SpamBayes fails to find and filter the most recently received email message when Outlook is started. Example: I started Outlook at about 11:40 PM. The most recent email in my inbox was received at 10:57 PM, when Outlook was not running. Here's the log from SpamBayes with verbosity 1. It clearly shows that SpamBayes did not find the message upon startup: --- System verbosity set to 1 No old pickle file to migrate Loaded bayes database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_bayes_database.db' Loaded message database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_message_database.db' Bayes database initialized with 25710 spam and 22311 good messages Loaded databases in 11.4339ms SpamBayes Outlook Addin, Binary version 0.81 (September 9, 2003) starting (with engine SpamBayes Beta2, version 0.2 (July 2003)) on Windows 5.1.2600 (Service Pack 1) using Python 2.3+ (#46, Aug 6 2003, 16:39:24) [MSC v.1200 32 bit (Intel)] SpamBayes: Watching for new messages in folder Inbox SpamBayes: Watching for new messages in folder Junk (pending) Processing missed spam in folder 'Inbox' by starting a timer New message timer started - id=25680, delay=1000 FAILED to add the toolbar item 'SpamBayesCommand.Manager' - (- 2147352567, 'Exception occurred.', (0, None, None, None, 0, - 2147467259), None) Deleted the dead popup control - re-creating The timer with id=25680 fired The new message timer found no new items, so is stopping --- ---------------------------------------------------------------------- Comment By: Jim Brown (brown2611) Date: 2004-01-30 17:57 Message: Logged In: YES user_id=963745 I can confirm this problem with Plugin 0.81 on Outlook 2002 SP-2 and XP Pro with Exchange 2000. I'm running SpamBayes in background mode with a 1.0 sec start delay and a 0.5 sec delay between messages. It appears that SpamBayes fails to find and filter the most recently received email message when Outlook is started. Example: I started Outlook at about 11:40 PM. The most recent email in my inbox was received at 10:57 PM, when Outlook was not running. Here's the log from SpamBayes with verbosity 1. It clearly shows that SpamBayes did not find the message upon startup: --- System verbosity set to 1 No old pickle file to migrate Loaded bayes database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_bayes_database.db' Loaded message database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_message_database.db' Bayes database initialized with 25710 spam and 22311 good messages Loaded databases in 11.4339ms SpamBayes Outlook Addin, Binary version 0.81 (September 9, 2003) starting (with engine SpamBayes Beta2, version 0.2 (July 2003)) on Windows 5.1.2600 (Service Pack 1) using Python 2.3+ (#46, Aug 6 2003, 16:39:24) [MSC v.1200 32 bit (Intel)] SpamBayes: Watching for new messages in folder Inbox SpamBayes: Watching for new messages in folder Junk (pending) Processing missed spam in folder 'Inbox' by starting a timer New message timer started - id=25680, delay=1000 FAILED to add the toolbar item 'SpamBayesCommand.Manager' - (- 2147352567, 'Exception occurred.', (0, None, None, None, 0, - 2147467259), None) Deleted the dead popup control - re-creating The timer with id=25680 fired The new message timer found no new items, so is stopping --- ---------------------------------------------------------------------- Comment By: Jim Brown (brown2611) Date: 2004-01-30 17:56 Message: Logged In: YES user_id=963745 I can confirm this problem with Plugin 0.81 on Outlook 2002 SP-2 and XP Pro with Exchange 2000. I'm running SpamBayes in background mode with a 1.0 sec start delay and a 0.5 sec delay between messages. It appears that SpamBayes fails to find and filter the most recently received email message when Outlook is started. Example: I started Outlook at about 11:40 PM. The most recent email in my inbox was received at 10:57 PM, when Outlook was not running. Here's the log from SpamBayes with verbosity 1. It clearly shows that SpamBayes did not find the message upon startup: --- System verbosity set to 1 No old pickle file to migrate Loaded bayes database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_bayes_database.db' Loaded message database from 'C:\Documents and Settings\brown\Application Data\SpamBayes\default_message_database.db' Bayes database initialized with 25710 spam and 22311 good messages Loaded databases in 11.4339ms SpamBayes Outlook Addin, Binary version 0.81 (September 9, 2003) starting (with engine SpamBayes Beta2, version 0.2 (July 2003)) on Windows 5.1.2600 (Service Pack 1) using Python 2.3+ (#46, Aug 6 2003, 16:39:24) [MSC v.1200 32 bit (Intel)] SpamBayes: Watching for new messages in folder Inbox SpamBayes: Watching for new messages in folder Junk (pending) Processing missed spam in folder 'Inbox' by starting a timer New message timer started - id=25680, delay=1000 FAILED to add the toolbar item 'SpamBayesCommand.Manager' - (- 2147352567, 'Exception occurred.', (0, None, None, None, 0, - 2147467259), None) Deleted the dead popup control - re-creating The timer with id=25680 fired The new message timer found no new items, so is stopping --- ---------------------------------------------------------------------- Comment By: Pete Marion (ima_user) Date: 2004-01-10 00:26 Message: Logged In: YES user_id=923210 I have confirmed this running Windows XP Home and Outlook 2002 sonnecting to a POP / SMTP server. Outlook Addin, Binary version 0.81 (September 9, 2003). ---------------------------------------------------------------------- Comment By: John Marrett (jmarrett) Date: 2003-10-24 01:17 Message: Logged In: YES user_id=642463 I am experiencing the same problem with SpamBayes version 0.81 (September 9, 2003) connected to Microsoft Exchange server 5.5. ---------------------------------------------------------------------- Comment By: Juergen Kremp (kremp) Date: 2003-10-11 01:05 Message: Logged In: YES user_id=850764 I installed Outlook Addin, Binary version 0.81 (September 9, 2003). It shows the same effect. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498103&aid=793830&group_id=61702 _______________________________________________ Spambayes-bugs mailing list [email protected] http://mail.python.org/mailman/listinfo/spambayes-bugs
