There are several different algorithms that can be used to search text. A complete linear search would not be the best choice for performance and therefore was not used. Adding more entries should not cause a performance problem, but with any search the less you have to search the quicker the search can take place.
You have to be careful when removing all the variants in the list because you may end up blocking valid domains. For example these entries are in the list: 8bit.co.uk, bessdeals.co.uk and blue-one.co.uk. If you removed all the variants you would just have co.uk in the list and that may block valid email. There is a command line utility being fine tuned that will remove duplicates from the phrase list and the url domain black list as well as automatically remove variants down to a minimum of 3 labels (if a.b.c.d and b.b.c.d are in the list it would be changed to b.d.c. If a.b.c.d.e and b.b.c.d.e are in the list it would be changed to b.c.d.e.) It will also have a mode where the variants in the list will be displayed so the user can specify how many labels to keep for those domains. This utility will also merge multiple lists. This utility should be included with the next update to the spam filters. Tripp ----- Original Message ----- From: "FastWave" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, November 18, 2003 4:41 PM Subject: Re: [IMail Forum] 'url-domain-bl.txt' Questions > Thanks, Tripp, for the verification on domain name variants of form 'xxx.domain.com' in the 'url-domain-bl.txt' file. > > I am inferring from your reply on whether or not the 'url-domain-bl.txt' file needs to be sorted that IMail performs an exhaustive linear search (ie. the entire 'url-domain-bl.txt' file is searched for exclusion of a domain name). That means that as entries are added to the 'url-domain-bl.txt' file, it will progressively take longer and longer to search the file. Therefore, would it not be more efficient to remove domain name variants in the file and replace them with a single domain name entry? > > The reason I am asking is that I performed a right-to-left sort of the Ipswitch 'url-domain-bl.txt' distribution file which revealed a high number of domain name variants. I would like to know the reason (technical, political, liability, etc.) why Ipswitch does not replace the many domain name variants in this file with a single domain name. If Ipswitch chooses not to do this with file distributions, I would still like to do this for our own 'url-domain-bl.txt' file and was wondering what the possible drawbacks are. > > All the best! > > Kim > > > ---------- Original Message ---------------------------------- > From: "Tripp Allen" <[EMAIL PROTECTED]> > Reply-To: [EMAIL PROTECTED] > Date: Mon, 17 Nov 2003 18:39:16 -0500 > > >> My understanding is that if 'bubb-rubb.biz' is listed in the > >'url-domain-bl.txt' file, that alone will eliminate all variants of the > >form: > >> > >> xxx.bubb-rubb.biz > >> > > > >That's correct. > > > >> Also, does the file 'url-domain-bl.txt' have to be ASCII sorted to work > >correctly > > > >No, sorting does not make a difference. > > > >Tripp > > > > > >To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html > >List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ > >Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/ > > > > To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html > List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ > Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/ > To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
