"Jason R. Mastaler" <[EMAIL PROTECTED]> writes:
> Let's distingiush between reading the data and matching an address
> though. Not taking TMDA wildcards into consideration, what if the
> "whitelist" in the DB contains 100K e-mail addresses.
Well, you've been using TMDA longer than any of the rest of us. :)
How big is your text whitelist on disk? Is 100K even close to likely,
per user?
> That is going to be extremely slow to search in TMDA, but probably
> much faster to do the search in the database, no?
Absolutely. You get the advantage of not pulling data and the
additional advantage of compiled C (most likely) doing the searching.
> If this is the case, would it be possible to only pull the list into
> Python for matching if it contains wildcards? Perhaps a '-wildcards'
> option or something could be added to from/to-sql.
I think that's an interesting compromise. If there are no wildcards,
a search is easy. It will require more work in the firstmatch() code,
because we'll still want the end user/administrator to provide the
SELECT statement, but we'll have to insert
sender = <sender1> OR sender = <sender2> ...
into the middle of it somewhere. The list of senders comes from the
parameter to the firstmatch() function.
There is one rather large problem with the idea in the case of
outgoing mail. A detail that I didn't mention in my original
description is that, because firstmatch() is called once for each
recipient in tmda-inject, the database would be hit each time.
My solution in the implementation was to cache the list that I
retrieved from the database and only make the request if I had no
cache. For the second and following recipients, I just search the
list already in memory. If, however, we use the database to perform
the search, then we'll again be hitting it for each recipient. That
may be ok, since we're not hauling back an entire list every time.
We'd probably have to work that out empirically, on a reasonably busy
system.
If it does turn out to be a problem, I have no answer for it off the
top of my head.
Tim
_________________________________________________
tmda-workers mailing list ([EMAIL PROTECTED])
http://tmda.net/lists/listinfo/tmda-workers
