Matt Juszczak wrote:
Paul,
Sure, but I'd also send "high priority" and "low priority" emails
through separate systems (sendmail/postfix/whatever instances.)
Well, what's to stop me from using the same database table for high
priority and low priority, but having the high priority background
process continuously loop to check for new items in the queue?
Even if I have the webs send out high priority directly, like you
said, that could cause some damage.
Perhaps I could create a centralized relay server that the webs use
for high priority, and have the webs send out high priority mail. But
at that point, I unfortunately have to duplicate the high priority
mail code =(
Ok, you're talking about two sorts of queue here.
(i) there's the queue of an SMTP-compliant mailer (I assume), and
(ii) there's a queue that you're maintaining of messages you want to
send; this might be a set of database rows, one per message, and
you're doing a "mail merge" process to fill in a template and push
messages gradually into queue (i)
Presumably you've got some rate control on (ii), and the mail merge
process is watching the length of the queue in (i) (and maybe some other
variables), so that you can control the load of the SMTP server.
One trouble with this is that a certain fraction of mail takes a
long time to deliver; an SMTP-compliant mailer will keep trying to
deliver a message for seven days. If you're sending enough mail,
you're eventually going to get a large "plug" of stuck messages that the
mail server is going to try to keep delivering and re-delivering.
Ultimately this is going to burn up resources on the mail server, which
will impact other things running on that machine: such as high-priority
mails you want to send.
Now, process (ii) can certainly stop putting messages in (i) once
the "plug" of stuck messages reaches a significant size, but that's
going to really slow down the bulk mail.
The main factor in mail server performance is the effect of fsync()
calls on a mechanical disk: mail delivery events really ought to be
transactional, since you don't want to deliver mail twice or fail to
deliver it. Fsync() doesn't (honestly) return until a chunk of metal
moves to a certain place; the bottleneck isn't so much like "you can do
so many a second" but more like a systemwide lock, since there can be
multiple processes trying to fsync(), such as syslogd or a database
server that's committing a transaction. The traffic jam can get backed
up, since other processes can be waiting for the the first process to
complete, can be holding more locks and so forth... So you end up with
a situation where the performance bottleneck is a real pain to
understand... It might even take you 5 minutes to get to a shell prompt
when you ssh in.
If you're on a virtual server, you may (or may not) have somebody
else doing a lot of fsync() calls, in which case your performance could
be hosed for reasons outside your observation and control. Just as
likely, the server will be programmmed to return from fsync() before
the fsync is done, which means someday you're going to have a big
database wreck...
For $200 a month you can rent a server that will do a great job
delivering email. Or you can spend 10-100x that trying to figure
problems out.
_______________________________________________
New York PHP Users Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk
http://www.nyphp.org/Show-Participation
