Re: Multi-RCPT vs. Single RCPT delivery - logic error?
On Fri, 28 Apr 2000, Dave Kitabjian wrote: We had a situation with a customer who was consulting for a college. So every few days, she had to send a 10MB PowerPoint file to about 50 recipients at that college. Under qmail, a separate thread was opened up for each qmail-remote. You are observing a rare corner of the time/space continuum for email. The DJB solution (see archives), which of course make perfect sense, is for someone at the college to run a local exploder to which your customer sends a single message. -- Jeff Hayward
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
On Fri, 28 Apr 2000 15:02:06 -0500 (EST), Chris Hardie [EMAIL PROTECTED] said: Chris we do web development for an organization that has a PR firm Chris develop brochures and then send them to us for posting on their Chris website. The files are often 7-10 MB in size, large enough to Chris be cumbersome for e-mail, small enough to make overnighting a Chris ZIP disk seem a little excessive. [...] Chris I'd be interested to hear if anyone's found a good general Chris solution to this in a production/business environment. There are a number of sites out there which allow WAN-based file sharing. At the risk of getting flamed... I worked on one which is targetted at exactly this problem. The "sender" uploads the file via WWW, selects who he wants to "send" it to, they get an email notification and download the file via the URL provided in the mail. It ain't rocket-science and folks have found it easy to use. It's totally free, there's currently no advertising, and only the sender has to get an account to upload. Currently user space is 50MB and files expire after 7 days. See: http://www.WhaleMail.com/
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
Dave Kitabjian [EMAIL PROTECTED] wrote: Regarding: http://web.infoave.net/~dsill/lwq.html#multi-rcpt Dave S, I'm having trouble accepting this logic. You mention 3 options: "Say you're an MTA, and one of your users sends a message to three people on hostx.example.com. There are several ways you could do this. 1. You could open an SMTP connection to hostx, send a copy of the message to the first user, send a copy to the second user, send a copy to the third user, then close the connection. 2. You could start three processes, each of which opens an SMTP connection to hostx, sends a copy of the message to one of the users, then closes the connection. 3. You could open an SMTP connection to host, send a copy of the message addressed to all three users, then close the connection. " and that qmail uses option #2. Clearly, the rank of efficiency is, from best to worst,: 3, 1, 2 That's the rank of bandwidth efficiency, yes. Bandwidth isn't the only consideration, though, and I think this section covers the others pretty well (speed, VERP's, simplicity of design). We had a situation with a customer who was consulting for a college. So every few days, she had to send a 10MB PowerPoint file to about 50 recipients at that college. Under qmail, a separate thread was opened up for each qmail-remote. 1) qmail was designed for well-connected hosts, 2) SMTP was not designed for this type of use. FTP or HTTP would be better because they're "pull" protocols, not "push" protocols, so they allow those with the desire and the bandwidth to retrieve the files, and, 3) this is one of those pathological cases I referred to where single RCPT is slower than multi-RCPT. We love qmail, and it's working very well for us in general. But I'm having a hard time reconciling your logic in this paragraph. Perhaps you could clarify for me. I don't see where my login needs reconciling. I said "Single RCPT delivery does use more bandwidth than multiple RCPT delivery..." and "there are pathological cases where it can be slower than multiple RCPT", but that's not inconsistent with single RCPT being faster in most cases and better overall. -Dave
RE: Multi-RCPT vs. Single RCPT delivery - logic error?
Thank you all for your feedback. I don't see where my login needs reconciling. Yes, I realized after I sent the email that your logic was not flawed. (Your login is probably okay, too ;) Thanks again. Dave (K)
Re[2]: Multi-RCPT vs. Single RCPT delivery - logic error?
Hello Chris, if you're a Mac user and have a (free) account on mac.com, you can mount a folder from there on your desktop to drop things on and any other mac.com user can mount it to read your files. That's more convenient than sending email, Windows users should take a look at FTP Netdrive, should be available somewhere at TUCOWS which gives them the ability to mount a FTP account as normal drive. Best regards, Gabriel
Multi-RCPT vs. Single RCPT delivery - logic error?
Regarding: http://web.infoave.net/~dsill/lwq.html#multi-rcpt Dave S, I'm having trouble accepting this logic. You mention 3 options: "Say you're an MTA, and one of your users sends a message to three people on hostx.example.com. There are several ways you could do this. 1. You could open an SMTP connection to hostx, send a copy of the message to the first user, send a copy to the second user, send a copy to the third user, then close the connection. 2. You could start three processes, each of which opens an SMTP connection to hostx, sends a copy of the message to one of the users, then closes the connection. 3. You could open an SMTP connection to host, send a copy of the message addressed to all three users, then close the connection. " and that qmail uses option #2. Clearly, the rank of efficiency is, from best to worst,: 3, 1, 2 -- We had a situation with a customer who was consulting for a college. So every few days, she had to send a 10MB PowerPoint file to about 50 recipients at that college. Under qmail, a separate thread was opened up for each qmail-remote. a) That means a total transfer of 500MB rather than simply 10MB. b) Secondly, since all outbound threads were tied up for a long time, all other mail was deferring, causing customer complaints. c) Finally, since all 50 messages were being received by the same remote SMTP server, the transfer was bottlenecked by the cpu and i/o of their single mail relay trying to receive 50 copies of the same thing. We love qmail, and it's working very well for us in general. But I'm having a hard time reconciling your logic in this paragraph. Perhaps you could clarify for me. Dave (K)
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
Thus said Dave Kitabjian on Fri, 28 Apr 2000 12:12:44 EDT: We had a situation with a customer who was consulting for a college. So every few days, she had to send a 10MB PowerPoint file to about 50 recipients at that college. Under qmail, a separate thread was opened up for each qmail-remote. a) That means a total transfer of 500MB rather than simply 10MB. b) Secondly, since all outbound threads were tied up for a long time, all other mail was deferring, causing customer complaints. c) Finally, since all 50 messages were being received by the same remote SMTP server, the transfer was bottlenecked by the cpu and i/o of their single mail relay trying to receive 50 copies of the same thing. I may be rehashing old topics, and I may sound a little bit old fashioned (even at age 26), but I don't believe email was ever meant to handle that large amount of traffic. Or, in other words SMTP != FTP I am still of the opinion that one should instruct users to use the right protocols for the right reasons. Hence, put the 10MB PowderPoint file in a public or private ftp directory and then include a URL to fetch it in the email. I realize that this doesn't necessarily fix what you may think is a problem, but this is how I would handle it. :-) Andy -- +== Andy == TiK: garbaglio ==+ |Linux is about freedom of choice| +== http://www.xmission.com/~bradipo/ ===+ PGP signature
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
Dave Kitabjian [EMAIL PROTECTED] wrote: Dave Sill: "Say you're an MTA [ sending three messages. you could: ] 1. ...[ send three copies through one connection, ]...then close the connection. 2. ...start three processes, each of which...sends a copy of the message... 3. ...send a [ single ] copy of the message addressed to all three users..." Dave Kitabjian: Clearly, the rank of efficiency is, from best to worst,: 3, 1, 2 That might be true, if the situation you described were complete, but it's not. The MTA handles hundreds (thousands) of messages per day, and in typical situations a very small fraction of them are both 1) the same message, and 2) bound for the same host. For qmail to implement solution #1 or #3 means that qmail must identify any mail traffic it can combine, and then must handle it specially. That takes some work. The extra code becomes a possible source of bugs and security holes. Does it speed qmail up noticeably? No, for several reasons: 1. Only a tiny percentage of email can even potentially benefit from this ``optimization''. Hence, even a large speedup would have a small overall effect. (The only major exception to this is mailing list traffic. If enough of your local users subscribe to a remote mailing list, then it's worth your while to set up a sublist.) (The other major exception, corporate email, isn't an exception at all. Corporate email typically runs at LAN speeds, where the difference in speed is negligible.) 2. Connection caching (strategy #1) is actually terrible for performance, because all mails for a given destination are serialized. They would arrive faster if they were delivered in parallel, which is what qmail does. Connection caching also impairs the remote mail admins' ability to limit throughput to levels his server can handle. If his server goes down temporarily, for example, then you will probably try to shove thousands of emails down his throat as soon as you see he's back up. This effect is what Dan calls ``opportunistic bombardment''; it's why sendmail typically clobbers recovering mail hosts. (After an outage, AOL typically is taken down again, several times, by incredible waves of sendmail bombardment.) Connection caching is also unfair. It means that once you have a connection, you exploit it to send everything you've got. Meanwhile, if the server is near capacity, others are completely denied service. qmail'l parallel delivery, which at first seems more greedy, is actually fairer--admins can easily limit per-site connections, causing qmail to wait its turn. Meanwhile sendmail users hog up connections, forcing their mail through without waiting their turn. 3. Option #3 is mainly harder for the outgoing server; it means that the server has to notice opportunities to combine emails into one message with several recipients. a. There are cases where a server will be fooled anyway. b. Ignoring that, his work will pay off only in a tiny minority of cases. c. This approach also has privacy implications: if I give a message with multiple recipients to another SMTP server, and some of those are BCC recipients, then the upstream server may violate privacy be recording the complete envelope. d. This approach makes things like VERPs impossible. VERPs are why ezmlm can handle bounces so conveniently. They actually have convenient uses for individuals, as well: it lets you map bounces of important emails to the exact address you were sending to (which may not be the address which caused the bounce). So your logic is fine by itself, but it misses the bigger picture of what's going on with email traffic. Len. -- Frugal Tip #4: Keep skipping town two days ahead of the collection agency.
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
Dave Kitabjian [EMAIL PROTECTED] wrote: We had a situation with a customer who was consulting for a college. So every few days, she had to send a 10MB PowerPoint file to about 50 recipients at that college. Under qmail, a separate thread was opened up for each qmail-remote. Warn her, firmly, to send Zip disks by email or to rent space on an FTP server. If she persists, kill her. Then find and kill the manager who would email me 6MB presentations, clogging my modem for half an hour--instead of waiting 45 minutes and giving it to me when I arrived at work. a) That means a total transfer of 500MB rather than simply 10MB. Right! That's not what email is for! Not only your pipe is being abused; the downstream servers are, too. And think of the poor recipients! If they use modems, then they will be _extremely_ ticked at the wait. Worse, many POP servers and/or clients will jam on such large messages, forcing retries. Some will fail forever, blocking receipt of any further emails until someone at the ISP deletes the offending message. This is a MAJOR case for customer education. Len. -- Frugal Tip #65: Get a cushy TMF job where you can get away with making goofy lists like this one for a living.
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
"Len Budney" [EMAIL PROTECTED] wrote: Warn her, firmly, to send Zip disks by email or to rent space on an FTP server.^ Should be ``mail'', of course. My patented ``matter emailer'' is not yet on the market. :) Len. -- Frugal Tip #14: Sell your used Q-Tips as modern art.
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
On Fri, 28 Apr 2000, Andy Bradford wrote: I may be rehashing old topics, and I may sound a little bit old fashioned (even at age 26), but I don't believe email was ever meant to handle that large amount of traffic. Or, in other words SMTP != FTP I am still of the opinion that one should instruct users to use the right protocols for the right reasons. Hence, put the 10MB PowderPoint file in a public or private ftp directory and then include a URL to fetch it in the email. I agree with this sentiment, but it's becoming increasingly difficult to find good ways to enforce it. Case in point: we do web development for an organization that has a PR firm develop brochures and then send them to us for posting on their website. The files are often 7-10 MB in size, large enough to be cumbersome for e-mail, small enough to make overnighting a ZIP disk seem a little excessive. The organization hosts their site with us, and so we could obviously instruct them to upload the files through FTP, but the PR firm shouldn't necessarily be able to do this. It gets more complicated when you think that it's not always going to be the same person at the PR firm sending the files, and that there are many cases where other third parties need to send us materials related to the site. Clearly it's a complicated issue, but it seems that as broadband access to the net becomes more common, businesses are going to expect to be able to use one "interface" to do all their communications, be it plain text messages or large multi-megabyte file transfers. I cringe every time someone sends me a 7 MB mail message, but it's difficult to explain to them why this is a bad idea. I'd be interested to hear if anyone's found a good general solution to this in a production/business environment. Chris -- Chris Hardie - - mailto:[EMAIL PROTECTED] -- http://www.summersault.com/chris/ --
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
I have customers who regularly send 100+MB attachements. Email is the most convenient way for them to do this. Especially with a local SMTP server in their network. Why waste time tyring to convince them otherwise? Dirk On Fri, Apr 28, 2000 at 03:02:06PM -0500, Chris Hardie wrote: On Fri, 28 Apr 2000, Andy Bradford wrote: I may be rehashing old topics, and I may sound a little bit old fashioned (even at age 26), but I don't believe email was ever meant to handle that large amount of traffic. Or, in other words SMTP != FTP I am still of the opinion that one should instruct users to use the right protocols for the right reasons. Hence, put the 10MB PowderPoint file in a public or private ftp directory and then include a URL to fetch it in the email. I agree with this sentiment, but it's becoming increasingly difficult to find good ways to enforce it. Case in point: we do web development for an organization that has a PR firm develop brochures and then send them to us for posting on their website. The files are often 7-10 MB in size, large enough to be cumbersome for e-mail, small enough to make overnighting a ZIP disk seem a little excessive. The organization hosts their site with us, and so we could obviously instruct them to upload the files through FTP, but the PR firm shouldn't necessarily be able to do this. It gets more complicated when you think that it's not always going to be the same person at the PR firm sending the files, and that there are many cases where other third parties need to send us materials related to the site. Clearly it's a complicated issue, but it seems that as broadband access to the net becomes more common, businesses are going to expect to be able to use one "interface" to do all their communications, be it plain text messages or large multi-megabyte file transfers. I cringe every time someone sends me a 7 MB mail message, but it's difficult to explain to them why this is a bad idea. I'd be interested to hear if anyone's found a good general solution to this in a production/business environment. Chris -- Chris Hardie - - mailto:[EMAIL PROTECTED] -- http://www.summersault.com/chris/ --
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
From: Dirk Harms-Merbitz [EMAIL PROTECTED] Date: Fri, 28 Apr 2000 13:07:26 -0700 I have customers who regularly send 100+MB attachements. Email is the most convenient way for them to do this. Especially with a local SMTP server in their network. Why waste time tyring to convince them otherwise? We have a customer who's internet connection kept getting unusably slow. It turned out that they had several employees who thought that email was a great way to pass pornography around. Because the spooler was on the LAN, it seemed fast to them, so they didn't realize they were clogging the pipe by emailing dirty pictures to a dozen friends (each at different servers, so no redesign of qmail could solve the problem). In this case, the solution was to have their management tell them not to do that, but if it had been legitimate data, the solution would have been to use an FTP server. email may be convenient, but that just means that other paths aren't convenient enough. It would be better to have a common folder on everyone's desktop where they could drop the files. This even works across the internet, if you're a Mac user and have a (free) account on mac.com, you can mount a folder from there on your desktop to drop things on and any other mac.com user can mount it to read your files. That's more convenient than sending email, and much more efficient. Chris -- Chris Garrigues virCIO http://www.DeepEddy.Com/~cwg/ http://www.virCIO.Com +1 512 432 4046 +1 512 374 0500 4314 Avenue C O- Austin, TX 78751-3709 My email address is an experiment in SPAM elimination. For an explanation of what we're doing, see http://www.DeepEddy.Com/tms.html Nobody ever got fired for buying Microsoft, but they could get fired for relying on Microsoft. PGP signature
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
Here's one interesting solution I heard about not too long ago: http://www.whalemail.com/ Another interesting solution would be to teach your MTA to automatically replace MIME attachments with a link to a Web page and a password, and decode and store the attachments on a Web server. Not appropriate for a lot of people, but interesting for a business that can get away with automagically munging people's email. -Scott. Chris Hardie [EMAIL PROTECTED] writes: On Fri, 28 Apr 2000, Andy Bradford wrote: I may be rehashing old topics, and I may sound a little bit old fashioned (even at age 26), but I don't believe email was ever meant to handle that large amount of traffic. Or, in other words SMTP != FTP I am still of the opinion that one should instruct users to use the right protocols for the right reasons. Hence, put the 10MB PowderPoint file in a public or private ftp directory and then include a URL to fetch it in the email. I agree with this sentiment, but it's becoming increasingly difficult to find good ways to enforce it. Case in point: we do web development for an organization that has a PR firm develop brochures and then send them to us for posting on their website. The files are often 7-10 MB in size, large enough to be cumbersome for e-mail, small enough to make overnighting a ZIP disk seem a little excessive. The organization hosts their site with us, and so we could obviously instruct them to upload the files through FTP, but the PR firm shouldn't necessarily be able to do this. It gets more complicated when you think that it's not always going to be the same person at the PR firm sending the files, and that there are many cases where other third parties need to send us materials related to the site. Clearly it's a complicated issue, but it seems that as broadband access to the net becomes more common, businesses are going to expect to be able to use one "interface" to do all their communications, be it plain text messages or large multi-megabyte file transfers. I cringe every time someone sends me a 7 MB mail message, but it's difficult to explain to them why this is a bad idea. I'd be interested to hear if anyone's found a good general solution to this in a production/business environment. Chris -- Chris Hardie - - mailto:[EMAIL PROTECTED] -- http://www.summersault.com/chris/ --
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
On Fri, 28 Apr 2000, Dirk Harms-Merbitz wrote: I have customers who regularly send 100+MB attachements. Email is the most convenient way for them to do this. Especially with a local SMTP server in their network. Why waste time tyring to convince them otherwise? Dirk E-mail protocols and software are not well equipped, by default, to deal with this kind of message size. (They're certainly getting better!) When a message bounces, some mailers send back the whole thing. If it is delayed in a queue, that's a 100 MB message being shuffled around (i.e. copied, transferred over a network, etc) using up system resources. Receiving a 100 MB attachment may be easy for you, but what about someone who happens to be checking their e-mail over a 56K modem for the weekend while their away from the office? Their mail client starts downloading a 100 MB attachment and all sorts of problems could result if the connection is lost or interrupted. FTP is much simpler, and much more binary in its success/failure. In general, you and your net contacts may all have the right infrastructure in place to use e-mail to send large files, but most of the rest of the world probably still doesn't, and when they go on using e-mail for such purposes uneducated about the implications, it can create a huge burden on system administrators. Chris -- Chris Hardie - - mailto:[EMAIL PROTECTED] -- http://www.summersault.com/chris/ --
Re: Multi-RCPT vs. Single RCPT delivery - logic error?
One service that may be of interest for situations like this is Driveway (www.driveway.com). They offer 25mb of free storage space available from the Web (you can buy more), and the space can be shared (you can also put a password on it). In Windows 98/2000, you can even set up a "Web folder" in Explorer, which lets you treat the remote storage space like a Windows share. I'd LOVE to see more people using that rather than email for big files. Some email clients are really lame about handling large attachments (ahem, Outlook, cough), and they get bogged down encoding/decoding the attachment in the foreground. shag = Judd Bourgeois | Email: [EMAIL PROTECTED] Senior Software Developer | Phone: 805.520.7170 CNM Network | Mobile: 805.807.1162 or http://www.cnmnetwork.com | [EMAIL PROTECTED] - Original Message - From: "Chris Hardie" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Fri 28 Apr 2000 13:02 Subject: Re: "Multi-RCPT vs. Single RCPT delivery" - logic error? On Fri, 28 Apr 2000, Andy Bradford wrote: I may be rehashing old topics, and I may sound a little bit old fashioned (even at age 26), but I don't believe email was ever meant to handle that large amount of traffic. Or, in other words SMTP != FTP I am still of the opinion that one should instruct users to use the right protocols for the right reasons. Hence, put the 10MB PowderPoint file in a public or private ftp directory and then include a URL to fetch it in the email. I agree with this sentiment, but it's becoming increasingly difficult to find good ways to enforce it. Case in point: we do web development for an organization that has a PR firm develop brochures and then send them to us for posting on their website. The files are often 7-10 MB in size, large enough to be cumbersome for e-mail, small enough to make overnighting a ZIP disk seem a little excessive. The organization hosts their site with us, and so we could obviously instruct them to upload the files through FTP, but the PR firm shouldn't necessarily be able to do this. It gets more complicated when you think that it's not always going to be the same person at the PR firm sending the files, and that there are many cases where other third parties need to send us materials related to the site. Clearly it's a complicated issue, but it seems that as broadband access to the net becomes more common, businesses are going to expect to be able to use one "interface" to do all their communications, be it plain text messages or large multi-megabyte file transfers. I cringe every time someone sends me a 7 MB mail message, but it's difficult to explain to them why this is a bad idea. I'd be interested to hear if anyone's found a good general solution to this in a production/business environment. Chris -- Chris Hardie - - mailto:[EMAIL PROTECTED] -- http://www.summersault.com/chris/ --
