Chris Berry wrote: >>From: Jesse Guardiani <[EMAIL PROTECTED]> >> > Sounds like over-optimizing to me >> >>Never heard of it. > > Hehe, well that has a bit of truth to it, but there is a point of > diminishing returns.
Absolutely. I just don't think this is it. (rewriting TMDA in ASM would take the cake IMHO.) > >> >, it's works fine now. What problem are >> > you trying to solve? >> >>Specifically, my qmail-smtpd processes run under a memory limiting program >>called 'softlimit'. If the email module loads incoming emails into RAM, >>then >>I have to set up softlimit to limit the size of my qmail-smtpd pipeline's >>RAM usage to the normal program usage + the size of the largest email I >>want to receive. > > That's good, I use that myself, pretty common in most qmail setups. Hmmm... I think you might have some misconceptions about the purpose of the softlimits program. I don't think it's intended to limit message size. I think it's intended to contain run-away processes. Take a look at the information I have written below to see what I mean: > >>Typically, my qmail-smtpd process uses 2.2M of RAM, and my python >>filtering script, with all modules loaded, uses 4.4M of RAM. So, the >>smallest I can set my softlimit is 6.6M, which would normally allow PLENTY >>(90) SMTP processes >>under my designated 600M server RAM limit. >> >>However, now I have to add my max message size to the mix, so if I want a >>max of 30 SMTP processes running simultaneously, I must set my softlimit >>to 20M, which gives me 12.4M per message. > > I'm not sure I follow your math there unless your max message size just > happens to be 5.8M You're right, I was off by one, but I'm not sure where you get 5.8M from: softlimit = 20M ---------------------- qmail-smtpd = 2.2M myPythonScript = 4.4M ---------------------- Remaining RAM for emails = 13.4M 600M of server RAM / 20M of RAM usage per process = 30 SMTP processes, MAX > >>If the email module didn't gobble the entire email up into RAM, then I >>wouldn't >>have to set my max message size with softlimit, and I could safely run >>with a higher maximum qmail-smtpd process count. > > True, but then your max message thruput is going to drop through the floor > as you messages are now stored on the hard drive which is accessed in > milliseconds instead of nanoseconds. I could be wrong, but I don't think qmail-smtpd stores incoming SMTP messages in RAM at all. I don't think softlimit was originally intended to limit the incoming SMTP mail message size in a normal qmail setup, either. Take a look at `man qmail-smtpd` and read the section about the 'databytes' file. I'm pretty sure the incoming emails are written directly to disk, or at least the next program in the pipeline, which is usually qmail-queue, which then writes the email to disk. qmail defaults to allowing email messages of unlimited size to be sent, unless the databytes control file says differently. > >>Also, on the TMDA front, if TMDA didn't read the entire email into RAM, >>then >>it would be slightly faster and less likely to eat up my RAM with >>concurrent >>deliveries. > > That seems like it would end up being slower to me, though less resourse > hungry. Not slower, but faster. We'd be eliminating a step. We currently do this: STDIN (which is either RAM or a temp file on disk) -> RAM -> DISK but I propose we do this: STDIN -> DISK (with the exception of the headers, of course.) > >>It all makes perfect sense to me. I'm not sure why everyone is so opposed >>to the idea. > > Because buying more ram is cheap, but trying to get a faster hard drive is > expensive. This argument might work on other mail servers (maybe), but not on a qmail server. Qmail servers are designed to be limited by the thruput of the disk. They don't use RAM or CPU unless they have to. And frankly, this is an excellent way to deal with email, especially when you consider that my T1's bandwidth will likely fill up before my disk will max out. > Regardless of all that though, it seems like the problem is > big > messages, not how TMDA handles them. I respectfully disagree. > Still, I'm sure if you manage to > write something solid that avoids the problems people have mentioned Jason > would probably take a good look at it. That's what I'm counting on. In my experience, programmers rarely refuse free code. -- Jesse Guardiani, Systems Administrator WingNET Internet Services, P.O. Box 2605 // Cleveland, TN 37320-2605 423-559-LINK (v) 423-559-5145 (f) http://www.wingnet.net _________________________________________________ tmda-workers mailing list ([EMAIL PROTECTED]) http://tmda.net/lists/listinfo/tmda-workers
