Jason R. Mastaler wrote: > Jesse Guardiani <[EMAIL PROTECTED]> writes: > >> I'm pretty concerned about it, actually. TMDA would be a lot quicker >> and more reliable, IMO, if it didn't try to read the body of a 10M >> message into memory. > > How would it be more reliable?
It wouldn't cause my server to run out of memory without warning under high load. The potential exists for that sort of thing with the current implementation. Then I'd be swapping to disk and the problem would compound itself. > How would it be quicker? Let's say you read an entire 10M message from STDIN to memory. You just copied 10M to memory. That takes a certain amount of CPU time. Then, when you're done with the message, you copy that memory to disk. Again, more CPU time. If you instead only read the headers from STDIN and copied them to memory, then did a straight copy from STDIN to disk for the rest of the message, you'd save CPU time. If you're running a single TMDA process, you probably wouldn't even notice the difference. But, if you're running 10 or 20 simultaneous TMDA processes, you probably WILL notice. Quickly. Besides, what if the file was 100M, or 200M? The larger the file, the more CPU (and perhaps more importantly - RAM) wasted. Sure, it's silly to send a file that large via email, but our privilaged customers do it all the time. They think it's more convenient than FTP. > TMDA needs > the contents of the message body at some point anyway. > > True, it would use less memory if it spooled the message body from a > temp file or something instead. However, since this is a more complex > procedure, I think it would be more error prone, and less reliable. Yeah, I'm not sure about that yet. I may need to write a new class, like PureHeaderParser or something. The standard Parser methods may not be capable of being overloaded to do this sort of thing reliably. I think it just depends on how the interface is designed... > > Memory is cheap, which is probably why this hasn't been a practical > concern in the past. I'm a big fan of doing it right the first time. Memory may be cheap, but in a server environment sloppy housekeeping shows up much more quickly. It's just unneccessary RAM usage, Jason. Plain and simple. I'm fairly confident that the same goals of reliability and speed can be accomplished without the extra RAM usage, and if that's the case, why shouldn't I pursue it? > > As I've also said in the past though, if someone wants to propose a well > designed solution, and then write some well-engineered code that > implements that solution, I'll certainly consider it. Well, no point in talking about it any more. I need to get coding. -- Jesse Guardiani, Systems Administrator WingNET Internet Services, P.O. Box 2605 // Cleveland, TN 37320-2605 423-559-LINK (v) 423-559-5145 (f) http://www.wingnet.net _________________________________________________ tmda-workers mailing list ([EMAIL PROTECTED]) http://tmda.net/lists/listinfo/tmda-workers
