RE: SMTPHandler connection timeout
So let's review the options: - use java.net.Socket.setSoTimeout(timeout) - fix Avalon so that it removes redundant triggers when they are updated - fix Avalon so that it calls notify every time a trigger is reset - can we extend the Avalon scheduler/timer classes with a new timer that does what we need (sorry I don't have the code handy to check how feasibly this is myself)? - implement a new timer mechanism purely within James Noel, you stated that you didn't like the setSoTimeout approach, can you clarify why James need to protect parseCommand as well as socket IO? Cheers Steve -Original Message- From: Serge Knystautas [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 22, 2002 6:30 AM To: James Developers List Cc: Shilpa Dalmia Subject: Re: SMTPHandler connection timeout Noel, Looks rather serious... like you said, perhaps building our own trigger mechanism is the way to go. More importantly, I just wanted to note that SchedulerNotifyInputStream will reset the trigger every 20,000 bytes, not every byte. So while bad, it's not the end of JVM's as we know it. :) Serge Knystautas Loki Technologies - Unstoppable Websites http://www.lokitech.com/ - Original Message - From: Noel J. Bergman [EMAIL PROTECTED] To: James-Dev Mailing List [EMAIL PROTECTED] Cc: Shilpa Dalmia [EMAIL PROTECTED] Sent: Tuesday, May 21, 2002 11:54 PM Subject: RE: SMTPHandler connection timeout Shilpa, You've got me convinced. Having read the thread, more James code and the relevant portions of Cornerstone, I believe that you are actually underestimating. If what appears to be the case from my reading of the code, and what you and Peter Donald say is happening, then you are underestimating the potential magnitude of the problem by orders. Trace through the doDATA method. It appears to be that SchedulerNotifyInputStream resets the trigger for EVERY BYTE OF DATA in the message stream; SchedulerNotifyOutputStream does the same when writing. If those are really resulting in a new queue entry for each byte of data (which is what appears to be the case), that's brutal, and really needs to be addressed either in Avalon or in James. As far as I can see, James is using this as a watchdog timer, which does not appear to be the design point for Avalon's facility, based upon their own comments, design and implementation. And even the workaround to Avalon of adding another thread or doing the replacement during the reset would be performance prohibitive given that the trigger is reset for each byte of data read from the stream. I've done a grep of James to find all uses of the trigger code, and it looks to be similar in all cases. Perhaps a more efficient watchdog solution can be devised, maybe tied to the handler instance. I'd sooner create a watchdog thread per handler than have 100s of thousands (or millions) of events in the scheduler queue. Am I reading this correctly? Your throughts? --- Noel -Original Message- From: Shilpa Dalmia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 21, 2002 19:26 To: '[EMAIL PROTECTED]' Subject: FW: SMTPHandler connection timeout You can look at the foll. avalon mailing list thread on this issue where they have given an explanation for this. http://www.mail-archive.com/avalon-dev@jakarta.apache.org/msg08660.html I have a implemented a temporary solution for this issue, which fixes the problem. Shilpa -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: SMTPHandler connection timeout
SchedulerNotifyInputStream will reset the trigger every 20,000 bytes Sorry ... that is really embarrassing. :-( I skimmed the code way too quickly. Even so, that's 52 queue entries per megabyte of message content plus the aforementioned entry per handler command. --- Noel -Original Message- From: Serge Knystautas [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 22, 2002 9:30 To: James Developers List Cc: Shilpa Dalmia Subject: Re: SMTPHandler connection timeout Noel, Looks rather serious... like you said, perhaps building our own trigger mechanism is the way to go. More importantly, I just wanted to note that SchedulerNotifyInputStream will reset the trigger every 20,000 bytes, not every byte. So while bad, it's not the end of JVM's as we know it. :) -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: SMTPHandler connection timeout
Steve, I didn't say that I don't like the setSoTimeout approach. What I said was that from the look of the code, the author(s) appear to be using the timer as a watchdog to protect not only against an I/O hang, but also a hang elsewhere in the processing of the handler. If that is not necessary, then go with setSoTimeout. Perhaps Serge and/or other authors of the handler can provide some insight into why they did it the way they did it. --- Noel -Original Message- From: Steve Short [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 22, 2002 11:50 To: 'James Developers List' Cc: Shilpa Dalmia Subject: RE: SMTPHandler connection timeout So let's review the options: - use java.net.Socket.setSoTimeout(timeout) - fix Avalon so that it removes redundant triggers when they are updated - fix Avalon so that it calls notify every time a trigger is reset - can we extend the Avalon scheduler/timer classes with a new timer that does what we need (sorry I don't have the code handy to check how feasibly this is myself)? - implement a new timer mechanism purely within James Noel, you stated that you didn't like the setSoTimeout approach, can you clarify why James need to protect parseCommand as well as socket IO? Cheers Steve -Original Message- From: Serge Knystautas [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 22, 2002 6:30 AM To: James Developers List Cc: Shilpa Dalmia Subject: Re: SMTPHandler connection timeout Noel, Looks rather serious... like you said, perhaps building our own trigger mechanism is the way to go. More importantly, I just wanted to note that SchedulerNotifyInputStream will reset the trigger every 20,000 bytes, not every byte. So while bad, it's not the end of JVM's as we know it. :) Serge Knystautas Loki Technologies - Unstoppable Websites http://www.lokitech.com/ - Original Message - From: Noel J. Bergman [EMAIL PROTECTED] To: James-Dev Mailing List [EMAIL PROTECTED] Cc: Shilpa Dalmia [EMAIL PROTECTED] Sent: Tuesday, May 21, 2002 11:54 PM Subject: RE: SMTPHandler connection timeout Shilpa, You've got me convinced. Having read the thread, more James code and the relevant portions of Cornerstone, I believe that you are actually underestimating. If what appears to be the case from my reading of the code, and what you and Peter Donald say is happening, then you are underestimating the potential magnitude of the problem by orders. Trace through the doDATA method. It appears to be that SchedulerNotifyInputStream resets the trigger for EVERY BYTE OF DATA in the message stream; SchedulerNotifyOutputStream does the same when writing. If those are really resulting in a new queue entry for each byte of data (which is what appears to be the case), that's brutal, and really needs to be addressed either in Avalon or in James. As far as I can see, James is using this as a watchdog timer, which does not appear to be the design point for Avalon's facility, based upon their own comments, design and implementation. And even the workaround to Avalon of adding another thread or doing the replacement during the reset would be performance prohibitive given that the trigger is reset for each byte of data read from the stream. I've done a grep of James to find all uses of the trigger code, and it looks to be similar in all cases. Perhaps a more efficient watchdog solution can be devised, maybe tied to the handler instance. I'd sooner create a watchdog thread per handler than have 100s of thousands (or millions) of events in the scheduler queue. Am I reading this correctly? Your throughts? --- Noel -Original Message- From: Shilpa Dalmia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 21, 2002 19:26 To: '[EMAIL PROTECTED]' Subject: FW: SMTPHandler connection timeout You can look at the foll. avalon mailing list thread on this issue where they have given an explanation for this. http://www.mail-archive.com/avalon-dev@jakarta.apache.org/msg08660.html I have a implemented a temporary solution for this issue, which fixes the problem. Shilpa -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: SMTPHandler connection timeout
Hi, The reason I raised this question was that we've had some memory leak issues in this section of the code. I posted messages on the avalon mailing list they acknowledged it as a problem with this kind of usage. Here's a brief description of the problem - The addTrigger call creates a job entry in the queue. The resetTrigger call reschedules the job, invalidating the old entry adding a new one. There's a monitor thread which wakes up when the first entry in the queue is ready to be scheduled at that time removes the invalidated entries. So if the connection timeout is say 5 mins. and there is constant SMTP activity, I estimate that this could lead to 15,000 or more invalid timer entries in the queue. (10 message per second * 5 SMTP commands per message * 300 seconds), potentially raising an out of memory exception. A fix has been suggested on the avalon mailing list for this, but I wanted see if we could use an alternate timeout mechanism in james. But if this mechanism is required, we should fix the avalon problem. Shilpa -Original Message- From: Noel J. Bergman [mailto:[EMAIL PROTECTED]] Sent: Monday, May 20, 2002 6:40 PM To: James Developers List Subject: RE: SMTPHandler connection timeout In SMTPHandler::handleConnection() method, why do we use the avalon scheduler trigger mechanism to handle a connection timeout? Instead why don't we use the java.net.Socket.setSoTimeout(timeout) method, Please consider the code: final PeriodicTimeTrigger trigger = new PeriodicTimeTrigger( timeout, -1 ); scheduler.addTrigger( this.toString(), trigger, this ); while (parseCommand(in.readLine())) scheduler.resetTrigger(this.toString()); socket.close(); scheduler.removeTrigger(this.toString()); Note that the trigger protects more than just socket I/O. It covers the processing of the command (parseCommand does a lot more than just parse). --- Noel -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: SMTPHandler connection timeout
Oh my. Not a nice defect in Avalon at all. Why don't they use an object pool for that class of object? And if they can invalidate when replacing, they ought to be able to remove invalid job entries, and return them to the pool. You should probably reply to me off-list with that response, rather than bother everyone else (of course, others are free to say that this is not off-topic). --- Noel -Original Message- From: Shilpa Dalmia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 21, 2002 13:31 To: 'James Developers List' Subject: RE: SMTPHandler connection timeout Hi, The reason I raised this question was that we've had some memory leak issues in this section of the code. I posted messages on the avalon mailing list they acknowledged it as a problem with this kind of usage. Here's a brief description of the problem - The addTrigger call creates a job entry in the queue. The resetTrigger call reschedules the job, invalidating the old entry adding a new one. There's a monitor thread which wakes up when the first entry in the queue is ready to be scheduled at that time removes the invalidated entries. So if the connection timeout is say 5 mins. and there is constant SMTP activity, I estimate that this could lead to 15,000 or more invalid timer entries in the queue. (10 message per second * 5 SMTP commands per message * 300 seconds), potentially raising an out of memory exception. A fix has been suggested on the avalon mailing list for this, but I wanted see if we could use an alternate timeout mechanism in james. But if this mechanism is required, we should fix the avalon problem. Shilpa -Original Message- From: Noel J. Bergman [mailto:[EMAIL PROTECTED]] Sent: Monday, May 20, 2002 6:40 PM To: James Developers List Subject: RE: SMTPHandler connection timeout In SMTPHandler::handleConnection() method, why do we use the avalon scheduler trigger mechanism to handle a connection timeout? Instead why don't we use the java.net.Socket.setSoTimeout(timeout) method, Please consider the code: final PeriodicTimeTrigger trigger = new PeriodicTimeTrigger( timeout, -1 ); scheduler.addTrigger( this.toString(), trigger, this ); while (parseCommand(in.readLine())) scheduler.resetTrigger(this.toString()); socket.close(); scheduler.removeTrigger(this.toString()); Note that the trigger protects more than just socket I/O. It covers the processing of the command (parseCommand does a lot more than just parse). --- Noel -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: SMTPHandler connection timeout
Shilpa, You've got me convinced. Having read the thread, more James code and the relevant portions of Cornerstone, I believe that you are actually underestimating. If what appears to be the case from my reading of the code, and what you and Peter Donald say is happening, then you are underestimating the potential magnitude of the problem by orders. Trace through the doDATA method. It appears to be that SchedulerNotifyInputStream resets the trigger for EVERY BYTE OF DATA in the message stream; SchedulerNotifyOutputStream does the same when writing. If those are really resulting in a new queue entry for each byte of data (which is what appears to be the case), that's brutal, and really needs to be addressed either in Avalon or in James. As far as I can see, James is using this as a watchdog timer, which does not appear to be the design point for Avalon's facility, based upon their own comments, design and implementation. And even the workaround to Avalon of adding another thread or doing the replacement during the reset would be performance prohibitive given that the trigger is reset for each byte of data read from the stream. I've done a grep of James to find all uses of the trigger code, and it looks to be similar in all cases. Perhaps a more efficient watchdog solution can be devised, maybe tied to the handler instance. I'd sooner create a watchdog thread per handler than have 100s of thousands (or millions) of events in the scheduler queue. Am I reading this correctly? Your throughts? --- Noel -Original Message- From: Shilpa Dalmia [mailto:[EMAIL PROTECTED]] Sent: Tuesday, May 21, 2002 19:26 To: '[EMAIL PROTECTED]' Subject: FW: SMTPHandler connection timeout You can look at the foll. avalon mailing list thread on this issue where they have given an explanation for this. http://www.mail-archive.com/avalon-dev@jakarta.apache.org/msg08660.html I have a implemented a temporary solution for this issue, which fixes the problem. Shilpa -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
RE: SMTPHandler connection timeout
In SMTPHandler::handleConnection() method, why do we use the avalon scheduler trigger mechanism to handle a connection timeout? Instead why don't we use the java.net.Socket.setSoTimeout(timeout) method, Please consider the code: final PeriodicTimeTrigger trigger = new PeriodicTimeTrigger( timeout, -1 ); scheduler.addTrigger( this.toString(), trigger, this ); while (parseCommand(in.readLine())) scheduler.resetTrigger(this.toString()); socket.close(); scheduler.removeTrigger(this.toString()); Note that the trigger protects more than just socket I/O. It covers the processing of the command (parseCommand does a lot more than just parse). --- Noel -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
