Dear Wiki user, You have subscribed to a wiki page or wiki category on "James Wiki" for change notification.
The following page has been changed by SergeyLubinskiy: http://wiki.apache.org/james/SummerOfCode2005/SergeyLubinskiyFastFail ------------------------------------------------------------------------------ Hello! - I'm Sergey Lubinskiy. I study at http://www.msu.ru/en/ deparmant of physics. + I'm Sergey Lubinskiy. I study at http://www.msu.ru/en/ department of physics. - I'm keen of programming although I don't specialize in it and posess certain + I'm keen of programming although I don't specialize in it and posses certain load of knowledge on Java, internet standards and protocols, general rules of open source software development, design patterns. I tend to think and code a bit too academically but I hope this will only - help me to participate in the summer of code programm. Here's my proposal. + help me to participate in the summer of code program. Here's my proposal. === Problem === James mail server performs reasonably well under normal load but heavy spam traffic and dos attacks significantly degrade performance. Under the current design the main process of decision making about each particular - mail message happens after the message has been fully recieved and filed + mail message happens after the message has been fully received and filed to disk or database. Combating spam requires us to adopt an alternative - strategy - we may wish to just cut tcp connections short based on ip and + strategy - we may wish to just cut TCP connections short based on IP and - smpt commands given before we even start to accept the message. + SMTP commands given before we even start to accept the message. - Another issue is that under the current desing very little control + Another issue is that under the current design very little control is possible over a running instance of James mail server - the main - configuration method is to edit config manually and then restart + configuration method is to edit configuration manually and then restart (which takes significant time). === Proposed solutions === @@ -33, +33 @@ In spirit the new block will follow the design of [http://james.apache.org/javadocs/org/apache/james/transport/JamesSpoolManager.html JamesSpoolManager]. While the later hosts a family of matcher - mailet pairs forming a "tree of responsibility" the - new smpt block will host a sequence of "protocolLets" - actors for line by line processing + new SMTP block will host a sequence of "protocolLets" - actors for line by line processing - of smpt protocol messages. + of SMTP protocol messages. Technically however it is going to be a bit different. - Firstly I intend to allow re-configuration, reordering, addition and removal of "protocolLets" via JMX. To facilate this I'm going to find a way to map: + Firstly I intend to allow re-configuration, reordering, addition and removal of "protocolLets" via JMX. To facilitate this I'm going to find a way to map: * "protocolLets" bean-style configuration interface * JMX allowed CompositeData and TabularData - * xml representation thereof (xstream-style) + * XML representation thereof (xstream-style) * possibly Phoenix configuration objects onto each other. - As configuration of "protocolLets" and their order is changed via JMX this immediately affects the running block - in fact I imagine that we would take down "protocolLets" configured with old values and instantiate "protocolLets" with new configuration to avoid concurrency issues. Same applies to the protocolLet[] array representing their sequence. Each protocolLet is going to be stateless and thus tread-safe. Perhaps we can design JMX configuration interface so that all the reordering and reconfiguration will take effect in a batch as partially configured "protocolLets" may cause server failures. + As configuration of "protocolLets" and their order is changed via JMX this immediately affects the running block - in fact I imagine that we would take down "protocolLets" configured with old values and instantiated "protocolLets" with new configuration to avoid concurrency issues. Same applies to the protocolLet[] array representing their sequence. Each protocolLet is going to be stateless and thus tread-safe. Perhaps we can design JMX configuration interface so that all the reordering and reconfiguration will take effect in a batch as partially configured "protocolLets" may cause server failures. - A key point here is to let the "protocolLets" see other blocks visible to the new smtp block iteself. This will simplify implementing such features as cammunication outer processes (perl-written policies taken from postfix) or talking TCP to outerworld hosts (rbl) - none of these actions actually belongs to "protocolLets" - little stateless decision makers. + A key point here is to let the "protocolLets" see other blocks visible to the new SMTP block itself. This will simplify implementing such features as communicating outer processes (perl-written policies taken from postfix) or talking TCP to outer world hosts (RBL) - none of these actions actually belongs to "protocolLets" - little stateless decision makers. Then I'm going to add a JMX method for saving this configuration on disk. (Does Tomcat do that already?). - One interesting possibility would be to allow new smtp block to save directly to svn - thus making + One interesting possibility would be to allow new SMTP block to save directly to SVN - thus making provision for possible failures. If timeframe permits we may consider doing beanshell integration - there's nothing impossible in "protocolLets" being written in beanshell, or probably jython, javascript. In this case the code of the "protocolLet" becomes its configuration. - I beleive that this "go from getter/setters via JMX CompositeData to XML will let some fresh wind into the old dynamic server reconfiguration problem. + I believe that this "go from getter/setters via JMX CompositeData to XML will let some fresh wind into the old dynamic server reconfiguration problem. - Technical implementation of these features still needs to be explored. In essence "protocolLets" are quite similar to blocks but Phoenix machinery most likely can not be reused as it is. I see the following opitions to investigate and choose from: + Technical implementation of these features still needs to be explored. In essence "protocolLets" are quite similar to blocks but Phoenix machinery most likely can not be reused as it is. I see the following options to investigate and choose from: * go for some quirks and tricks and make use of existing Phoenix internals as they are * create a patched Phoenix (and push for new version being rolled out with these patches) - * run.. say Fortress inside new smpt block (I'm trying to pick a container living iside asf) + * run.. say Fortress inside new SMTP block (I'm trying to pick a container living inside ASF) * DIY container plumbing pico-container style (shouldn't be too much code) closely along the lines of the current [http://james.apache.org/javadocs/org/apache/james/transport/JamesSpoolManager.html JamesSpoolManager] implementation === Advantages for ASF === - * A solid foundation for implementing a feature heavily demanded by current internet realities - smtp fast fail + * A solid foundation for implementing a feature heavily demanded by current internet realities - SMTP fast fail * Possible a new insight on the issue of reconfiguring components inside IoC containers === Deliverables === - Totally revamped drop-in replacment for the existing [http://james.apache.org/javadocs/org/apache/james/smtpserver/SMTPServer.html SMTPServer]. Possibly patches or pieces of coded targeted for Avalon containers. + Totally revamped drop-in replacement for the existing [http://james.apache.org/javadocs/org/apache/james/smtpserver/SMTPServer.html SMTPServer]. Possibly patches or pieces of coded targeted for ASF containers. === Schedule === * start active coding around June the 22nd, the expected end of examination session - * deliver version 0.1 by July the 12th, this version should prove that configuraion and more importantly reconfiguraion of "protocolLets" along the path: "inspect class via reflection - build JMX type descriptor - configure via JMX - save to xml" is feasable and desirable + * deliver version 0.1 by July the 12th, this version should prove that configuration and more importantly reconfiguration of "protocolLets" along the path: "inspect class via reflection - build JMX type descriptor - configure via JMX - save to XML" is feasible and desirable - * deliver version 0.2 by July 25 - "protocolLets" actually able to see other blocks visible to smtp server (they get them via setter methods), the new smtp block actually able to accept main and feed it into the existing processor pipeline + * deliver version 0.2 by July 25 - "protocolLets" actually able to see other blocks visible to SMTP server (they get them via setter methods), the new SMTP block actually able to accept main and feed it into the existing processor pipeline - * deliver version 0.5 by August 14 - some protocolLets actually make sense and do some useful spam blocking + * deliver version 0.5 by August 14 - some protocolLets actually make sense and do some useful Spam blocking After August 14 fix react to bug reports and requests for enhancements. If time permits look into integration with scripting languages. @@ -106, +106 @@ } }}} - Which is shamelessly compiled out of FailFast and [http://mail-archives.apache.org/mod_mbox/james-server-dev/200506.mbox/[EMAIL PROTECTED] "07 Jun 2005 message on server-dev] + Which has been shamelessly compiled out of FailFast and [http://mail-archives.apache.org/mod_mbox/james-server-dev/200506.mbox/[EMAIL PROTECTED] "07 Jun 2005 message on server-dev”] - I hope this is going to be fun! :-) -
