Together with a couple of colleagues I'm investigating whether to replace Windows SMTP server with James as the transport mechanism for one of our products. One of the parts of the investigation involves load testing James to see how remote delivery performs under load and compare it with Windows SMTP server.
Here are some info about the config and the tests; I have a very detailed doc if you're interested: - PC is a Dell PE4600 dual 1790Mhz CPU 2GB RAM running Windows 2000 and James 2.1.3 running as a service. - There are three servers, server1 runs Windows SMTP server to send emails, server2 acts as a relay and runs James in one set of tests and Windows SMTP server in the other, and server3 is the final destination of the messages and it runs Windows SMTP server . - One set of tests sent 1.04KB plain text emails, the other 649KB G711 voice; these two sets of tests were performed first using James and then using Windows SMTP server. - We sent 100 messages per SMTP session. - We measured max msgs per minute, CPU usage at different stages, number of threads, memory usage and various leaks. - As for James, we set the spoolmanager threads to 12, as per per <http://nagoya.apache.org/wiki/apachewiki.cgi?JamesByTheNumbers> ; delayTime for remote delivery set to 300000; deliveryThreads set to 20. We're using file as the spool type. The results were quite disappointing, and we saw several errors. Since we quite like the idea of using open source code and we like the flexibility, portability and extra features of the James code, I'm hoping that somebody in this list can give us advice about these problems, ex. have you seen these errors? Is there anything in the configuration that you know how to optimize? The problem is that if the performance is too poor, and we see these types of errors, we're not going to be able to sell this to management. First the errors. We enabled DEBUG logging; note that for each of the following errors, the frequency of errors increased with traffic. Here are some examples: - In mailet log: Could not connect to SMTP host: jdccisco-000003.ecsbu-lab-sea.cisco.com, port: 25; nested exception is: java.net.BindException: Address already in use: connect at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:911) at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:158) at javax.mail.Service.connect(Service.java:233) at javax.mail.Service.connect(Service.java:134) at javax.mail.Service.connect(Service.java:86) at com.sun.mail.smtp.SMTPTransport.connect(SMTPTransport.java:95) at org.apache.james.transport.mailets.RemoteDelivery.deliver(RemoteDelivery.java:305) at org.apache.james.transport.mailets.RemoteDelivery.run(RemoteDelivery.java:758) at java.lang.Thread.run(Unknown Source) In mailstore logs: 25/11/03 15:26:57 ERROR mailstore: Exception retrieving mail: java.lang.RuntimeException: Exception caught while retrieving an object, cause: java.io.FileNotFoundException: C:\JAVA\JAMES\james-2.1.3\apps\james\var\mail\outgoing\4D61696C313036393830323831373132362D383437372D746F2D6A6463636973636F2D3030303030332E65637362752D6C61622D7365612E636973636F2E636F6D.Repository.FileObjectStore (The system cannot find the file specified), so we're deleting it... good riddance! ...and... 25/11/03 15:27:24 ERROR mailstore: Exception retrieving mail: java.lang.RuntimeException: Exception caught while retrieving an object, cause: java.io.FileNotFoundException: C:\JAVA\JAMES\james-2.1.3\apps\james\var\mail\spool\4D61696C313036393830323834343137312D39323539.Repository.FileObjectStore (The system cannot find the file specified), so we're deleting it... good riddance! - In smptserver logs: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.read1(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(Unknown Source) at sun.nio.cs.StreamDecoder$CharsetSD.implRead(Unknown Source) at sun.nio.cs.StreamDecoder.read(Unknown Source) at java.io.InputStreamReader.read(Unknown Source) at java.io.BufferedReader.fill(Unknown Source) at java.io.BufferedReader.readLine(Unknown Source) at java.io.BufferedReader.readLine(Unknown Source) at org.apache.james.smtpserver.SMTPHandler.readCommandLine(SMTPHandler.java:527) at org.apache.james.smtpserver.SMTPHandler.handleConnection(SMTPHandler.java:381) at org.apache.james.util.connection.ServerConnection$ClientConnectionRunner.run(ServerConnection.java:462) at org.apache.avalon.excalibur.thread.impl.ExecutableRunnable.execute(ExecutableRunnable.java:47) at org.apache.avalon.excalibur.thread.impl.WorkerThread.run(WorkerThread.java:80) ...and... 25/11/03 18:58:00 ERROR smtpserver: Unknown error occurred while processing DATA. javax.mail.MessagingException: Unable to retrieve the data: Connection reset; nested exception is: java.net.SocketException: Connection reset at org.apache.james.core.MimeMessageInputStreamSource.<init>(MimeMessageInputStreamSource.java:125) at org.apache.james.core.MailImpl.<init>(MailImpl.java:173) at org.apache.james.smtpserver.SMTPHandler.processMail(SMTPHandler.java:1394) at org.apache.james.smtpserver.SMTPHandler.doDATA(SMTPHandler.java:1253) at org.apache.james.smtpserver.SMTPHandler.parseCommand(SMTPHandler.java:608) at org.apache.james.smtpserver.SMTPHandler.handleConnection(SMTPHandler.java:381) at org.apache.james.util.connection.ServerConnection$ClientConnectionRunner.run(ServerConnection.java:462) at org.apache.avalon.excalibur.thread.impl.ExecutableRunnable.execute(ExecutableRunnable.java:47) at org.apache.avalon.excalibur.thread.impl.WorkerThread.run(WorkerThread.java:80) As for performance, it looks like James, in this test, performs way worse than Windows SMTP server. Some examples: - when sending 1.04KB voice messages, James maxes out at 1860 per minute, while Windows SMTP at 1920; however, at these number James is using 70% of CPU, and Windows SMTP 8%. - when sending 649KB voice messages, James maxes out at 222 per minute, while Windows SMTP at 420; at these number James is using 96% of CPU, and Windows SMTP 15%. - If McAfee online access scan is enabled, James throughput declines, while MS SMTP server does not. Again, please let me know if you have any helpful information, or if you're intested in seeing more info about our tests. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
