Hi all, Jason says that MSSQL has locking problems under load. This does not happen with MySQL.
I have personally stress-tested James using MySQL repositories and results are good. I have two remarks though: 1. As you overload James and the spool size grows, the performance decreases. This happens because the "listMessagesSQL" request takes longer and MySQL returns huge ResultSets to James. The size effect of growing spools can be limited by adding something like "LIMIT 100" to the "listMessagesSQL" request at the sqlResources.xml file level. 2. After thousands of messages, the MySQL tables get "tired". It is highly recommended to periodically "OPTIMIZE TABLE spool" in order to keep your performance linear through time. This can be automated via a cron job. This is the result of my experience with James+MySQL. I am looking forward similar recommendations or feedback on James+PostgreSQL. Also, some people seem to consider that file-based repositories are less reliable than db ones. File-based repositories are also unable to ensure FIFO delivery, which in my opinion is a major drawback of such persistency model. As I have personally observed +50% performance gains on some systems by simply moving from MySQL-based to file-based repositories, I would like someone to comment on this one. Are there any good reasons (apart from the FIFO thing) why I should keep using db-based repositories provided that file-based ones perform much better? Which are the real risks? Diego -----Message d'origine----- De : Hut Carspecken [mailto:[EMAIL PROTECTED] Envoye : mercredi 17 septembre 2003 16:54 A : 'James Users List' Objet : RE: Scaling JAMES onto More Than One Server Jason, Thank you for the extra information! You use file-based repositories because MySQL has locking issues? I should take another look at MySQL. You mentioned that you offload the DNS server work to Qmail, did you disable the DNS server in JAMES? Well, you have been very helpful and if I am not mistaken the DL580 holds all four mail servers (Qmail and JAMES). That gives me a good idea what kind of hardware we need. Many Thanks Jason! Hut -----Original Message----- From: Jason Webb [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 17, 2003 3:58 AM To: 'James Users List'; [EMAIL PROTECTED] Subject: RE: Scaling JAMES onto More Than One Server During our tests with the 2 Qmail, 2 James server system we used old boxes no faster than 450Mhz PII with 256Mb RAM in each. The database was MySQL running on a SUN Ultra 10, but we used file based repositories on each of the James servers. The DL580 was a customers server and they had everything bar the database on it. We don't normally use database-backed spool or mail repositories. Most of our customers use MSSQL which has some serious locking problems when running under load. You have the sequence correct. Qmail gets the mail, our mailets process the mail and give it back to Qmail for transmission. This gives use both speed and reliability without resorting to expensive clustering solutions. -- Jason > -----Original Message----- > From: Hut Carspecken [mailto:[EMAIL PROTECTED] > Sent: 16 September 2003 23:06 > To: 'James Users List' > Subject: RE: Scaling JAMES onto More Than One Server > > > This is very interesting, Jason. So you use a total of four > physical servers. The Qmail server receive and send the mail > while your James servers do the processing involved with the > mailets. Once the James servers receive mail from your Qmail > servers,they process the messages according to your mailets > and then give them back to your Qmail servers to send. Do I > have this correct? Also, are all of your servers (both Qmail > and JAMES)the HP Proliant DL580 model? Do you use a separate > server to host the JAMES databases or are they located on the > same servers? > > Many thanks! > > Hut > > -----Original Message----- > From: Jason Webb [mailto:[EMAIL PROTECTED] > Sent: Tuesday, September 16, 2003 3:22 PM > To: 'James Users List'; [EMAIL PROTECTED] > Subject: RE: Scaling JAMES onto More Than One Server > > > Hi Hut. > We use a slightly different method to scale James onto > multiple servers. Two QMail servers work as a pair (we've > used both DNS load balancing using MX records or a load > balancer) and then having the QMail boxes deliver to a set of > James servers. The James servers then use the QMail boxes as > gateways as QMail is better at delivering mail to crappy > servers than James at the moment (don't get me started on > AOL). This also offloads the DNS processing from the James > servers. As far as figures go we never really thrashed this > system so I don't have any hard evidence on how good an > approach this is. > > As a piece of anecdotal evidence we had a HP Proliant DL580 > (4 x 2.4 GHz Xenons with 2 stripped disks, mirrored). Due a > mistake by yours truly I managed to cause a loop whereby our > MLM software delivered just over 500k messages to itself in > an hour. Believe it or not, our customer was very impressed! > > Our standard leak test for new versions of our code & James > involves transmitting 1.5 million messages over about 5 days > to another server. This is using a 400 Mhz PII as the James > server, an identical server using James to receive the mails > and an 133Mhz Pentium to send the mails. > > As a company we have no problems in using James as the core > of our MLM system. It works, day in, day out. :) > > Hope this helps. > > -- Jason > > -----Original Message----- > From: Hut Carspecken [mailto:[EMAIL PROTECTED] > Sent: 16 September 2003 21:06 > To: [EMAIL PROTECTED] > Subject: Scaling JAMES onto More Than One Server > > Good Afternoon, > > I am familar with Noel's capacity numbers; however, I was > wondering if anyone has experience scaling JAMES to run on > multiple servers or to off load some of it functionality to > another server. I am looking at handling several hundred > thousand messages a day. > > > > Hut Carspecken > 972-371-5638 > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
