Thanks for the detailed information. I can definitively say your
performance numbers suck because you're using OptimizeIt. I used OptimizeIt
to find the bottlenecks in James prior to the 1.2.1 release, specifically
the TownSpoolRepository, and while that was a while ago, I do remember James
ran roughly 5 to 10 faster when not within OptimizeIt. I believe the
profiling prevents the JIT from running, or even if it does, the profiling
puts an enormous load on everything. It's great for finding hotspots, but
it doesn't represent true system performance.
As for the reason James slows down as the spool grows... it's because each
time the spooler thread goes to grab a message from the db, it gets the
message_id and last_updated (and maybe another field or two) of ALL messages
in the spool to determine what message is not locked, to determine which it
should grab. This isn't really avoidable as the DB query won't tell you
which messages are locked, so you have to get a list, and try one by one to
acquire an in-memory lock on that message. With more control over the
result set, you could set it up to read only 20 records, and only read the
remaining 4980 if all those 20 records were locked. That would obviously
boost performance significantly, but again, we would need probably raw JDBC
access, not Town (or Turbine or other OR mapping software).
There are some quick thoughts. I usually set my delivery threads between 5
and 15 depending on how many messages I'm sending out. The only real
benefit I've found with increasing threads is to overcome the latency of
remote servers. If you're delivering messages to a local machine, having 2
or 15 threads really makes little effect on throughput.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Mark Lim" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Wednesday, June 13, 2001 12:28 PM
Subject: James performance
> Thanks for the initial responses. For the record, we're looking only at
> James' SMTP performance. However, we may "deliver" messages to James by
> scribbling right to the spool.
>
> The test rig was a 4 processor E450 w/ 1Gb of memory. James was fed mail
from
> a "pounder", which simply sends a number of fixed size messages via smtp
to a
> target. The tester used OptimizeIt, a profiler which came with Sun's JRE
> v.1.2.2, which was the VM used in the test.
>
> A quote from the report:
>
> " Under a variety of different tests, James seemed to be able to suck
> down very small SMTP messages (~2k) at a max rate of about 5
> messages/sec under a pounding load. The profiler shows that James
> spends a significant amount of time in various I/O routines, possibly
> because it is always doing a cycle of "read from socket, write to
> spool, read from spool, filter through mailets, write to inbox". "
>
>
> He ran a couple of tests where he preloaded the spool, picked up the
message,
> processed it and then dropped it. This was an attempt to isolate the
> processing stage alone. BusyWait is a mailet that simply spins for
> a number of "milliseconds", taking up processor time to simulate work on
the
> message.
>
> 1000 8Kb messages, 8 spoolmanagerthreads
> no BusyWait: 182 s
> BusyWait 100: 184 s
> BusyWait 1000: 278 s
>
> 1000 100Kb messages, 8 spoolmanagerthreads
> no BusyWait: 217 s
> BusyWait 100: 213 s
> BusyWait 1000: 325 s
> BusyWait 10000: 2222 s
>
> Sadly, the rates with no Busywait drop non-linearly with spool size. A
spool
> of 500 members is processed in 79 seconds (6.3 msgs/sec) but a spool of
> 5000 members is processed in 47 minutes (1.8 msgs/sec). So when
> the spool gets behind, it'll be a struggle to catch up. Spiky behavior is
> very common in mail flow rates as seen from the MTA (start of the business
> day, stock alerts, the AnnaKournikova worm...). While processing the
spool,
> the CPU usage levels off at 25%, unless BusyWait is being demanding.
>
> How many spoolmanagerthreads do people typically use? Any other tuning
> tips?
>
>
> Mark Lim Software Yokozuna
> Brightmail, Inc.
> 415-365-6192
> http://www.brightmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
