Unfortunately, the numbers in the table from my previous post were
measured without the profiler.
Many thanks for the info.
If we proceed with James you'll probably hearing a lot more from us.
Mark Lim Software Yokozuna
Brightmail, Inc.
415-365-6192
http://www.brightmail.com
On Wed, 13 Jun 2001, Serge Knystautas wrote:
> Thanks for the detailed information. I can definitively say your
> performance numbers suck because you're using OptimizeIt. I used OptimizeIt
> to find the bottlenecks in James prior to the 1.2.1 release, specifically
> the TownSpoolRepository, and while that was a while ago, I do remember James
> ran roughly 5 to 10 faster when not within OptimizeIt. I believe the
> profiling prevents the JIT from running, or even if it does, the profiling
> puts an enormous load on everything. It's great for finding hotspots, but
> it doesn't represent true system performance.
>
> As for the reason James slows down as the spool grows... it's because each
> time the spooler thread goes to grab a message from the db, it gets the
> message_id and last_updated (and maybe another field or two) of ALL messages
> in the spool to determine what message is not locked, to determine which it
> should grab. This isn't really avoidable as the DB query won't tell you
> which messages are locked, so you have to get a list, and try one by one to
> acquire an in-memory lock on that message. With more control over the
> result set, you could set it up to read only 20 records, and only read the
> remaining 4980 if all those 20 records were locked. That would obviously
> boost performance significantly, but again, we would need probably raw JDBC
> access, not Town (or Turbine or other OR mapping software).
>
> There are some quick thoughts. I usually set my delivery threads between 5
> and 15 depending on how many messages I'm sending out. The only real
> benefit I've found with increasing threads is to overcome the latency of
> remote servers. If you're delivering messages to a local machine, having 2
> or 15 threads really makes little effect on throughput.
>
> Serge Knystautas
> Loki Technologies
> http://www.lokitech.com/
> ----- Original Message -----
> From: "Mark Lim" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Sent: Wednesday, June 13, 2001 12:28 PM
> Subject: James performance
>
>
> > Thanks for the initial responses. For the record, we're looking only at
> > James' SMTP performance. However, we may "deliver" messages to James by
> > scribbling right to the spool.
> >
> > The test rig was a 4 processor E450 w/ 1Gb of memory. James was fed mail
> from
> > a "pounder", which simply sends a number of fixed size messages via smtp
> to a
> > target. The tester used OptimizeIt, a profiler which came with Sun's JRE
> > v.1.2.2, which was the VM used in the test.
> >
> > A quote from the report:
> >
> > " Under a variety of different tests, James seemed to be able to suck
> > down very small SMTP messages (~2k) at a max rate of about 5
> > messages/sec under a pounding load. The profiler shows that James
> > spends a significant amount of time in various I/O routines, possibly
> > because it is always doing a cycle of "read from socket, write to
> > spool, read from spool, filter through mailets, write to inbox". "
> >
> >
> > He ran a couple of tests where he preloaded the spool, picked up the
> message,
> > processed it and then dropped it. This was an attempt to isolate the
> > processing stage alone. BusyWait is a mailet that simply spins for
> > a number of "milliseconds", taking up processor time to simulate work on
> the
> > message.
> >
> > 1000 8Kb messages, 8 spoolmanagerthreads
> > no BusyWait: 182 s
> > BusyWait 100: 184 s
> > BusyWait 1000: 278 s
> >
> > 1000 100Kb messages, 8 spoolmanagerthreads
> > no BusyWait: 217 s
> > BusyWait 100: 213 s
> > BusyWait 1000: 325 s
> > BusyWait 10000: 2222 s
> >
> > Sadly, the rates with no Busywait drop non-linearly with spool size. A
> spool
> > of 500 members is processed in 79 seconds (6.3 msgs/sec) but a spool of
> > 5000 members is processed in 47 minutes (1.8 msgs/sec). So when
> > the spool gets behind, it'll be a struggle to catch up. Spiky behavior is
> > very common in mail flow rates as seen from the MTA (start of the business
> > day, stock alerts, the AnnaKournikova worm...). While processing the
> spool,
> > the CPU usage levels off at 25%, unless BusyWait is being demanding.
> >
> > How many spoolmanagerthreads do people typically use? Any other tuning
> > tips?
> >
> >
> > Mark Lim Software Yokozuna
> > Brightmail, Inc.
> > 415-365-6192
> > http://www.brightmail.com
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
