I take it from your comments below that a subsystem (POP3, SMTP etc) would be responsible for doing it's own stats logging?
Would this model cope with such things as collecting the number of emails in a users account or counting the number of messages in the spool? These actions would often be generated on a time interval, rather than continously generated (I want to know how many emails an hour I delivered, for example). I supposed this could be the job of a stats timer that would gather such information and update the stats registry as appropriate. The other thing that any implementor of the stats interface would need to be aware of is the volume of data being stored. Such information needs to be regularly written to disk, otherwise the VM will simply blow up in your face. The Qmail system I worked on was chatty, but all it did was write to disk based log files straight away, and leave any stats analysis to other near-line programs. I would not want to see mail processing hammered due to logging taking excessive CPU,memory or I/O. Counters are fine in memory, but more heavy information (email size, to , from etc) needs to written to disk. I might be belabouring the point here, but there also needs to be someway of getting the information out of James into a useful format. But on the whole I like the idea. -- Jason > -----Original Message----- > From: Noel J. Bergman [mailto:[EMAIL PROTECTED]] > Sent: 23 August 2002 20:59 > To: James Developers List > Subject: RE: [PATCH]Simple stats and monitoring for James > > > Jason, > > The implementation of mailInboundQueue() is a bit off. If > such information is desired, the behavior should be delegated > to the actual user repository, which can optimize it. For > example, with a JDBC repository, the value is "select > count(*) from ...", rather than selecting and iterating over > a row set. > I was trying to write a high-level implementation so I didn't have to care about the underlying storage, but I think on balance you're correct. The performance would suck :-( > You should be prepared to see the other delegated statistics. > For example, I expect that the user repository data model > will eventually support per user quota / billing related statistics. > > Based upon this, I'd like to propose that now that you've > provided a Q&D PoC to show that a stats service is A Good > Thing, that we take a step back and look at the model. I'd > like to suggest that a Statistics Server maintain a top level > Registry of Statistic objects. There can be a basic set of > common Statistic classes, such as those typically used for > counting some item (incident or traffic count), and other > items. A basic Statistic interface might be something like: > > public interface Statistic > { > /* it might be nice to know which bucket this is > without having to > * keep track of the key separately. */ > String getKey(); > > /* Describe this for logger/admin console */ > String getDescription(); > > /* String representation of the contents of the bucket */ > String toString(); > } > > You could also add direct support for rendering to the > various output interfaces. > > A statistics registry might be something like: > > public interface StatisticRegistry > { > // Existing stat object for String; else null > Statistic getInstance(String); > > // Existing stat object for String; else install > Class.newInstance(); > Statistic getInstance(String, Class); > > // Existing stat object for String; else install parameter > Statistic getInstance(String, Object); > } > > Objects that contribute to a given statistic will > acquire/register an appropriate Statistic instance. The > instance may be one of several common types, such as a > counter, or something custom. Those accumulating or > analyzing the statistic will further need to know what type > of Statistic it is so that they can mutate it; others may > only care about rendering it. I can think of several common > types off-hand, and a few custom ones (e.g., queue length, > which is a read-only real-time value that could have current, > max and avg components). > > This is a quick strawman to promote discussion. > > --- Noel > > -----Original Message----- > From: Jason Webb [mailto:[EMAIL PROTECTED]] > Sent: Friday, August 23, 2002 6:51 > To: 'James Developers List' > Subject: RE: [PATCH]Simple stats and monitoring for James > > I've fixed the embarrassing problems you mentioned... > > I will extended the stats package to allow any subsystem (NNTP, POP3 > etc) to log their own stats. > > This will be of the form: > > public void logSubSystemStats(String subsystemName,String > key,float value); > > These stats will be written out a ONE log file (at the > moment) as and when they are received. I will use another > thread to do a "write-behind" style buffer to ensure the main > server is not crippled when stats logging is active. > > The type of items that would use this style could include: > Connection counts for all sub-systems NNTP message posts (key > is groupname?) User account mail counts > > However, I will keep the mail send & receive logging, as the > mail objects contain enough information to make the above > method useless. The bulk of the logging we want is the > "who-sent-what-to-who" type thing. They will be logged to a > file that contains just mail tranmission stats. > > -- Jason > > > -- > To unsubscribe, e-mail: > <mailto:james-dev-> [EMAIL PROTECTED]> > For > additional commands, > e-mail: <mailto:[EMAIL PROTECTED]> > > -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
