Hi, Shilpa. How about stripping the caching functionality of your filememrepository to a cacherepository instead? This cacherepository could then cache the messages and send them to another real persistent repository, thus making the cache functionality pluggable to all available repositories.
Hiran ----------------------------------------- Hiran Chaudhuri SAG Systemhaus GmbH Elsenheimerstra�e 11 80687 M�nchen Germany [EMAIL PROTECTED] Phone +49-89-54742-134 Fax +49-6151-9234-5134 > -----Original Message----- > From: Shilpa Dalmia [mailto:[EMAIL PROTECTED]] > Sent: Monday, May 13, 2002 10:47 PM > To: 'James Developers List' > Subject: diff files - writing mails to the spool store in between > processo rs > > > Attached are the files added/changed to provide the following > functionality. > > 1. There are foll. 3 new files added. > mailrepository/FileMemMailRepository.java > mailrepository/FileMemSpoolRepository.java > mailrepository/MimeMessageMemorySource.java > Implement a new mailstore "filemem". This is based on the > AvalonMail store > (disk store) that first stores the mail messages on the disk > & caches in > memory. All subsequent reads & writes are done from the cache. > > 2. > transport/LinearProcessor.java (diff file attached) > Avoid renaming the mail messages while writing back to the > spool after a > processor finishes its processing. > > 3. > transport/JamesSpoolManager.java (diff file attached) > Remove the message from the spool only if the message is in > GHOST state or > there are no more recipients attached to the message. > > > > > Shilpa Dalmia > PostX Corporation > > -----Original Message----- > From: Serge Knystautas [mailto:[EMAIL PROTECTED]] > Sent: Thursday, May 09, 2002 3:51 PM > To: James Developers List > Subject: Re: writing mails to the spool store in between processors > > > Shilpa, > > Both sound like good ideas... we're about to do an update to > the latest > Avalon, so if you could wait until that's complete before you > send any > diffs, that would be great. > > Also, I'd be just as interested in what tests you've setup... > we could > really use those as well. > -- > Serge Knystautas > Loki Technologies - Unstoppable Websites > http://www.lokitech.com/ > > Shilpa Dalmia wrote: > > I'm putting down a step by step listing of the proposed > solution with the > > effect it will have on current processing. > > > > Problem we're trying to solve > > 1. Avoid having to rename the mail message everytime the > message is being > > written back to the spool to be transferred to the next processor. > > 2. Improve performance by minimizing disk I/O for disk spooling. > > > > Solution > > > > Part 1: > > > > Do not rename the message before writing to the spool. > Remove the message > > only if it has finished processing (check the state is GHOST or no > > recipients to be processed). > > > > Advantage: > > 1. Reduce disk i/o by not having to rewrite the same > message again to the > > disk, just writing the header is enough. > > 2. Does not affect too many files. The change has to be made only in > > LinearProcessor.java (remove the renaming of mail message). > > I've tested this solution & it gives a 40% boost in the performance. > > > > snippet of service method() in LinearProcessor (changes > highlighted in > > bold.) > > > > //mail.setName(newName(mail)); > > spool.store(mail); > > mail = null; > > continue; > > } else { > > //Ok, we made it through with the same > state... move it to > > the next > > // spot in the array > > unprocessed[i + 1].add(mail); > > } > > > > > > and > > JamesSpoolManager.java (check if the mail is in GHOST state > or has no more > > recipients, before removing it) > > > > if (mail.getState().equals(mail.GHOST) || > > mail.getRecipients() == null || > > mail.getRecipients().size() == 0) > > { > > spool.remove(key); > > getLogger().info("==== Removed from > spool mail " > > + mail.getName() + " ===="); > > } > > mail = null; > > > > Part 2: > > > > We could further improve the performance (minimize disk > i/o) by caching > the > > message when it is first written to the spool. I've created > a new protocol > > "filemem" based on the existing "file" protocol which uses both disk > > spooling and memory spoling. Create a memory spool > repository & write the > > spool message to this spool after it has been written to > the disk spool. > All > > subsequent spool accesses, by different processors, would > be through the > > memory spool. Once processing is done, message will be > removed from both > the > > spools. Disk spool will be used for retries. > > > > I have implemented & tested these solutions. If you think > it is worth > > incorporating it in the James code base, I'd be happy to do that. > > > > Shilpa Dalmia > > PostX Corporation > > > -- > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > > -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
