Bernd Fondermann [mailto:[EMAIL PROTECTED] wrote on: 09 October 2008 22:26 > Hi, > > It's high time to move the discussion back to server-dev, me thinks. > > Until now, this was primarily discussed at JIRAs MIME4J-72 > https://issues.apache.org/jira/browse/MIME4J-72 > and additional discussion in thread > MIME4J - remove parent reference from Body? > > Thanks to Markus Wiederkehr for starting this initiatives. > > The underlying requirement is motivated by the fact that we > don't want > large email parts to be stored in memory - because either one part > exceeds the available heap or a set of parts in sum do. > > So the requirement is to have those large parts stored on disk. > > The current mime4j implementation uses "temporary storage", > making use > of temporary files as supported by the JDK. > > This is a choice made easy by the JDK, yet it has its shortcomings. > Tempory files of this kind might not be so temporary. They > are collected > only on JVM termination. For long running applications, where JVM > termination happens not that often, its easy to run out of disk space > and file descriptors. > Thus it was proposed to extend the mime4j API to dispose of temporary > files, make them shorter lived. > > Another requirement voiced is to be able to copy emails. To save > resources, it was proposed to implement this by re-using (not > deep-copying) those objects which are not altered, including possibly > reusing one temporary file for more than part. > > I don't think that JDK temp files are neccessarily a good > choice. I can > imagine different implementations using a specific disk > partition, user > account, quota, directory - anything where the developers can > take more > control about the storage. > An optimization would be to keep parts up to a few KB in > memory and only > starting to write out to disk if a part grows beyond that point (or > available heap drops below a threshold). > All this would require a more sophisticated resource management than > what we have now. Moreover, I firmly believe that the current > temp file > stuff already does, when disposing and copy-on-write is combined! > > What we have now (without dispose()) is a more or less immutable, > threadsafe, kind-of-idempotent object model. This did fundamentally > changes with introducing dispose(). Not neccessarily a bad thing, but > worth noting and discussing. > > I suggest to take a close look at the recent changes and proposed > patches concerning resource management optimization. Let's > not include > it in the upcoming release (or delay the release) and make sure we > understand it and assure it works properly. > > Ideally, I'd like to see this resource management stuff > separated from > the core entities and parts object model, so we can properly > test it and > let our users make a proper choice depending on their use cases. For > short running, simple usage the existing and extended naive temporary > file management might be a reasonable choice. For long running, > multithreaded applications a more sophisticated approach (to be > implemented!) might be more appropriate. (And I must admit that I'm a > user from the latter camp.) > > I am not trying to artificially complicate things, just > throwing my own > requirements and architectural experiences in. > > Thanks for listening, > > Bernd
Bernd This all makes total sense to me. As you say, different use cases can be addressed by decoupling the resource provider from the resource consumer. The configured resource provider implements the chosen resource management policy. A resource management policy for long running applications would need to be efficient in their consumption and release of resources. Another policy concern is security. For instance, without other safeguards, temporarily storing messages elements in a file system potentially exposes them to scrutiny by others. Simple implementations might be happy running in memory. A decoupled architecture using injected components accommodates these and other use cases as they are discovered. Cheers -- Steve --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
