hi Michiel It sounds goot to me. Some more cleaning up. I suppose you are in a better position than me to asses backwards-compatibility issues.
Ernst > -----Oorspronkelijk bericht----- > Van: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Namens Michiel Meeuwissen > Verzonden: dinsdag 29 maart 2005 22:56 > Aan: Discussion list for developers > Onderwerp: [Developers] [Optimization] Caching of 'blobs' and > Image's'handleCache'. > > > Tonight I intended to make the changes I proposed for the > 'images' framework ready for submission as a 'hack'. See > thread 'MMBase Imaging, request for comments on changes.' of > last week. > > While cleaning up the mess last week, I actually completely > dropped the 'handle cache' which was present in > 'ImageCaches', so I wanted to look into that and to see what > would need to be done here if we want to be sure to not loose > something valueable. It seemed a little issue. > > But no, if one looks into this, one soon remarks that the > mess extends a bit outside the image builders. > > I will recapitulate. > > - byte[]-fields are 'lazy' loaded. This means that the storage layer > will not automaticly fill them. > > - On a explicit request for such a field on the MMObjectNode > object, the > field is filled. > MMObjectNode also anticipates lazy loading of large text-fields, but > this seems not supported by the new Storage layer. I don't > know about > other storage-layers. > > - Such a actively loaded field is never unloaded. So, once the memory > is taken, is is taken as long as the MMObjectNode instance is in > memory. Which can be quite long, because I think the node-cache is > normally made very large, and can contain every node appearing > on the site. > > This is also valid for new nodes. > > - The goal of 'handleCache' of ImageCached seemed to be to cache the > byte-arrays of 'small' icache-objects (hard-coded on > 100kb). First of > all I didn't see the point of that, because I suppose most of the > memory of a completely loaded image-object would be in the > hande-field > any way, so you could just as easily cache the node itself. > > The method 'getCkeyNode' which existed in ImageCaches did > the work. It > actually did not return a Node, nor a Ckey. It should have > been named > 'getImageBytesByCKey', and it indeed was wrapped by a function > 'getCachedImageContainer' in the images builder. Now it makes some > sense: you can set the node-cache to a very small size, and use > getImageContainer to get the byte-arrays only. Then the > handle-cache is used > more or less sensible. > getImageContainer returns a 'ByteFieldContainer' object > (new in 1.7), > which is supposed to be for values in caches. > > It is very nice that you can get the bytes without the node, but I > think it is never used. Perhaps in servdb. But ImageServlet will > always need the complete node, because it wants to sends some > meta-information, and is implemented through bridge anyway. > > > I have succesfuly tried to make the following changes, and > ask for opinions. > > > - First of all, the 'lazy' loading of handle fields needs a clearer > implementation. Also suggested by all the comments in MMObjectNode > about the issue. > > - I decided to never 'load' a field which is not not loaded by > storage. So, I removed the 'storeValue''s from the > 'SHORTED' blocks in > MMObjectNode. This eliminates the need for unloading it. > > - If the value of such a field is requested from the > MMObjectNode, that > is recognized by the value '$SHORTED' (changed to a constant). The > value is then searched from a dedicated 'blob-cache', and if not > present, explicitely fetched from storage (using > 'getShortedByte' and > 'getShortedText) and put into this cache. This is > completely done in > 'getValue' so no specialized code is needed in 'getStringValue' and > getByteValue'. > > - The BlobCache is a specialized cache implementation > org.mmbase.cache.BlobCache, connecting a key with imagenumber and > fieldname to a large object (byte array or string) It is > returned by a > new method MMObjectBuilder#getBlobCache(String fieldName), which on > default always returns the same object (one instance of BlobCache), > but you can override that behaviour per builder, per field. In > Abstract-Images I did that and named it 'ImageHandles', so images > handle fields have their own dedicated cache then. Also > useful because > the 'ImageHandles' entry exists in all caches.xml's... > > - BlobCache of course uses the Cache-feature 'maxEntrySize', > so the 100k > limit becomes configurable, and you have full control on how much > memory you want to spent on 'blobs'. And you can e.g. > easily configure > that you want to cache a lot of small images (most images should be > smaller that 100k!), and e.g. a few large attachments. > > - Since you can also switch 'blobs on disk' to true, you may also > consider disabling this cache, and trusting the OS that it uses all > remaining memory to cache disk-actions. I have no idea if this makes > sense, but at least it is an option, and a possible entry for the > 'best practices' document of the performance sub-project. > > - BlobCache entries remain existing if the node itself > dissappears from > the node-cache. That could be changed, but it did seem as two > different things, so I did not do that. > > > - ByteValueContainer actually ended up to be used no-where, except in > the now deprecated method 'getCkeyNode', which I only did not remove > yet, because it is used in a vwms 'ImageMaster' of the scan > application. I think ByteValueContainer can be removed (just change > ImageMaster a bit), or made deprecated at least. > > > I hope some people actually read the above story, and even > better, agreed, or - also good - would like to comment. > > Otherwise I suppose we will decide in the 'optimization' > project what to do, and if my proposals make sense. > > > Michiel > > -- > Michiel Meeuwissen mihxil' > Mediacentrum 140 H'sum [] () > +31 (0)35 6772979 nl_NL eo_XX en_US > > > > _______________________________________________ > Developers mailing list > [email protected] > http://lists.mmbase.org/mailman/listinfo/developers > _______________________________________________ Developers mailing list [email protected] http://lists.mmbase.org/mailman/listinfo/developers
