On Fri, Oct 16, 2009 at 12:06 PM, Rickard Öberg <[email protected]> wrote:
> How can we fix this? One way to make the pain smaller is to simply only > deserialize properties/associations when they are actually used. Reading the > whole state of an entity should be reasonably ok, even though it may load a > lot of state that won't be used. Another way to do it is to use > key-value-stores in an entirely different way compared to now: instead of > having key-<blob> we could do key-property/association/manyassociation. Isn't this the equivalent of "Loading Policy" in JDO/JPA ?? I think this is a "per EntityStore type issue" more than it is a client code issue, although the client code will have a huge influence which loading strategy is the best for a given EntityStore. So, I tend to lean towards introducing "Loading Policy" in the MapES first, and perhaps even my favorite approach of 'self learning' against a use-case, i.e. the ES will internally keep track of which properties (and possibly associations) that a particular use-case uses, and pre-load those upon a request. Another issue that I can see is that of performance in for instance JDBM... I would assume that making many key lookups are relatively expensive, and we would need to look at "Lookup/Size speed ratio", i.e. how many bytes larger blob in the single lookup is required cost wise for each new key-lookup? And that is needed at the particular MapES implementation, as the values for JDBM would be dramatically different from a network based. But, you probably realize that reading the Blob is one step and creating the Property instances is another with its own overhead, potentially very substantial, and here it is probably a fixed time per instance, i.e. per first use --> create. I think it is a complex area, with room for significant speed improvements in large entities, but I also think that the answers will surprise us once we start measuring. We need a Performance Expert :-) who can dedicate himself/herself to chasing performance in ES and Indexing/Query. Cheers -- Niclas Hedhman, Software Developer http://www.qi4j.org - New Energy for Java I live here; http://tinyurl.com/2qq9er I work here; http://tinyurl.com/2ymelc I relax here; http://tinyurl.com/2cgsug _______________________________________________ qi4j-dev mailing list [email protected] http://lists.ops4j.org/mailman/listinfo/qi4j-dev
