Niclas Hedhman schrieb:
I think the main concern is about versioning and handling of
consistency, you recognized that A will receive modified data the
second time around, but there would be other cases as well. The ACTUAL
solution can be divided into 2 parts;
1. What is known as CopyOnWrite, i.e. any cached state is shared
until modified.
CopyOnWrite would be very cool but does not help in situations where writers
outside the JVM can access and modify the backend. I think CopyOnWrite would
help with efficiency not consistency.
2. Disk-based cache extension, so that only N MB is in-memory and
then overflows to disk.
Most backends are fast enough to reload entities every time they are needed -
they have to do it when starting/loading an UoW anyway. Re-storing all
unmodified entities in some other disk based cache would introduce a lot of
disk/cpu overhead. Holding just the lastModified timestamps should be enough. (?)
If there would be an SPI the plugin any UoW cache implementation, then one could
decide which is the proper strategy for a given application. I my case with lots
of unmodified entities I would use soft-reference cache, where in other
situations with lots of modified entities a disk-overflow-cache would be better.
I doubt that soft references are truly the solution.
It is more likely that better modeling could have prevented this in
the first place, as I have a hard time seeing that many entities need
to be part of a UoW. Could you elaborate a bit on the model and/or
use-cases??
Say displaying a list of spatial features as the result of an query (maybe a
bounding box) in a GIS app. The user selects the rectangle of the bbox, the
result gets displayed in a table. The user then sorts the table for one column,
picks an entity, opens an editor...
Unfortunatelly sorting in the backend is not an option because most properties
are computed. So IFAIS I need to fetch all entities to sort them. Just a few
entities are used afterwards to display the visible lines of the table. Almost
all entities are probably never used again. The cool thing with the soft
references is that, *if* the user makes some more selection and/or sorting, then
all this is very fast as long as there is enough memory to hold all the entities.
Thanks for the response and help.
-Falko
Cheers
Niclas
2011/8/1 Falko Bräutigam <[email protected]>:
Hi all,
in one of my apps users are working with a lot of entities (~10000) quite
frequently. The entire user session runs inside one UoW. As the UoW caches
all entities it ever reads this leads to memory problems. Loading 10000
entities allocates ~100MB. This is ok for a short period while the user
actually works with the entities - but the UoW cache holds the entire 100MB
for the whole user session.
So I tried a weak/soft reference cache in UnitOfWorkInstance. This seems to
work but there are some open questions/problems:
- The store and some concerns need to use weak/soft references too. Ok did
it.
- If the store uses soft references, then it needs a way to get notified if
an entity has been modified, so that it can make a strong reference to it
(in order to not loose modified state). Currently I dont see a way to let
the store know (without introducing a new method).
- My simple approach introduces a race condition that may lead to lost
update in the worst case. Situation: A reads entity and presents it in the
UI. Then GC reclaims the entity. Then B reads/modifies/commits this entity.
Then A reads the entity again (together with new timestamp from B), modifies
something that B has modified and writes back to the store loosing changes
from B.
- Is an UoW supposed to be accessed by concurrent threads? If yes then
check/set entities from cache code in UnitOfWorkInstance needs to be
synchronized with the cache.
Any ideas? And/or are there any other problems/pitfalls I'm missing so far?
Thanks,
Falko
--
Falko Bräutigam
http://polymap.org
_______________________________________________
qi4j-dev mailing list
[email protected]
http://lists.ops4j.org/mailman/listinfo/qi4j-dev
--
Falko Bräutigam
http://polymap.org
_______________________________________________
qi4j-dev mailing list
[email protected]
http://lists.ops4j.org/mailman/listinfo/qi4j-dev
