Re: datacache

[Patrick Linskey]

But, Patrick, you lead a wider discussion on the implicit design

to comply and perhaps requires a revisit.

assumption: "the data cache is always correct with respect to the database". Going forward, with pessimistic transaction being part of the spec, this view of DataCache being the 'in-memory replica of database' will be harder

Bear in mind that OpenJPA bypasses the cache when using pessimistic locking.

2. Inconsistent in-memory model:

As I've mentioned before, I believe that any such inconsistencies should be treated as bugs, not design features.

Can we analyze the data in DataCache to detect such data that are prone to

inconsistencies and hence leave them from being cached?  perhaps yes.

We can definitely detect such issues.

Does it worth it? no, because assuming that data is mostly consistent, it will penalize the consistent model to save some careless programmer who could not keep their model consistent. Also InverseManager already can do

that at L1 cache level, for some inconsistencies.

I do not believe that w can deem up-front and in general that such inconsistencies are not worth resolving. I believe that we must look at these bugs individually when determining how to resolve things.

3. retiring the 'optimization' on refresh

that impacted behavior of a single instance. The collection form used

(but not the same) code for a single instance versus a collection of
instance is risky.

there were two refreshInternal() methods, one with a single instance and other with a collection. They had very similar code except a boolean flag loadAll() for better performance and I kept that. Maintaining very similar

-Patrick

On May 29, 2008, at 8:42 AM, Craig L Russell wrote:

What measurements, if any, did you perform along with the change? What is the performance impact of calling the plural method in the single- instance case?

On May 28, 2008, at 10:02 AM, Pinaki Poddar wrote:

Hi Patrick,

Historically, we've assumed that for the purposes of refresh(), the
data cache is always correct with respect to the database.

regard. My suggestion is leaned towards spec-compliance (or rather my

the user want".

Yes, I understand the import/impact of my suggestion to bypass DataCache altogether for refresh(). And hence I value what you are saying in this assumption of what the mythical 'user' wants as refresh() behavior). The spec says for refresh() as in JavaDoc: "Refresh the state of the instance from the database, overwriting changes made to the entity, if any". I read it as: "the user wants refresh() to hit the database". You can always refute that assumption because of the core axiom:" Nobody really knows what does

Craig

The issue here is that the data cache is not a specified concept. So we are really on our own here. The concept of refresh is in JPA, but the concept of evict is not. So I think that refresh should work as advertised with or without the data cache. Which means that data should be retrieved from the database (not the cache). But clearly, some applications will operate in an environment where the data cache is always kept current, by the global cache manager. In these cases, it is sufficient to refresh from the cache and not go to the database. For this situation, I'd prefer that the user set a flag in the EntityManagerFactory that says RefreshFromCache if the global cache manager can guarantee that the cache is kept in sync with the database. And I'd go farther, by asking that this concept be put into the JPA spec explicitly, if the expert group adopts the cache API.

flag was there and Broker.refreshInternal() used it to call the

behind that flag?

But, Patrick, you lead a wider discussion on the implicit design

to comply and perhaps requires a revisit.

2. Inconsistent in-memory model:
The other issue being discussed in this thread i.e. "inconsistent
in-memory data model" also tends to show limitation of the view that

Database represents relation/data differently than DataCache does.

in the database because while Java (and DataCache) represents a

for ambiguity.

inconsistencies and hence leave them from being cached?  perhaps yes.

that at L1 cache level, for some inconsistencies.


3. retiring the 'optimization' on refresh

that impacted behavior of a single instance. The collection form used

(but not the same) code for a single instance versus a collection of
instance is risky.




Patrick Linskey-2 wrote:

The other relevant bit is the flag FORCE_LOAD_REFRESH in StoreManager. The StoreManager.loadAll(). That is the only place the flag had been used. However, DataCacheStoreManager ignored it. In fact, I used it now to bypass DataCache for refresh(). May be some one can comment on the original intent assumption: "the data cache is always correct with respect to the database". Going forward, with pessimistic transaction being part of the spec, this view of DataCache being the 'in-memory replica of database' will be harder DataCache is a replica of database albeit from a representational form. Inconsistent relations, under some mapping, automagically becomes consistent bi-directional relation with a pair of variables (and hence leaving room for inconsistency), database has a single foreign key column and hence no scope Can we analyze the data in DataCache to detect such data that are prone to Does it worth it? no, because assuming that data is mostly consistent, it will penalize the consistent model to save some careless programmer who could not keep their model consistent. Also InverseManager already can do there were two refreshInternal() methods, one with a single instance and other with a collection. They had very similar code except a boolean flag loadAll() for better performance and I kept that. Maintaining very similar

b) bypasses Data cache for refresh() altogether

Historically, we've assumed that for the purposes of refresh(), the
data cache is always correct with respect to the database. I.e., the
assumption has been that if changes were made out-of-band, then the
developer would call DataCache.evict() prior to refresh(). It would
seem that your suggestion significantly changes this assumption.


Broker.evict() also calls DataCache.evict(). We should consider this

changing the semantics of refresh().

For the case of Broker.evict() calls, the Broker has a special setting (setEvictFromDataCache()) that allows configuration of whether or not precedent and the prior assumptions when coming up with a strategy for

c) throws away 'optimization' for a single entity

Can you describe this in more detail? Optimizations are generally
there for a reason -- getting rid of it might very well lead to sub-
optimal behavior.

-Patrick

On May 28, 2008, at 9:38 AM, Pinaki Poddar wrote:

Hi Craig,
Committed revision 660753 towards a model that
a) corrects refresh() behavior of single, clean entity
b) bypasses Data cache for refresh() altogether
c) throws away 'optimization' for a single entity

Pinaki


Craig L Russell wrote:

Hi Pinaki,

On May 27, 2008, at 3:00 PM, Pinaki Poddar wrote:

After some more analysis of refresh() issue...


hits the
database -- irrespective of whether Data Cache is active or not.
That does
not appear spec compliant.

1. it is observed that a refresh of a single, clean instance never

I agree.

it
reads from Data Cache; on any stronger lock it hits the database.
I am of the opinion that all refresh() must bypass data cache
altogether

user to

of-
band
modifications may have taken place on the database).

2. refresh() behaves differently on current lock level. With NO LOCK always -- because refresh() seems to express explicit intent of the read data from database (say when the application thinks that out-

I agree.

single

leaves
some maintenance concern of similar but not same code blocks) is
another
suggestion.

3. There is an 'optimization' on BrokerImpl.refresh() -- one for a entity and other for a collection. Removing that optimization (which

Seems that if the user calls refresh on a single entity or on a

we're smart, but we're not omniscient.

Craig

collection, then we should hit the database every time. Who are we to know that the database hasn't changed in the last millisecond? Sure,

Comments?


--
View this message in context:
http://www.nabble.com/datacache-tp17326391p17501042.html

Sent from the OpenJPA Developers mailing list archive at Nabble.com.

Craig Russell

Architect, Sun Java Enterprise System http://java.sun.com/ products/

jdo
408 276-5638 mailto:[EMAIL PROTECTED]
P.S. A good JDO? O, Gasp!

--
View this message in context:
http://www.nabble.com/datacache-tp17326391p17502434.html

Sent from the OpenJPA Developers mailing list archive at Nabble.com.

--
Patrick Linskey
202 669 5907

--
View this message in context: 
http://www.nabble.com/datacache-tp17326391p17517403.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:[EMAIL PROTECTED]
P.S. A good JDO? O, Gasp!

--
Patrick Linskey
202 669 5907

smime.p7s
Description: S/MIME cryptographic signature