Re: [jira] Commented: (OPENJPA-370) LoadFetchGroup annotation was not recognized during the fetch1

Thu, 01 Nov 2007 07:46:40 -0800 

Hi Pinaki,

On Oct 31, 2007, at 9:46 PM, Pinaki Poddar wrote:


Hi Craig,
Let us keep two issues namely a) correctness and b) performance
separate.
The root question is
  a) is current implementation of LoadFetchGroup incorrect?
The secondary question is
  b) does current implementation of LoadFetchGroup result in optimal
performance?

My current position on (a) is: no evidence conclusively suggests it to
be incorrect.
The evidence I have is that the test case that's checked in runs and the test case is incorrect according to my interpretation of the load fetch group. In fact, there is no test method in TestFetchGroup that will exercise the load fetch group.
Are there tests in the BEA suite (perhaps written to JDO interface) that we can use to see how it works with Kodo? That might give us a clue as to where the anomaly lies. 

My current position on (b) is: Given fundamental data structures and
strategies,  the current implementation is optimal but not minimal in
SQL generated. To determine all the requisite fields within a single
project of SQL query as in proposed alternative to minimize SQL may
result in overall performance reduction as per Patrick's observation.
I'm speculating here, but I don't believe that you can explain a 20% performance regression by some additional code running when loading instances. You can explain it by executing more than the number of SQL statements. So my working hypothesis is that we're generating too much SQL when running with the patch compared to without the patch. 

I will try to support (a) with my own test cases (as positive example)
and (b) analyze Teresa's test cases (as negative example) to ascertain
my position or otherwise, eat my own words.
I'd like to see how your analysis of the currently checked in test case compares with mine (it's a comment on issue https:// issues.apache.org/jira/browse/OPENJPA-370 ) and see if we agree on the proper behavior. Then we can talk about implementation. 

Craig


Even if (a) is provably true, we should seek solution within current
pathways than to introduce changes that may impair overall performance. 



Pinaki Poddar
972.834.2865



-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: Wednesday, October 31, 2007 11:14 PM
To: [email protected]
Subject: Re: [jira] Commented: (OPENJPA-370) LoadFetchGroup
annotation was not recognized during the fetch1

Hi Pinaki,

On Oct 31, 2007, at 8:55 PM, Pinaki Poddar wrote:


Hi Craig,
  I value your observation that we should be careful in using three
terms: 'accessed', 'loaded' and 'fetched' in this context. Let me
state my (limited or even inaccurate) understanding of these terms:
  accessed : a field f is accessed when user application attempts to
get its value for a managed instance X.


ok


  loaded: a field f is loaded in X if the value for the field is
considered to be the same as that of the datastore.


I'd prefer to consider a field loaded after it's been fetched
from the datastore. The issue with using loaded in our
discussion is that "load fetch group" loses its context.


  fetched: a field f is fetched when its value is read from

datastore

and set to the instance X

  You may correct the above by my following statements are based on
the above understanding.


Activating a fetch group doesn't do anything.

A more accurate statement would be "execution of a query or find()
operation with a fetch configuration that includes fetch group named
"f4f5".


This is wrong. Only one SQL should be issued, to fetch a specific
instance.

I beg to differ on qualifying it as 'wrong'. Number of SQL issued to
meet a user contract can not be mandated. Trying to issue

only one SQL

in this case within current implementaion/data structure strategy
perhaps will lead to us to a solution similar ro what Teresa

outlines.

Ok, but I'd say a key design decision is to limit the number
of SQL statements issued in order to retrieve fields from a
single instance to exactly one.



So this isn't exactly a good example, because it matters whether p1
(the instance of PObject) is already loaded into memory or not.

I should have underlined that everything is hollow as a pre-
condition. I agree that if f3 is pre-loaded then this must

not result

in fetching f4 and f5 from datastore.


Here's where I think we are having a problem. If p1 is hollow,
then one SQL statement is all you should need to fetch  the
non- relationship fields. The current fetch plan will result
in fetching f1, f2, and f3. If you get these fields and then
decide to go back and get f4 and f5, it's wrong. It's not
acceptable to go to the datastore to fetch fields and then
after you get them, to go back and get more because of a "load
fetch group" analysis.

Craig



Pinaki Poddar
972.834.2865



-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: Wednesday, October 31, 2007 10:02 PM
To: [email protected]
Subject: Re: [jira] Commented: (OPENJPA-370) LoadFetchGroup
annotation was not recognized during the fetch1

Hi Pinaki,

On Oct 31, 2007, at 6:19 PM, Pinaki Poddar wrote:


The implementation is ensuring that the user instance is

loaded with

requisite fields that can be specified by
A) fetch configuration
B) fields that are directly accessed
C) fields that are included because they are member of a
load-fetch-group of another field which is being fetched/accessed.


Not exactly. Accessed, yes. Fetched, no. This distinction

is critical

to understanding how load fetch groups are supposed to work.


For concreteness, let us consider the following

@Entity
public class PObject {
        @Id
        private long id;        
        private Integer f2;
        @LoadFetchGroup("f4f5")
        private Integer f3;
        private Integer f4;
        private Integer f5;

Where FetchGroup "f4f5" consists of (you guessed it right:) {"f4",
"f5"}.

Consider f3 being accessed either by direct load such as
Pobject.getF3()
or activating a FetchGroup {"f3"}.


Activating a fetch group doesn't do anything. Going to the back end
to fetch instances causes the fetch plan to be analyzed and fetch
groups to actually make something happen.

So this isn't exactly a good example, because it matters whether p1
(the instance of PObject) is already loaded into memory or not.

So if you em.find(PObject.class, 1) then the current fetch plan is
analyzed and if f3 is not in it, then f3 isn't fetched. Full stop.


The resultant PObject instance with f1,f2 unloaded and f3,f4,f5
loaded.
However, this will result in two separate SQL being issued.
SELECT t0.id, t0.f3 FROM pobject t0
SELECT t0.f4, t0.f5 FROM pobject t0 WHERE t0.id = ?


This is wrong. Only one SQL should be issued, to fetch a specific
instance.


The first SQL is resulted because f3 is included in current fecth
configuration while f4 and f5 are not.


If this is the case, then you should stop here. The load

fetch group

for f3 must be ignored.


The second SQL is resulted of StateManagerImpl.load() that

detects f3

has a fetch group "f4f5". It is part of the current fecth

session and

not a result of post-fetch analysis.


Here's where the trouble is. If f3 is not part of the fetch plan,
then it should not be fetched.


However the way it is done is by
adding "f4f5" to the active fetch configuration temporarily

and going

through similar cycle that caused the first SQL.


This should only occur if p1 is loaded, f3 is not loaded, and
f3 is accessed.


The concern is I am observing via few test cases this mechanics
working to fulfill the user contract. However, Teresa/Kevin
reported/observed that LoadFetchGroup is not working. Further
investigation on this failure is required before introducing
alternative solution.


So we need to look at the test cases to validate them

before we "fix"

anything.

Craig



Regards --


Pinaki Poddar
972.834.2865



-----Original Message-----
From: Craig Russell (JIRA) [mailto:[EMAIL PROTECTED]
Sent: Wednesday, October 31, 2007 6:52 PM
To: [email protected]
Subject: [jira] Commented: (OPENJPA-370) LoadFetchGroup

annotation

was not recognized during the fetch1


   [
https://issues.apache.org/jira/browse/OPENJPA-370?page=com.atla
ssian.jira.plugin.system.issuetabpanels:comment-tabpanel#action
_12539221 ]

Craig Russell commented on OPENJPA-370:
---------------------------------------


Before the instance is returned to the caller, the fields

are checked again if they cause any other field be loaded because
LoadFecthGroup. At that point, if field f has a

LoadFetchGroup L and

L is not part of the active fetch configuration then

*temporarily*

add L to the active FetchConfiguration and go for another 'fetch'
(i.e. from the datastore).

This sounds wrong. The effect of the load fetch group

should be part

of the fetch strategy, and no post-fetch analysis should be

done. The

only time the load fetch group is used is if a field f is

accessed

and it's not already fetched.

The intent of the load fetch group is to augment the fetch

plan under

which the persistent instance was fetched. It's designed

to provide

an intelligent fetch strategy for the lower-usage cases

where some

use needs field f1 (not in any fetch group in the current

fetch plan)

and when using field f1 you want to also fetch fields f2,

f3, and f4,

that are also not part of the current fetch plan.



LoadFetchGroup annotation was not recognized during the fetch1
--------------------------------------------------------------

                Key: OPENJPA-370
                URL:

https://issues.apache.org/jira/browse/OPENJPA-370

            Project: OpenJPA
         Issue Type: Bug
         Components: kernel
   Affects Versions: 1.0.1, 1.1.0
           Reporter: Teresa Kan
           Assignee: Teresa Kan
            Fix For: 1.0.1, 1.1.0

        Attachments: OPENJPA_370_2.patch, TestFetchGroup.zip


Employee class has a LoadFetchGroup annotation defined on

the Rating field, when getRating was called, the address

should be

returned also. However, openjpa did not handle the LoadFetchGroup
correctly, therefore, address was not eargly fetched.

public class FGEmployee{
    @Id
    private int id;

    @OneToOne(fetch=FetchType.LAZY)
    private FGAddress address;

    @Basic(fetch=FetchType.LAZY)
    @LoadFetchGroup("AddressFetchGroup")
    private String rating;

    @ManyToOne(fetch=FetchType.LAZY)
    private FGManager manager;
..
}


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.




Notice:  This email message, together with any attachments, may
contain information  of  BEA Systems,  Inc.,  its

subsidiaries and

affiliated entities,  that may be confidential,  proprietary,
copyrighted  and/or legally privileged, and is intended

solely for the

use of the individual or entity named in this message. If

you are not

the intended recipient, and have received this message in error,
please immediately return this by email and then delete it.


Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/
jdo
408 276-5638 mailto:[EMAIL PROTECTED] P.S. A good JDO? O, Gasp!




Notice:  This email message, together with any attachments, may
contain information  of  BEA Systems,  Inc.,  its subsidiaries
and  affiliated entities,  that may be confidential,  proprietary,
copyrighted  and/or legally privileged, and is intended

solely for the

use of the individual or entity named in this message. If

you are not

the intended recipient, and have received this message in error,
please immediately return this by email and then delete it.


Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/ jdo 
408 276-5638 mailto:[EMAIL PROTECTED] P.S. A good JDO? O, Gasp!
Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. 

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:[EMAIL PROTECTED]
P.S. A good JDO? O, Gasp!

Attachment: smime.p7s
Description: S/MIME cryptographic signature

Reply via email to