Hi Joe,
Your (original) understanding of weak references + data contexts in
cayenne 3 is correct.
Cayenne 2 did used strong references. Cayenne 3, however, uses weak
refs. So as long as your application doesn't have any references to
the object(s), it (they) should be properly GC'd.
As for the relationships issue... I wrote an app awhile ago that had
relationships where the "many" side of the relationship could easily
be in the thousands, and even ten thousands.
What I wound up doing was implementing a custom
"LowMemoryRelationshipList" that circumvented some of the internals of
cayenne to handle the relationship manually.
As for the relationship-imposed list/memory-based filtering... this
same app allowed you to navigate directly from one row to a separate
view for relatedd objects. For instance, if viewing a list of users,
there would be a column for "attendance" that would report on the
number of attendance records associated, and then allow you to click
the # of associated attendance records to go directly to a listing of
attendance records only for that user, and all of that was done by
direct sql querying. What's more, the same exact code handles all
entities and all relationships. One of the nice things about cayenne
is the very rich set of metadata it makes available to you about your
objects, and I was able to leverage that to build the system. So, I
think you don't necessarily /have/ to abandon cayenne-managed
relationships, although it will take a bit of creativity to keep as
much out of memory as possible. :)
Robert
On Apr 8, 2010, at 4/87:12 PM , Joe Baldwin wrote:
Michael,
thanks for your input. I have started analyzing this in detail and
the problem has become multidimensional.
Assuming a webstore pattern:
First, if the DataContext never goes out of scope, and the
DataContext never releases references to DataObjects, then
eventually *all* of the products will be loaded into the
DataContext. I can't imagine that this would be the default behavior.
Second: webstore reality trumps delegation to DBMS of filtering &
sort-ordering, since there are not only search-based fetches, but
productList relationships to "Category" and "Vendor" objects. I
was told by your team in a previous email that I (as designer) have
no access to modifying or overriding relationship select-queries.
Therefore if I use the Cayenne relationship (for a Vendor or
Category) to get a list of Products, I would would be forced to
apply a list-based filter or sort-ordering.
*If* the DataContext never releases DataObjects, then eventually (as
a result of the relationship imposed list-based filtering/ordering),
every Product will be faulted and loaded into memory.
Michael, this can't be correct because then it would mimic the
behavior of a memory leak.
On the other hand, if it is correct, then my only option left is to
completely disable all Cayenne relationships, and do custom select
queries which replace the relationships while including the required
filtering and sort ordering. This kind-of sounds screwy as well.
Therefore, I am now more confused about this than when I started. :)
My hope is that the DataContext now allows more control over garbage
collecting objects.
Joe
On Apr 8, 2010, at 7:32 PM, Michael Gentry wrote:
My Q4 comment may be based more upon my usage of Cayenne 2 than 3. I
know there is some weak reference stuff in 3 that I haven't looked
into yet and perhaps it will automatically clean things up now (which
would be wonderful). I've always taken the approach of evicting
things I no longer wanted so I could control my memory footprint
better. Maybe Andrus/Andrey/etc will chime in with better info here
if I am incorrect on this one.
mrg
On Thu, Apr 8, 2010 at 5:49 PM, Joe Baldwin
<[email protected]> wrote:
Hi Michael,
Thank you for your methodical answers.
RE Q3
I suspected that this might be one of my problems, and I am trying
to redesign so that I only filter & order small result lists.
RE Q4
Oops! OK, this one totally threw me. Given my understanding of
the behavior and usage of the DataContext is somewhere between
"wow" to "hmmm, I didn't know that", Perhaps I need to review a
primer. :)
But seriously:
Since I am deploying a web-app, my understanding is that in that
type of scenario the container (via the Tomcat CayenneFilter) is
instantiating and controlling the DataContext (and I am,
presumably now accessing it via
BaseContext.getThreadObjectContext();) However, beyond that piece
of understanding, I am not sure what to do with it in order to
manage objects.
I read your docs once more and it says that the DataContext uses
"weak references to store registered objects". I *thought* this
meant that (assuming I use the default configuration), that the
DataContext would allow the DataObjects to be garbage collected,
if there is no reference (Java Reference) and if I did not
implement query caching.
I am confused by reading you Q4 comment and then the "Memory
Management Strategy" documentation.
(Note: I am attempting to learn more about Cayenne so that I can
optimize (and re-factor) my code. I am still uncertain whether I
should be implementing caching, not implementing caching,
initiating garbage collection (after freeing objects), or just
sitting back and letting Cayenne DataContext handle it.)
Thanks,
Joe
On Apr 8, 2010, at 5:18 PM, Michael Gentry wrote:
Hi Joe,
Assuming 2000 products fetched with a paginated query and a page
size
of 50, I believe the answers are ...
Q1: 2000 hollow objects are created. Cayenne fetches the primary
keys of the objects first in a paginated query.
Q2: Whichever page (or pages) you access in the paginated list will
fault those objects for that page (grouped by 50 at a time) into
memory.
Q3: If you try to filter or sort in-memory, then it will have to
fetch
all of the objects. Might be better to re-issue a query to the DB.
Q4: If your list pointers go out-of-scope, but the DataContext is
still in-scope, then the objects will still be in the DataContext
until you evict them.
mrg
On Thu, Apr 8, 2010 at 4:24 PM, Joe Baldwin <[email protected]
> wrote:
Andrus,
In this same context, I have a question about what behavior I
should expect to see.
Let us assume for this scenario:
1. a web deployment with Tomcat, typical webstore pattern
2. 2000-5000 products with an attribute "name"
3. custom search engine built on top of cayenne
a. mostly standard configuration
b. set PageSize to 50
c. using (presumably) default caching
d. custom factory class method used to create a select query
4. a user searches for "guitar" substring in the name field
Questions:
1. Based on my tests, it appears that "hollow" objects are
created but no faults are fired. (true?)
2. If I display any product data for a subset less than
PageSize, then only <PageSize> number of objects are faulted
(true ?)
3. If I execute "filterObjects" or "orderList" on the resulting
list, I assume the fault is fired for all objects in the list.
(true ?)
4. What normally happens to these objects if my list pointers go
out of scope? (Are they GarbageCollected per Java rules?)
(Note: I have not been able to get much visibility in
understanding (3) & (4), but I think my tests show that 1 & 2
are correct.)
Joe
On Apr 8, 2010, at 12:42 PM, Andrus Adamchik wrote:
BTW, we started on some monitoring hooks inside Cayenne runtime
in 3.0 per http://issues.apache.org/jira/browse/CAY-1173 ,
however then it was decided that it'll work much better with
the 3.1 DI-based stack. So this is waiting for its time.
Andrus
On Apr 7, 2010, at 6:16 PM, Joe Baldwin wrote:
I am attempting to do more performance monitoring (with an eye
towards optimizing my design & use of DataObject lists and
lifetime).
The first thing I would like to do is monitor how many
DataObjects are created and which JSP session they belong.
My idea was to simply create a constructor for the DataObject
of interest and place monitoring hooks inside. I would assume
that I could also create a finalize method to do similar things.
Is this the best way to create custom monitoring using Cayenne
DataObjects or is there a better way?
(BTW, even after reading the docs, I am still somewhat
mystified by how my configuration specifications control the
life cycle and performance. So I am hoping these custom hooks
will make my design decisions more visible, especially with
multiple users.)
Thanks,
Joe
