Robert,
OK, so I sort of understand the premise of weak references.
That brings up some questions:
1. Given that I set the Object Cache to 500 (mostly as a first guess)
2. what happens when the object count gets to 500?
a. I presume the "weak reference" model would initiate Java-GC on
un-referenced objects
b. If this is true, then what happens if all of the objects are
referenced? (Does Cayenne start throwing exceptions?)
3. How do I monitor the data object memory?
- I found this in the API:
((org.apache.cayenne.access.DataContext)BaseContext.getThreadObjectContext()).getObjectStore().getDataRowCache().size()
4. Would it be advisable to execute this method to release objects after a
session?
-
((org.apache.cayenne.access.DataContext)BaseContext.getThreadObjectContext()).getObjectStore().getDataRowCache().clear()
Thanks (but still confused :) )
Joe
On Apr 8, 2010, at 8:34 PM, Robert Zeigler wrote:
> Hi Joe,
>
> Your (original) understanding of weak references + data contexts in cayenne 3
> is correct.
> Cayenne 2 did used strong references. Cayenne 3, however, uses weak refs. So
> as long as your application doesn't have any references to the object(s), it
> (they) should be properly GC'd.
>
> As for the relationships issue... I wrote an app awhile ago that had
> relationships where the "many" side of the relationship could easily be in
> the thousands, and even ten thousands.
> What I wound up doing was implementing a custom "LowMemoryRelationshipList"
> that circumvented some of the internals of cayenne to handle the relationship
> manually.
>
> As for the relationship-imposed list/memory-based filtering... this same app
> allowed you to navigate directly from one row to a separate view for relatedd
> objects. For instance, if viewing a list of users, there would be a column
> for "attendance" that would report on the number of attendance records
> associated, and then allow you to click the # of associated attendance
> records to go directly to a listing of attendance records only for that user,
> and all of that was done by direct sql querying. What's more, the same exact
> code handles all entities and all relationships. One of the nice things
> about cayenne is the very rich set of metadata it makes available to you
> about your objects, and I was able to leverage that to build the system. So,
> I think you don't necessarily /have/ to abandon cayenne-managed
> relationships, although it will take a bit of creativity to keep as much out
> of memory as possible. :)
>
> Robert
>
> On Apr 8, 2010, at 4/87:12 PM , Joe Baldwin wrote:
>
>> Michael,
>>
>> thanks for your input. I have started analyzing this in detail and the
>> problem has become multidimensional.
>>
>> Assuming a webstore pattern:
>>
>> First, if the DataContext never goes out of scope, and the DataContext never
>> releases references to DataObjects, then eventually *all* of the products
>> will be loaded into the DataContext. I can't imagine that this would be the
>> default behavior.
>>
>> Second: webstore reality trumps delegation to DBMS of filtering &
>> sort-ordering, since there are not only search-based fetches, but
>> productList relationships to "Category" and "Vendor" objects. I was told
>> by your team in a previous email that I (as designer) have no access to
>> modifying or overriding relationship select-queries. Therefore if I use
>> the Cayenne relationship (for a Vendor or Category) to get a list of
>> Products, I would would be forced to apply a list-based filter or
>> sort-ordering.
>>
>> *If* the DataContext never releases DataObjects, then eventually (as a
>> result of the relationship imposed list-based filtering/ordering), every
>> Product will be faulted and loaded into memory.
>>
>> Michael, this can't be correct because then it would mimic the behavior of a
>> memory leak.
>>
>> On the other hand, if it is correct, then my only option left is to
>> completely disable all Cayenne relationships, and do custom select queries
>> which replace the relationships while including the required filtering and
>> sort ordering. This kind-of sounds screwy as well.
>>
>> Therefore, I am now more confused about this than when I started. :)
>>
>> My hope is that the DataContext now allows more control over garbage
>> collecting objects.
>>
>> Joe
>>
>>
>>
>> On Apr 8, 2010, at 7:32 PM, Michael Gentry wrote:
>>
>>> My Q4 comment may be based more upon my usage of Cayenne 2 than 3. I
>>> know there is some weak reference stuff in 3 that I haven't looked
>>> into yet and perhaps it will automatically clean things up now (which
>>> would be wonderful). I've always taken the approach of evicting
>>> things I no longer wanted so I could control my memory footprint
>>> better. Maybe Andrus/Andrey/etc will chime in with better info here
>>> if I am incorrect on this one.
>>>
>>> mrg
>>>
>>>
>>> On Thu, Apr 8, 2010 at 5:49 PM, Joe Baldwin <[email protected]> wrote:
>>>> Hi Michael,
>>>>
>>>> Thank you for your methodical answers.
>>>>
>>>> RE Q3
>>>> I suspected that this might be one of my problems, and I am trying to
>>>> redesign so that I only filter & order small result lists.
>>>>
>>>> RE Q4
>>>> Oops! OK, this one totally threw me. Given my understanding of the
>>>> behavior and usage of the DataContext is somewhere between "wow" to "hmmm,
>>>> I didn't know that", Perhaps I need to review a primer. :)
>>>>
>>>> But seriously:
>>>> Since I am deploying a web-app, my understanding is that in that type of
>>>> scenario the container (via the Tomcat CayenneFilter) is instantiating and
>>>> controlling the DataContext (and I am, presumably now accessing it via
>>>> BaseContext.getThreadObjectContext();) However, beyond that piece of
>>>> understanding, I am not sure what to do with it in order to manage objects.
>>>>
>>>> I read your docs once more and it says that the DataContext uses "weak
>>>> references to store registered objects". I *thought* this meant that
>>>> (assuming I use the default configuration), that the DataContext would
>>>> allow the DataObjects to be garbage collected, if there is no reference
>>>> (Java Reference) and if I did not implement query caching.
>>>>
>>>> I am confused by reading you Q4 comment and then the "Memory Management
>>>> Strategy" documentation.
>>>>
>>>> (Note: I am attempting to learn more about Cayenne so that I can optimize
>>>> (and re-factor) my code. I am still uncertain whether I should be
>>>> implementing caching, not implementing caching, initiating garbage
>>>> collection (after freeing objects), or just sitting back and letting
>>>> Cayenne DataContext handle it.)
>>>>
>>>> Thanks,
>>>> Joe
>>>>
>>>>
>>>>
>>>>
>>>> On Apr 8, 2010, at 5:18 PM, Michael Gentry wrote:
>>>>
>>>>> Hi Joe,
>>>>>
>>>>> Assuming 2000 products fetched with a paginated query and a page size
>>>>> of 50, I believe the answers are ...
>>>>>
>>>>> Q1: 2000 hollow objects are created. Cayenne fetches the primary
>>>>> keys of the objects first in a paginated query.
>>>>>
>>>>> Q2: Whichever page (or pages) you access in the paginated list will
>>>>> fault those objects for that page (grouped by 50 at a time) into
>>>>> memory.
>>>>>
>>>>> Q3: If you try to filter or sort in-memory, then it will have to fetch
>>>>> all of the objects. Might be better to re-issue a query to the DB.
>>>>>
>>>>> Q4: If your list pointers go out-of-scope, but the DataContext is
>>>>> still in-scope, then the objects will still be in the DataContext
>>>>> until you evict them.
>>>>>
>>>>> mrg
>>>>>
>>>>>
>>>>> On Thu, Apr 8, 2010 at 4:24 PM, Joe Baldwin <[email protected]>
>>>>> wrote:
>>>>>> Andrus,
>>>>>>
>>>>>> In this same context, I have a question about what behavior I should
>>>>>> expect to see.
>>>>>>
>>>>>> Let us assume for this scenario:
>>>>>> 1. a web deployment with Tomcat, typical webstore pattern
>>>>>> 2. 2000-5000 products with an attribute "name"
>>>>>> 3. custom search engine built on top of cayenne
>>>>>> a. mostly standard configuration
>>>>>> b. set PageSize to 50
>>>>>> c. using (presumably) default caching
>>>>>> d. custom factory class method used to create a select query
>>>>>> 4. a user searches for "guitar" substring in the name field
>>>>>>
>>>>>> Questions:
>>>>>> 1. Based on my tests, it appears that "hollow" objects are created but
>>>>>> no faults are fired. (true?)
>>>>>> 2. If I display any product data for a subset less than PageSize, then
>>>>>> only <PageSize> number of objects are faulted (true ?)
>>>>>> 3. If I execute "filterObjects" or "orderList" on the resulting list, I
>>>>>> assume the fault is fired for all objects in the list. (true ?)
>>>>>> 4. What normally happens to these objects if my list pointers go out of
>>>>>> scope? (Are they GarbageCollected per Java rules?)
>>>>>>
>>>>>> (Note: I have not been able to get much visibility in understanding (3)
>>>>>> & (4), but I think my tests show that 1 & 2 are correct.)
>>>>>> Joe
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Apr 8, 2010, at 12:42 PM, Andrus Adamchik wrote:
>>>>>>
>>>>>>> BTW, we started on some monitoring hooks inside Cayenne runtime in 3.0
>>>>>>> per http://issues.apache.org/jira/browse/CAY-1173 , however then it was
>>>>>>> decided that it'll work much better with the 3.1 DI-based stack. So
>>>>>>> this is waiting for its time.
>>>>>>>
>>>>>>> Andrus
>>>>>>>
>>>>>>>
>>>>>>> On Apr 7, 2010, at 6:16 PM, Joe Baldwin wrote:
>>>>>>>
>>>>>>>> I am attempting to do more performance monitoring (with an eye towards
>>>>>>>> optimizing my design & use of DataObject lists and lifetime).
>>>>>>>>
>>>>>>>> The first thing I would like to do is monitor how many DataObjects are
>>>>>>>> created and which JSP session they belong.
>>>>>>>>
>>>>>>>> My idea was to simply create a constructor for the DataObject of
>>>>>>>> interest and place monitoring hooks inside. I would assume that I
>>>>>>>> could also create a finalize method to do similar things.
>>>>>>>>
>>>>>>>> Is this the best way to create custom monitoring using Cayenne
>>>>>>>> DataObjects or is there a better way?
>>>>>>>>
>>>>>>>> (BTW, even after reading the docs, I am still somewhat mystified by
>>>>>>>> how my configuration specifications control the life cycle and
>>>>>>>> performance. So I am hoping these custom hooks will make my design
>>>>>>>> decisions more visible, especially with multiple users.)
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Joe
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
