Pierre -- Pierre Frisch [EMAIL PROTECTED]
On Jan 9, 2008, at 15:04, Drew Thoeni wrote:
James,Implemented this yesterday and it works as advertised, that is, perfectly. I can calculate all the stats for the 18,000 parents and 100,000,000 children in an hour too! Thanks for the pattern.Best, Drew On Jan 6, 2008, at 5:55 PM, James C. Lee wrote:I should clarify that there are separate editing contexts for the parentsand children: public void foo() { NSArray parents; NSArray children; Enumeration anEnum; EOEditingContext ecParent, ecChildren; ecParent = new EOEditingContext(); // fetch parents with ecParent... anEnum = parents.objectEnumerator(); while (anEnum.hasMoreElements()) { aParent = (Parent)anEnum.nextElement(); ecChildren= new EOEditingContext(); children = fetch_with_editingContext(ecChildren); // processing... children = null; ecChildren(); ecChildren= null; System.gc(); } } ^James-----Original Message----- From: [EMAIL PROTECTED] ple.com [mailto:webobjects-dev-bounces+jcl_applewodev=dreamissary.com@ lists.apple.com] On Behalf Of James C. Lee Sent: Sunday, January 06, 2008 2:46 PM To: 'WebObjects-Dev List' Subject: RE: Large Array Clogging Drew, Try this: public void foo() { NSArray parents; NSArray children; Enumeration anEnum; EOEditingContext editingContext; // fetch parents anEnum = parents.objectEnumerator(); while (anEnum.hasMoreElements()) { aParent = (Parent)anEnum.nextElement(); editingContext = new EOEditingContext(); children = fetch_with_editingContext(editingContext); // processing... children = null; editingContext.dispose(); editingContext = null; System.gc(); } } ^James-----Original Message----- From: [EMAIL PROTECTED] ple.com [mailto:webobjects-dev-bounces+jcl_applewodev=dreamissary.com@ lists.apple.com] On Behalf Of Chuck Hill Sent: Sunday, January 06, 2008 1:57 PM To: Drew Thoeni Cc: WebObjects-Dev List Subject: Re: Large Array Clogging On Jan 6, 2008, at 11:53 AM, Drew Thoeni wrote:I have two tables in a parent-child relationship. Some of the parents have a few children (say a few dozen) and others have many children (over 100,000). I'm trying to run through theparents andpre-calculate some statistics for their children (for example, average, mode, standard deviation, etc.).I'd be tempted to take a long, hard look at some of the aggregate functions in Wonder that generate SQL for you. Batchprocessing isnot EOF's strongest area due to the overhead of objectcreation andgarbage collection.All this is working except when I reach a specific parent(#457) theprocessing just hangs. I ran this through debug and it stalls at the creation of the array for the children. However, the system would have already processed a similar size array (#30 has 118,000 children) and it seems it's in a loop. Activity monitor says the java process is taking 100%. Other background. I only save the ec after every 20updates to theparent records (to reduce write time). And I use ec.invalidateAllObjects() right after saving to clear our memory.Rather than do that, I'd create a new EC and just not retain _any_ references to the objects previously processed, the previous EC etc. EOF snapshot counting and Java GC should take care of the rest.Finally, there is nothing wrong with the data inparent/children of#457 as I can start the process there (or a few parents ahead of this) and it runs fine. This seems like a memory problem, but I don't get an outof memoryexception.Before OutOfMemory, there is memory starvation. I think that is what is happening to you. Increase the heap size and see ifthat delaysthe problem to a later parent. You could also launch with a JVM parameter of -verbose:gc to log when GC happens. The solution is to clean up your handling of objects and editing contexts. Chuck -- Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solvespecificproblems.http://www.global-village.net/products/practical_webobjects_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/jcl_applewodev%40dreamissary.com This email sent to [EMAIL PROTECTED] _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/jcl_applewodev%40dreamissary.com This email sent to [EMAIL PROTECTED] _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/drewt7%40comcast.net This email sent to [EMAIL PROTECTED]_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/pierre%40apple.com This email sent to [EMAIL PROTECTED]
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
