Re: Fetch Optimization Advice
Could you please explain how that is better than the solutions proposed by Mike? Thanks, Ben On Thu, Jul 9, 2009 at 10:59 AM, Mr. Frank Cobia frank.co...@f2technology.com wrote: Just in case anybody cares or is looking at similar problems. I came up with an even better solution. I just programmatically remove the unused attributes from the model at application startup. Thanks, Frank On Jul 8, 2009, at 4:37 PM, Mr. Frank Cobia wrote: Sorry. I meant never use them in the report or the command line application. They are used by my web apps. Frank On Jul 8, 2009, at 4:34 PM, Mike Schrag wrote: Is there any advantage to creating a view and pointing my new entity to that view as opposed to just pointing the entity directly to the table and leaving out the fields I am not interested in? It seems like the view would just add overhead, but you may know some advantage to using it that I am not aware of. Oh, if you NEVER use the fields, then by all means, just remove them from your entity definition (assuming they're not NOT NULL fields). ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/frank.cobia%40f2technology.com This email sent to frank.co...@f2technology.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/frank.cobia%40f2technology.com This email sent to frank.co...@f2technology.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/bchew%40smarthealth.com This email sent to bc...@smarthealth.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Fetch Optimization Advice
Just in case anybody cares or is looking at similar problems. I came up with an even better solution. I just programmatically remove the unused attributes from the model at application startup. Thanks, Frank On Jul 8, 2009, at 4:37 PM, Mr. Frank Cobia wrote: Sorry. I meant never use them in the report or the command line application. They are used by my web apps. Frank On Jul 8, 2009, at 4:34 PM, Mike Schrag wrote: Is there any advantage to creating a view and pointing my new entity to that view as opposed to just pointing the entity directly to the table and leaving out the fields I am not interested in? It seems like the view would just add overhead, but you may know some advantage to using it that I am not aware of. Oh, if you NEVER use the fields, then by all means, just remove them from your entity definition (assuming they're not NOT NULL fields). ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/frank.cobia%40f2technology.com This email sent to frank.co...@f2technology.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/frank.cobia%40f2technology.com This email sent to frank.co...@f2technology.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Fetch Optimization Advice
I have created a WebObjects command line app (uses ERXMainRunner) that generates a report for each user (40,000+) on my system and it is running too slow. I have come up with two ways to optimize it and I was wondering which you thought is better. The report uses a table (TableA) that has a lot of fields, most of which are not relevant to the report. It also has a Text field which could be rather large. One option is to create TableAReport that points to the same database table, but only include the few (small) fields that I need for the report. I know the downside to this is that WebObjects can get confused when there are two entities pointing to the same data, but this command line app is only run once a day, it is read only and (hopefully) only runs for a few minutes. The other is to go to raw row fetches, but I lose some of the nice EOF stuff like traversing relationships. (TableA has two relationship I use). Any Advice? Thanks, Frank ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Fetch Optimization Advice
speaking without knowledge of any specifics of your app, it's generally number of roundtrips that is the big performance killer rather than the number of columns that come back, unless you have an especially large number of columns that have especially large data ... if it's #2, and this is just reporting, i would recommend making some custom views that only pull back what you need and make new eo's that sit on top of those views. my first thought is that you're probably not prefetching effectively, but that's a completely naive comment -- just what screws most people for performance. Turn on sql debug and do you see a ton of faults of to-one's and to-many's? if so, those are prime candidates for ERXBatchFetchUtilities to prefetch them in bulk. if the fetches you see are appropriately faulting, only THEN would i start to look at other things like column counts and raw rows. ms On Jul 8, 2009, at 4:19 PM, Mr. Frank Cobia wrote: I have created a WebObjects command line app (uses ERXMainRunner) that generates a report for each user (40,000+) on my system and it is running too slow. I have come up with two ways to optimize it and I was wondering which you thought is better. The report uses a table (TableA) that has a lot of fields, most of which are not relevant to the report. It also has a Text field which could be rather large. One option is to create TableAReport that points to the same database table, but only include the few (small) fields that I need for the report. I know the downside to this is that WebObjects can get confused when there are two entities pointing to the same data, but this command line app is only run once a day, it is read only and (hopefully) only runs for a few minutes. The other is to go to raw row fetches, but I lose some of the nice EOF stuff like traversing relationships. (TableA has two relationship I use). Any Advice? Thanks, Frank ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/mschrag%40mdimension.com This email sent to msch...@mdimension.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Fetch Optimization Advice
There are definitely a lot of fetches since the reports are custom to each user and there are more than 40,000 users, but the fetch does indeed pull a lot of rows and each row has a large text description that I am not using. I am using prefetching for the two relationships I use, but I am using the prefetching on ERXFetchSpecification. I have not looked at ERXBatchFetchUtilities. I will take a look at that. Is there any advantage to creating a view and pointing my new entity to that view as opposed to just pointing the entity directly to the table and leaving out the fields I am not interested in? It seems like the view would just add overhead, but you may know some advantage to using it that I am not aware of. Thanks, Frank On Jul 8, 2009, at 4:24 PM, Mike Schrag wrote: speaking without knowledge of any specifics of your app, it's generally number of roundtrips that is the big performance killer rather than the number of columns that come back, unless you have an especially large number of columns that have especially large data ... if it's #2, and this is just reporting, i would recommend making some custom views that only pull back what you need and make new eo's that sit on top of those views. my first thought is that you're probably not prefetching effectively, but that's a completely naive comment -- just what screws most people for performance. Turn on sql debug and do you see a ton of faults of to-one's and to- many's? if so, those are prime candidates for ERXBatchFetchUtilities to prefetch them in bulk. if the fetches you see are appropriately faulting, only THEN would i start to look at other things like column counts and raw rows. ms On Jul 8, 2009, at 4:19 PM, Mr. Frank Cobia wrote: I have created a WebObjects command line app (uses ERXMainRunner) that generates a report for each user (40,000+) on my system and it is running too slow. I have come up with two ways to optimize it and I was wondering which you thought is better. The report uses a table (TableA) that has a lot of fields, most of which are not relevant to the report. It also has a Text field which could be rather large. One option is to create TableAReport that points to the same database table, but only include the few (small) fields that I need for the report. I know the downside to this is that WebObjects can get confused when there are two entities pointing to the same data, but this command line app is only run once a day, it is read only and (hopefully) only runs for a few minutes. The other is to go to raw row fetches, but I lose some of the nice EOF stuff like traversing relationships. (TableA has two relationship I use). Any Advice? Thanks, Frank ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/mschrag%40mdimension.com This email sent to msch...@mdimension.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/frank.cobia%40f2technology.com This email sent to frank.co...@f2technology.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Fetch Optimization Advice
Is there any advantage to creating a view and pointing my new entity to that view as opposed to just pointing the entity directly to the table and leaving out the fields I am not interested in? It seems like the view would just add overhead, but you may know some advantage to using it that I am not aware of. Oh, if you NEVER use the fields, then by all means, just remove them from your entity definition (assuming they're not NOT NULL fields). ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Fetch Optimization Advice
Sorry. I meant never use them in the report or the command line application. They are used by my web apps. Frank On Jul 8, 2009, at 4:34 PM, Mike Schrag wrote: Is there any advantage to creating a view and pointing my new entity to that view as opposed to just pointing the entity directly to the table and leaving out the fields I am not interested in? It seems like the view would just add overhead, but you may know some advantage to using it that I am not aware of. Oh, if you NEVER use the fields, then by all means, just remove them from your entity definition (assuming they're not NOT NULL fields). ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/frank.cobia%40f2technology.com This email sent to frank.co...@f2technology.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Fetch Optimization Advice
Yeah you could setup a Report EOModel that has a different entity definition on the same table, too. The view approach potentially makes it a little easier to define fancier queries that you can just treat like a regular (non-editable) EO. ms On Jul 8, 2009, at 4:37 PM, Mr. Frank Cobia wrote: Sorry. I meant never use them in the report or the command line application. They are used by my web apps. Frank On Jul 8, 2009, at 4:34 PM, Mike Schrag wrote: Is there any advantage to creating a view and pointing my new entity to that view as opposed to just pointing the entity directly to the table and leaving out the fields I am not interested in? It seems like the view would just add overhead, but you may know some advantage to using it that I am not aware of. Oh, if you NEVER use the fields, then by all means, just remove them from your entity definition (assuming they're not NOT NULL fields). ms ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/frank.cobia%40f2technology.com This email sent to frank.co...@f2technology.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/mschrag%40mdimension.com This email sent to msch...@mdimension.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Fetch Optimization Advice
To paraphrase Mike, you really need to discover _why_ it is slow before you can start to address how to make it faster. If you just guess at what might be wrong, you are likely to expend a lot of effort to little avail. Knowledge is key in optimization. Chuck On Jul 8, 2009, at 1:24 PM, Mike Schrag wrote: speaking without knowledge of any specifics of your app, it's generally number of roundtrips that is the big performance killer rather than the number of columns that come back, unless you have an especially large number of columns that have especially large data ... if it's #2, and this is just reporting, i would recommend making some custom views that only pull back what you need and make new eo's that sit on top of those views. my first thought is that you're probably not prefetching effectively, but that's a completely naive comment -- just what screws most people for performance. Turn on sql debug and do you see a ton of faults of to-one's and to- many's? if so, those are prime candidates for ERXBatchFetchUtilities to prefetch them in bulk. if the fetches you see are appropriately faulting, only THEN would i start to look at other things like column counts and raw rows. ms On Jul 8, 2009, at 4:19 PM, Mr. Frank Cobia wrote: I have created a WebObjects command line app (uses ERXMainRunner) that generates a report for each user (40,000+) on my system and it is running too slow. I have come up with two ways to optimize it and I was wondering which you thought is better. The report uses a table (TableA) that has a lot of fields, most of which are not relevant to the report. It also has a Text field which could be rather large. One option is to create TableAReport that points to the same database table, but only include the few (small) fields that I need for the report. I know the downside to this is that WebObjects can get confused when there are two entities pointing to the same data, but this command line app is only run once a day, it is read only and (hopefully) only runs for a few minutes. The other is to go to raw row fetches, but I lose some of the nice EOF stuff like traversing relationships. (TableA has two relationship I use). Any Advice? Thanks, Frank ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/mschrag%40mdimension.com This email sent to msch...@mdimension.com ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/chill%40global-village.net This email sent to ch...@global-village.net -- Chuck Hill Senior Consultant / VP Development Practical WebObjects - for developers who want to increase their overall knowledge of WebObjects or who are trying to solve specific problems. http://www.global-village.net/products/practical_webobjects ___ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list (Webobjects-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
