On 12/19/05 7:51 PM, Sam Vilain wrote:
> On Mon, 2005-12-19 at 17:30 -0500, John Siracusa wrote:
>>>> I think you misunderstand what the tests are doing.  They're meant to
>>>> highlight exactly the features that you're describing.  Fetching objects
>>>> and
>>>> some of their associated objects in other tables is a common task.  Many of
>>>> the tests revolve around this task (with various twists).  There are other,
>>>> earlier tests that measure the performance of single-row loads, and so on.
>>> Well, I still think it's timing of the wrong thing
>> What should it be timing?  The particular test you were looking at (search
>> products and categories and code names) represents a common, practical
>> task, and does in fact reward ORMs that can accomplish this task with as few
>> queries as possible.
> 
> OK, it's just been my experience that the common, practical tasks are
> only indirectly a performance problem - in that, if the simple task ends
> up being run 1,000 times in a transaction, it is obviously far too many
> - and so it should be easy to translate that into some code that does
> all the stuff it needs to up front.

I agree.

> if you have a good way of specifying that you want to prefetch the relations
> you will be following, then you only need three selects.  The total time may
> be longer or shorter, the the important thing is that the database is doing
> less work, and therefore the application will scale further.

Well, I could hard-code a benchmark that simply compares the number of
queries run, but that'd be pretty boring ;)  I'm not sure that's something
that can be benchmarked.  It was my expectation that the ORMs that used more
queries would be slower, and the tests that bench.pl runs bear this out in
both wall clock and CPU time.  Beyond that, I'm not sure what else could be
done with regards to benchmarking.

If you need to know how many queries an ORM will run for a given task, that
information should be present (either explicitly or by implication) in that
ORM's docs.  Worst case, there's always the source.

(Also, the need for such information will depend on the environment.  Fewer
queries is only "more scalable" when the db server is the bottleneck.  What
about situations where the db server is a monster but an app has to run on a
wimpy old server with an 800MHz CPU?  The real world is always odd :)
 
> With Tangram, you basically pass the same filter object to prefetch that
> you can use for select - this makes this kind of code very simple.

Great, I'll be sure to bug you when it comes time to write the Tangram
versions of those tests :)

> If you are using Benchmark, you might also want to read this
> thread ("Re: goto-& Speed"):
> 
>  http://xrl.us/i9zx (Link to hop.perl.plover.com)

Yeah, well, there's only so much that can be done to isolate performance
differences down to the last cycle.  I'm content to try keep the task and
the "infrastructure" (db server and tables, inflate/deflate objects, etc.)
the same.  Anything beyond that is out of my hands, and is likely of only
academic interest anyway.

-John




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Rose-db-object mailing list
Rose-db-object@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rose-db-object

Reply via email to