Martin Langhoff wrote:
With these hit and record numbers (esp if complex querying and sorting is needed) it's propably more efficient doing separate tables and optimizing table and query structures and just live with more complex replication than extending midgard. Though it depends on many factors.
For large amount of data (Esp when used for sorting) avoid using parameters, put all data needed for sorting serialized to, for example, article->abstract (which is returned by mgd_list_topic_articles()
I believe that writing scalable applications that rely on this technique to run the equivalent of complex SQL queries is... between impossible and not worth it. But maybe my view is skewed by the types of projects I work on, which tend to imply several million hits per month on complex SQL queries, over a tens of thousands of records. We focus a lot on optimizing our queries, caching and pre-processing as much as possible.
Modifying the schema does not fit my "single file" replication ideal since it requires distributing the modified schema and (propably, haven't event tested it) some way of creating the new structures to the target db for the first time.This way there is no need to rely on external files or databases and install/upgrade can be done using single repligard package.
A simple makefile, or a couple of scripts are an alternative. You can't package everything in repligard (although Andrey proposed an interesting alternative that might mean you can), but you can always package it somehow. I am not a purist ;)
Single file, single command, db connections and abstraction is handled transparently by midgard (and since midgard hides the connection, I can't use the same connection, so I need a mysql username and password stored somewhere and open a new connection when manipulating mysql straight). and I am a purist ;) (sort of).So far I have not come across anything that would absolutely require separate DB structure for efficiency reasons and I have opted doing everything by extending midgard, due to simpler replication.
What are the -- presumably important -- benefits of replication that prompt you to add significant complexity and limit the scalability of your application, just to avoid using a simple makefile or shell script?
Some day I'll propably come across a case where must for efficiency reasons use database directly (since there is no way to organize the data to parent midgard objects in sensible way).
Untill that day, as long as the object structure is sensible and some data caching is done then scalability should not be any more of a problem as with straight db queries. The added complexity (not that there is too much of it) is hidden from the main application by the wrapper classes.
>I don't want to start a flamewar. It is a matter of trade-offs,
Of course not, and it's always about trade-offs, I like to keep everything I can contained in Midgard and thus am willing to take (in these cases) insignificant perfomance hit to that end. Should the perfomance prove to be significant (during all these years I have learned to be pretty good in judging what kind of performance hit I'm taking from certain approach so generally I'm not wasting time on a bad design) then the approach must change.
and a subtle one at that, so I am ok with anything you choose to do. I would like to understand the reasoning behind it. There is always the strong chance I am missing a few clues, and it might be a good practice to contemplate for my own projects.The key to making efficient midgard applications using objects that don't directly conform to any midgard object is to minimize the number of queries you need to make (by placing extra data serialized to fields returned with list functions and using list_all functions when whole trees of data are needed, the tree can be reconstucted from the returned data by little array juggling) and choosing the base object such that it allows native filtering/sorting for the most important filter/sort requirement you have.
I'm kind of bad at explaining this but it works for me.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
