I'm with Matt on this. One thing we have done with one of our apps that
needs to build reports for example, is we created a summary table to reduce
the amount of data to extract from the database and therefore reduce the
amount of data to process as well. If you can think of ways to aggregate
many smaller data points into summarised datasets you should do so as it can
substantially reduce your server workload.

To extrapolate on my example, we had data on individual customer emails, and
customers would want to pull statistical reports on this for over a months
worth of data with a daily breakdown. Because the reports don't need to be
specific to the second, we created scripts that create summary/aggregated
data in another table that gives a total per email address per day. Now if
we want details on a specific users email stats for a month we pull only 30
records instead of getting data on every email that guy ever received.

It seems like a lot of work now to get something like that running, but over
the long term it helps keep your load on your server low, queries are
faster, and therefore the longevity of your application is vastly improved.

On Thu, May 5, 2011 at 1:22 AM, Matt Robinson <m...@lazycat.org> wrote:

> On 4 May 2011, at 23:57, Michał Piotrowski wrote:
>
> > 2011/5/4 Marco Pivetta <ocram...@gmail.com>:
> >> SQL is way faster than PHP processing. I would really go for a filtered
> >> join. That's what SQL is there for :)
> >> Just be sure to filter by index and to return a limited resultset.
> >>
> >> Sequential scan in PHP has to be ABSOLUTELY avoided :)
> >
> > I'm not sure how to do this if I need a multiple dynamically added
> conditions.
>
> SQL (certainly in mySQL) supports conditionals in queries, so if the data
> you use to make the choice is in the database, you can do it all in the
> query.
>
> Another way to approach your problem is to have your web app trigger events
> when certain actions occur - this way you're approaching issues like "have
> already been purchased by at least one of his friends" from the other end.
> When someone buys a product, add it to each friend's recommendation list.
> Then your look-up query is much more simple. You can even queue these events
> up and process them later, since it's usually not vital that such
> information is "live".
>
> As an aside: typically very complex queries like this are done by
> background processes only periodically (e.g. every few hours) and stored in
> look-up tables or similar. Particularly as you start getting many users,
> complex queries quickly become a bottleneck, so you produce them offline
> beforehand and store them in a way that's simple and quick to fetch by your
> site's PHP scripts (in a simple cache table or object store). You never want
> a simple request page to be doing so much heavy lifting that it'll slow down
> and prevent you scaling later. If the heavy processing stuff is split off
> into a separate process, you can also move that process to another server
> later.
>
>
> -- Matt
>
> --
> If you want to report a vulnerability issue on symfony, please send it to
> security at symfony-project.com
>
> You received this message because you are subscribed to the Google
> Groups "symfony users" group.
> To post to this group, send email to symfony-users@googlegroups.com
> To unsubscribe from this group, send email to
> symfony-users+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/symfony-users?hl=en
>



-- 
Gareth McCumskey
http://garethmccumskey.blogspot.com
twitter: @garethmcc
identi.ca: @garethmcc

-- 
If you want to report a vulnerability issue on symfony, please send it to 
security at symfony-project.com

You received this message because you are subscribed to the Google
Groups "symfony users" group.
To post to this group, send email to symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en

Reply via email to