The reason for this is because we may process the same Criteria object repeatedly across different modules, actions and views
On Tue, Mar 10, 2009 at 11:21 AM, Crafty_Shadow <[email protected]> wrote: > > I cannot understand why you prefer to cache model objects. > A much better option in my opinion is to cache the rendered view for > the site. > This way you skip any code that uses the model data to generate the > presentation, and is the fastest option. If you cannot cache the > entire action, cache partials or template fragments. > > Then you could make use of the object's save() method to clear the > cache. Even better, notify an event and let a dedicated class take > care of the cache clearing. There was a good blog post about this a > week or so ago: > > http://www.symfony-project.org/blog/2009/02/21/using-the-symfony-event-system > > On Mar 10, 8:54 am, Gareth McCumskey <[email protected]> wrote: > > I have actually come across a rather interesting way to use the memory > cache > > as specified in the book ( > http://www.symfony-project.org/book/1_1/18-Performance#chapter_18_sub... > > ) > > > > The way it is described to store data in the cache needs three things; a > > unique name that can be easily determined, the value related to that name > > and how long that value stays cached. In our application, our database > > records will very very rarely see any updates. Once records are inserted > > they are only ever retrieved making them ideal for caching. But we cannot > > store every database record into memory. We do have queries that run very > > frequently, however, and each Criteria object i sunique for each query > with > > the various values and so on. The problem is you cannot store an object, > > like the Criteria object that is built up before you run a Propel query > with > > doSelect or similar methods. So, if you serialize the Criteria object, > you > > have a string. But this is a very long string (one of the serialized > > Criteria objects I tested with was over 1000 characters long). But you > can > > convert it to a hash value. Naturally there is one problem with a hash; > the > > possibility that two different string would create the same hash. So > create > > two hashes and concatenate them, an SHA1 hash and an MD5 hash. This can > then > > be the name of the item you are storing in memory. Run the DB query and > you > > have the value of the query, which can then also be stored into the > cache. > > > > As a brief example for what I mean by overriding the doSelect method for > a > > model class: > > > > private static function doSelect($c) > > { > > $serialized_c = serialize($c); > > $md5_serialized = md5($serialized_c); > > $sha1_serialized = sha1($serialized_c); > > > > $cache = new sfAPCCache(); > > > > if ($cache->has('doSelect'.$md5_serialized.$sha1_serialized)) > > { > > $query_value = > > $cache->get('doSelect'.$md5_serialized.$sha1_serialized); > > } > > else > > { > > $query_value = parent::doSelect($c); > > $cache->set('doSelect'.$md5_serialized.$sha1_serialized, > $query_value, > > 60); > > } > > > > return $query_value; > > > > } > > > > Any comments would be great on this or any problems with doing this kind > of > > thing that I may not have seen please feel free to let me know > > > > On Tue, Mar 10, 2009 at 8:06 AM, Gareth McCumskey <[email protected] > >wrote: > > > > > Thanks for that. I have actually been looking at the function cache > amongst > > > others and there is a lot we can do there as our DB records, once > inserted, > > > are not likely to change. In fact if they do it means we are having a > > > problem as we store email data for a number of companies in them. > Therefore > > > function caching and even memory caching records as we extract them > from db > > > would probably help us a lot. It does mean more work code-wise and > isn't a > > > "quick-fix", so we plan to start looking at this once we hit Beta where > > > performance will be a major requirement. > > > > > The old system is faster simply because it follows no design pattern > except > > > procedural and that is where its speed lies. There are no ORM's, > classes or > > > anything like that, and SQL queries are sent straight through to the > > > database using handcoded, dynamic SQL queries as opposed to an ORM > generated > > > one and the resultsets are manipulated directly in each "view". In fact > > > there are only views, there is little seperation of business logic and > > > presentation. > > > > > The reason we need symfony for this new version is that we are going to > be > > > adding more advanced features that would "complicate" the product > beyond > > > what a procedural style would allow us to maintain. We are already > > > struggling to keep the older system maintained and enhanced for our > > > customers as it is. symfony, Propel and even Prototype with > scriptaculous > > > help alleviate these maintenance and extensibility issues. > > > > > On Mon, Mar 9, 2009 at 9:13 PM, Richtermeister <[email protected]> > wrote: > > > > >> Hi Gareth, > > > > >> after reading all this I feel your time is most likely best spent in > > >> smart caching, since it sounds like the DB is not your bottleneck. > > >> What's easy to overlook when working with symfony, is that compared to > > >> straight procedural "get data -> display data" scripts, rendering > > >> templates with hydrated objects is slower, albeit more flexible. So, > > >> if your previous site was coded in a bad way, it was probably using a > > >> lot of "view specific" code, so it's hard to compete with that on pure > > >> speed considerations. The only way to mitigate that is by using all > > >> forms of caching, and yes, this may include the function cache > > >> (although I don't like it much). However, the "higher up" you can > > >> cache, i.e. a complete action, the less you have to cache on the model > > >> level. > > > > >> Just my 2 cents. Good luck, > > >> Daniel > > > > >> On Mar 9, 8:42 am, Sumedh <[email protected]> wrote: > > >> > My 2 cents...slow query log in mysql should help a lot... > > > > >> > Please let us know your insights at the end of your exercise... :) > > > > >> > On Mar 9, 3:41 pm, Gareth McCumskey <[email protected]> wrote: > > > > >> > > I just tried using Propel 1.3 on our application and while I would > > >> love to > > >> > > continue using it (as it seemed to produce a little more > efficieny) we > > >> can't > > >> > > use it for now because the servers that the app will run on are > Centos > > >> 4 > > >> > > with PHP 5.1.x as its maximum version for now. The sysadmins here > say > > >> that > > >> > > to force an upgrade to 5.2.x would be a hard task as to retain > RedHat > > >> > > support it means they would need to upgrade to Centos 5. > > > > >> > > I am currently looking at the chapter about Optimising symfony and > the > > >> > > function cache seems to be something we cna consider doing in a > lot of > > >> our > > >> > > model calls from the action to help speed things up, especially > for > > >> model > > >> > > methods that access historical data (i.e. stuff dated in the past > that > > >> > > obviously wont change on subsequent calls) but these are > relatively > > >> large > > >> > > coding changes which we will probably only do during our beta > > >> development > > >> > > phase. > > > > >> > > I am still looking through more advise recieved from this post and > I > > >> have to > > >> > > thank everyone for their input. I honestly didn't expect this > response > > >> and > > >> > > it has been fantastic and very helpful. > > > > >> > > On Mon, Mar 9, 2009 at 12:49 AM, Crafty_Shadow < > [email protected]> > > >> wrote: > > > > >> > > > Symfony 1.1 came by default with Propel 1.2 > > >> > > > You can try upgrading to 1.3 (it isn't really a trivial task, > but it > > >> > > > shouldn't be a big problem) > > >> > > > There is thorough explanation on the symfony site how to do it: > > >> > > >http://www.symfony-project.org/cookbook/1_1/en/propel_13 > > >> > > > It should fare a measurable increase in performance. Also, a > site > > >> that > > >> > > > makes good use of cache should have caching for absolutely > > >> everything > > >> > > > not session-dependent. I find it hard to imagine a php app, no > > >> matter > > >> > > > how fast, that would run faster than symfony's cached output. > > > > >> > > > Alvaro: > > >> > > > Is your plugin based on Propel 1.3? > > >> > > > If you believe you have made significant improvements to Propel, > why > > >> > > > not suggest them for version 2.0, which is still under heavy > > >> > > > development? > > > > >> > > > On Mar 8, 4:33 pm, alvaro <[email protected]> wrote: > > >> > > > > At the company I developed a symfony plugin to optimize the > Propel > > >> > > > > queries and also the Propel hydrate method, improving even 5 > times > > >> > > > > query speed and also memory usage. > > > > >> > > > > The plugins supports joins and thanks to PHP features the > plugin > > >> > > > > returns Propel objects populated with custom AS columns. > > > > >> > > > > We are thinking on release it on the following weeks so stay > tuned > > >> :) > > > > >> > > > > Regards, > > > > >> > > > > Alvaro > > > > >> > > > > On Mar 8, 2009, at 10:20 PM, Gareth McCumskey wrote: > > > > >> > > > > > We have put numerous caching techniques into effect, from > Cache- > > >> > > > > > Expires headers to compression of static files like js and > html > > >> > > > > > files. Currently we use symfony 1.1 and Propel as the ORM. > We > > >> have > > >> > > > > > identified the bottleneck generally as being the application > > >> > > > > > processing after the db queries have run to extract the > data. > > > > >> > > > > > The entire point of my question was to get some info on > general > > >> tips > > >> > > > > > and tricks we can try out to see if anything helps or if > perhaps > > >> we > > >> > > > > > have missed any obvious issues that may actually be the > cause of > > >> the > > >> > > > > > slow performance we are getting. As it is I have gotten > quite a > > >> few > > >> > > > > > and look forward to getting into the office tomorrow to try > them > > >> > > > > > out. Anymore is greatly appreciated. > > > > >> > > > > > Of course I am looking through the code to see if there is > > >> anyway we > > >> > > > > > can streamline it on that end, but every little bit helps. > > > > >> > > > > > Gareth > > > > >> > > > > > On Sun, Mar 8, 2009 at 12:27 PM, Crafty_Shadow < > > >> [email protected]> > > >> > > > > > wrote: > > > > >> > > > > > Gareth, you didn't mention what version of symfony you were > > >> using, > > >> > > > > > also what ORM (if any). > > >> > > > > > The best course of optimization will depend on those. Also, > as > > >> already > > >> > > > > > mentioned, caching is your best friend. > > > > >> > > > > > On Mar 8, 9:43 am, Gareth McCumskey <[email protected]> > > >> wrote: > > >> > > > > > > Well, consider a single database table that looks > something > > >> like > > >> > > > > > this: > > > > >> > > > > > > From_address > > >> > > > > > > to_address (possibly multiple addresses comma-seperated) > > >> > > > > > > headers > > >> > > > > > > spam_report > > >> > > > > > > subject > > > > >> > > > > > > And we would have millions of those records in the > database. > > >> > > > > > Repeated > > >> > > > > > > entries, especially on to_address, means the data > > > > ... > > > > read more ยป > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en -~----------~----~----~----~------~----~------~--~---
