Hi Jarno, just so I'm clear on this, am I correct in saying
routing config cache = cache/backend/dev/config/config_routing.yml.php
routing data cache = cache/backend/dev/config/routing/
symfony.routing.data.cache
The config cache runs fine for us - using APC (after upping
apc.max_file_size - it's 1.9MB) loads in ~0.06 secs
The impact for us too is the data cache - if we disable it, we don't
use RAM, avoid the un/serialize() problem, but performance impact is
high when it has to match a number of routes.
If we enable it, we will either get a v. large
symfony.routing.data.cache file or if we use
lookup_cache_dedicated_keys we will use up ram in apc due to the per-
url caching of all the routes on the page.
Interested on the thoughts of caching in real PHP file to take
advantage of APC opcode cache, but I'm still concerned this file will
be v. large.
Thanks,
Andy
On Sep 16, 2:23 pm, Jarno Rantanen <jarno.ranta...@mederra.fi> wrote:
> Hello,
>
> Fabian, you are right, using sfNoCache will still make calls to
> serialize() and unserialize(), and this is unnecessary overhead. But
> since the cache adapter won't actually *store* anything, the amount of
> data being un-/serialized won't get to grow to proportions in which it'd
> start eating resources like crazy. This is why it solved the problem.
>
> arrigby, the routing config cache is separate from the routing data
> cache. Our issue was exclusively with the latter, and our routing
> config cache is still active.
>
> Initially we were thinking we'd have to implement some other caching
> strategy ourselves, but since the performance gain won't be stellar we
> are considering simply leaving the routing data cache off (with the
> cache: null Fabian pointed out).
>
> I'm assuming that due to the way the cache and routing systems work it
> wouldn't be possible to use a var_export() and include() -combination
> for storing the routing data cache..? It would be way more efficient
> resource-wise, but would obviously only work with local files (ie. no
> APC/memcached/etc), and could not store objects (except via some
> __set_state() magic method).
>
> - Jarno
>
> On Wed, 2009-09-16 at 06:04 -0700, arri...@gmail.com wrote:
> > Hi,
>
> > Turning off the cache isn't really an option for us due to our
> > (relatively) large number of routes and the number of calls to them.
> > Our config_routing.yml.php contains 615 routes. For 56 links, we'd be
> > back to 1.3 secs just for the routing to work out the links.
>
> > re: lookup_cache_dedicated_keys in #5811 - this is an option, however
> > I think it suffers from the same problem i.e. a cache per instance of
> > url. Please correct me if i'm wrong. With APC cache i think we will
> > start getting short of ram pretty quick.
>
> > On Sep 16, 1:47 pm, Fabian Lange <fabian.la...@symfony-project.com>
> > wrote:
> > > Hi,
> > > not an complete answer, but try "cache: null" instead of the sfNoCache,
> > > because null will prevent any serialisation calls like here:
>
> > > if (!is_null($this->cache))
> > > {
> > > $this->cache->set('symfony.routing.configuration',
> > > serialize($this->routes));
> > > }
>
> > > sfNoCache will not prevent these lines to run. cache==null will
>
> > > Fabian
>
> > > On Wed, Sep 16, 2009 at 2:10 PM, Jarno Rantanen
> > > <jarno.ranta...@mederra.fi>wrote:
>
> > > > Hi,
>
> > > > We recently ran to a similar issue after upgrading from 1.2.4 to 1.2.8
> > > > on a
> > > > site with busy forums and hence quite a lot of different URLs; the size
> > > > of
> > > > the routing data cache became huge (the file was around 30 MB).
> > > > Needless to
> > > > say, unserialize()ing a string of 30 MB or so is a bit inefficient.
> > > > Combined with some obscure bug in PHP (5.2.8) which allowed this
> > > > operation
> > > > to bypass the memory_limit-directive, our server quickly ate up its 16
> > > > GB or
> > > > RAM and started swapping. Down we went.
>
> > > > It was particularly troublesome to identify what caused the huge memory
> > > > consumption, since because of the bug we didn't get any "memory
> > > > exhausted"
> > > > PHP fatal errors in our logs. When we finally did, the situation was
> > > > immediately resolved by turning off the routing data cache (substituting
> > > > sfFileCache with sfNoCache in factories.yml). The performance penalty
> > > > of
> > > > not having this cache on seems negligible.
>
> > > > I found some related discussion in:
>
> > > >http://www.symfony-project.org/blog/2009/04/03/lazy-routing-deseriali...
> > > >http://trac.symfony-project.org/ticket/6308and
> > > >http://trac.symfony-project.org/ticket/5811
>
> > > > Especially depressing was the comment in #5811 "cause sites encountering
> > > > the issue have people that know how to fix it". :) I realize producing
> > > > a
> > > > one-size-fits-all solution for this is nontrivial, but this really took
> > > > us
> > > > by surprise and caused quite a headache. Since, according to the
> > > > benchmarks
> > > > by "rs" in #5811, the single-key caching strategy doesn't really serve
> > > > any
> > > > number of URLs well, should the default strategy be somehow changed in a
> > > > future release?
>
> > > > My 0.02 €,
>
> > > > - Jarno
>
> > > > On Wed, 2009-09-16 at 03:24 -0700, arri...@gmail.com wrote:
>
> > > > Hi.
>
> > > > Noticed the routing has had performance changes in 1.3 which is great.
> > > > The cache is now recommended to be off but we have a severe impact on
> > > > our system if we do that.
>
> > > > Would like to open a discussion re: possible improvement if this is
> > > > ok.
>
> > > > We have lots of routes - our app is built around admin generator and
> > > > therefore sfPropelRouteCollection. The routing performance changes in
> > > > 1.3 appear to be when loading routes, but the performance penalty for
> > > > our app is when matching a route i.e. url_for().
>
> > > > e.g. below is output from xhprof
>
> > > > Current Function
> > > > sfPatternRouting::getRouteThatMatchesParameters 56 0.1% 1,360,102
> > > > 82.9%
> > > > Exclusive Metrics for Current Function 173,620 12.8%
>
> > > > Parent function
> > > > sfPatternRouting::generate 56 100.0% 1,360,102 100.0%
>
> > > > Child functions
> > > > sfObjectRoute::matchesParameters 22,455 48.5% 1,115,255 82.0%
> > > > sfRoute::setDefaultParameters 23,148 50.0% 50,048 3.7%
> > > > sfRoute::matchesParameters 693 1.5% 21,179 1.6%
>
> > > > Bit difficult to read sorry, but basically
> > > > sfPatternRouting::getRouteThatMatchesParameters() is called 56 times
> > > > (for the 56 links we have) but it takes 1.3 seconds to execute this
> > > > code. This is on athlon x2 4600+.
>
> > > > The problem seems to be that sfObjectRoute::matchesParameters() is
> > > > called 22,455 times taking 1.1 seconds of the total time.
>
> > > > OK, so if we enable the file cache we get good speeds again, however
> > > > the problem is the amount of data in this cache. Because we have so
> > > > many routes and are using admin gen, we hit this problem:
>
> > > > route is
> > > > supplier:
> > > > class: sfPropelRouteCollection
> > > > options:
> > > > model: Supplier
> > > > module: supplier
> > > > prefix_path: supplier
> > > > with_wildcard_routes: true
>
> > > > if we have 1000 suppliers, we get an entry in the routing cache for
> > > > each of these urls. Also, in the layout of the pages we have menu
> > > > links. e.g. menu/admin, menu/stock. (there's approx 100 of these)
>
> > > > The problem is, because the url is supplier/1/edit, supplier/2/edit,
> > > > supplier/3/edit the context when generating the cache key is different
> > > > because the uri changes. This means for every supplier, we also have
> > > > all the menu links stored in the cache for that instance of supplier.
> > > > So in the cache we have 1,000*100 = 100,000 entries in the cache - and
> > > > that is for one module (we currently have 64 admin generated modules +
> > > > 30 odd non-admin gen). e.g. our routing cache contains this:
>
> > > > ["parse_/supplier/1/edit_2ed71130b65896bc589714492c3cc85c"]=>
> > > > array(3) {
> > > > ...
>
> > > > ["parse_/supplier/2/edit_8179119534b860346bafa3dedb2bad53"]=>
> > > > array(3) {
> > > > ...
> > > > ["parse_/supplier/3/edit_118c94f897cfc7ae68a124e8dd692f4a"]=>
> > > > array(3) {
>
> > > > and
>
> > > > ["generate_logout_e4ed1a882c44f3cf31e7ed18e8686b65_516891fa3258fae54e8eaed2b2727dcb"]
> > > > =>
> > > > string(7) "/logout"
> > > > ...
> > > > ["generate_logout_e4ed1a882c44f3cf31e7ed18e8686b65_54e6d2c24d12203af92e3f785f1a567d"]
> > > > =>
> > > > string(7) "/logout"
> > > > ...
> > > > ["generate_logout_e4ed1a882c44f3cf31e7ed18e8686b65_2ed71130b65896bc589714492c3cc85c"]
> > > > =>
> > > > string(7) "/logout"
>
> > > > i.e. a application "logout" link for each of the 3 supplier pages I
> > > > have just visited - the last md5 is different for each one (different
> > > > context)
>
> > > > Other concern is the size of the data will get so large that the
> > > > overhead of read/unserialize()/serialize()/write() will cause a large
> > > > performance penalty.
>
> > > > Specifying each route by name is an option, though not one I wish to
> > > > go down if possible - plus I don't think it will correct the 'size of
> > > > data' issue.
>
> > > > Possible solution (though I do not understand the routing enough) but
> > > > is it possible to not generate the cache key on the parameter values,
> > > > but just on the parameter keys? so, md5(serialize(array_keys
> > > > ($params))) instead. Or can the value of a param determine a different
> > > > route? Similarly, is it possible for the context to not include the
> > > > uri, or use the internal uri instead of real one e.g. supplier/:id/
> > > > edit - that way the context would be the same regardless of id.
>
> > > > Hope this makes sense. I'm sure it's not an easy one to resolve...
>
> > > > Thanks,
> > > > Andy.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"symfony developers" group.
To post to this group, send email to symfony-devs@googlegroups.com
To unsubscribe from this group, send email to
symfony-devs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-devs?hl=en
-~----------~----~----~----~------~----~------~--~---
