Re: [Radiant] Slow page load times
Anton, I'm not denying that the site could be optimized significantly,
but those optimizations are orders of magnitude less important than
fixing this problem I'm having with 30s refreshes. Until I resolve
that, all these other techniques are superfluous. As they say, work on
the bottlenecks. Besides, these images are being cached just fine by
repeat visits. But the caching problem we're having is making the site
unusable. Your suggestions are right on the money; I have been
studying these techniques as well, but I'd like to deal with this
larger problem first. The fact remains that two refreshes in a row
both get 200 responses, so rack-cache is not working or else something
in radiant is screwing things up. Thanks for your suggestions.
On Wed, Jan 26, 2011 at 9:21 PM, Anton J Aylward
wrote:
> Carl Youngblood said the following on 01/25/2011 05:01 PM:
>
>> to respond to Anton's previous comments, firebug shows that nearly all
>> the delay is spent waiting for the server.
>
> Of course it is, but that isn't what I meant.
>
> You have a lot of images of your page.
> You have javascript and CSS.
> You also have in-line Javascript and in-line CSS which is a drag on
> performance!
>
> You have images that sequence. That big block run under javascript.
> Why not combine those images into one and use a CSS sprite? Yes, a
> bigger download but only ONE download. And there are various ways you
> can optimise the image so that its size is actually less than the size
> of the sum of the images.
>
>
> Your page layout is such that your browser cannot render ANYTHING until
> it has all arrived from the server. If it has to scale images or wait
> for (all) the images to arrive to work out the layout then things are
> even worse!
>
> Of your "30 seconds", 27 of them are the latency of downloading those
> HTML and embedded CSS images. You have over 60 images on that page.
>
> If they are fetched sequentially, waiting for one to complete before
> fetching the next, you have a worst-case situation. You should set up
> your browser to have multiple persistent parallel connections and fetch
> them in parallel. IE8 defualts to only 2 parallel connections, which
> for the type of page you have is crippling.
>
> Explaiantion at
> http://en.wikipedia.org/wiki/HTTP_persistent_connection
>
> There is also a differnece in how http/1.0 and http/1.1 handle this.
> I see you are using 1.0.
>
> See
> https://addons.mozilla.org/en-US/firefox/addon/extra-fasterfoxx/
> whcih tweaks these settings:
> user_pref("network.http.max-connections", 40);
> user_pref("network.http.max-connections-per-server", 16);
> user_pref("network.http.max-persistent-connections-per-proxy", 12);
> user_pref("network.http.max-persistent-connections-per-server", 6);
> user_pref("nglayout.initialpaint.delay", 0);
>
> However this only makes sense if your Apapche is configured with http
> 'keep-alive'
> http://httpd.apache.org/docs/1.3/keepalive.html
>
> See also
> http://www.technewsworld.com/story/it-management/64924.html?wlc=1295996776
>
>
> These pages on page design and layout address my other points
> http://developer.yahoo.com/performance/rules.html
> As it says:
>
> Reducing the number of HTTP requests in your page is the place to start.
> This is the most important guideline for improving performance for first
> time visitors
>
> You have a killer number of requests on your front page!
>
>
> Layout of the elements of the page can also be critical
> Take a look at the Cuzillion design tool to optimize your layout.
> http://stevesouders.com/cuzillion/
>
>
>
>
>
> Looking at the soruce of your pages I see you are very ehavy on your DOM
> elements. You keep using constructs like
>
>
>
>
> These could be merged in the CSS, making the CSS smaller (cna why not
> compress it?) and fewer DOM elelment for the browser to deal with.
>
> The number of DOM elements is easy to test, just type in Firebug's console:
> document.getElementsByTagName('*').length
>
>
> --
> Wherever you see a successful business, someone once made a courageous
> decision.
> --Peter F. Drucker
>
Re: [Radiant] Slow page load times
Carl Youngblood said the following on 01/25/2011 05:01 PM:
> to respond to Anton's previous comments, firebug shows that nearly all
> the delay is spent waiting for the server.
Of course it is, but that isn't what I meant.
You have a lot of images of your page.
You have javascript and CSS.
You also have in-line Javascript and in-line CSS which is a drag on
performance!
You have images that sequence. That big block run under javascript.
Why not combine those images into one and use a CSS sprite? Yes, a
bigger download but only ONE download. And there are various ways you
can optimise the image so that its size is actually less than the size
of the sum of the images.
Your page layout is such that your browser cannot render ANYTHING until
it has all arrived from the server. If it has to scale images or wait
for (all) the images to arrive to work out the layout then things are
even worse!
Of your "30 seconds", 27 of them are the latency of downloading those
HTML and embedded CSS images. You have over 60 images on that page.
If they are fetched sequentially, waiting for one to complete before
fetching the next, you have a worst-case situation. You should set up
your browser to have multiple persistent parallel connections and fetch
them in parallel. IE8 defualts to only 2 parallel connections, which
for the type of page you have is crippling.
Explaiantion at
http://en.wikipedia.org/wiki/HTTP_persistent_connection
There is also a differnece in how http/1.0 and http/1.1 handle this.
I see you are using 1.0.
See
https://addons.mozilla.org/en-US/firefox/addon/extra-fasterfoxx/
whcih tweaks these settings:
user_pref("network.http.max-connections", 40);
user_pref("network.http.max-connections-per-server", 16);
user_pref("network.http.max-persistent-connections-per-proxy", 12);
user_pref("network.http.max-persistent-connections-per-server", 6);
user_pref("nglayout.initialpaint.delay", 0);
However this only makes sense if your Apapche is configured with http
'keep-alive'
http://httpd.apache.org/docs/1.3/keepalive.html
See also
http://www.technewsworld.com/story/it-management/64924.html?wlc=1295996776
These pages on page design and layout address my other points
http://developer.yahoo.com/performance/rules.html
As it says:
Reducing the number of HTTP requests in your page is the place to start.
This is the most important guideline for improving performance for first
time visitors
You have a killer number of requests on your front page!
Layout of the elements of the page can also be critical
Take a look at the Cuzillion design tool to optimize your layout.
http://stevesouders.com/cuzillion/
Looking at the soruce of your pages I see you are very ehavy on your DOM
elements. You keep using constructs like
These could be merged in the CSS, making the CSS smaller (cna why not
compress it?) and fewer DOM elelment for the browser to deal with.
The number of DOM elements is easy to test, just type in Firebug's console:
document.getElementsByTagName('*').length
--
Wherever you see a successful business, someone once made a courageous
decision.
--Peter F. Drucker
Re: [Radiant] Slow page load times
Hi,
I too do experience slow page loads. especially with the sns extension.
I updated the sns extension to allow for cache busting css and js urls
and for a start of action caching the parsed textasset.
https://github.com/iteh/radiant-sns-extension
slightly modified
* to add a cache busting timestamp allowing for really far future expires
headers
* update the layout to re-render pages so this gets set if you modify a
stylesheet or javascript file
* respect the Rails asset_host setting
* Rails cache the render of an object and expire it on changing the asset
did send a pull request to:
https://github.com/radiant/radiant-sns-extension
I think it would be great for radiant to benefit from some caching on the
action level, not just Rack::Cache.
cu edi
On 25.01.2011, at 23:01, Carl Youngblood wrote:
> On Mon, Jan 24, 2011 at 2:27 PM, William Ross wrote:
>> 4. Now you can be fairly sure that the problem is in a radiant extension.
>> You will need to poke around in the console (in production mode on the live
>> site) to find out more. Here are some values you could check:
>> SiteController.cache_timeout (should be a duration, most likely 5.minutes)
>> Radiant::Config['dev.host'] (make sure this isn't the same as your
>> production host)
>> Page.find_by_parent_id(nil).cache? (that's the home page. cached? should be
>> true)
>> That last will most likely return false, and you will need to track down
>> why.
>> best,
>> will
>
> THanks for all these tips Will. I am indeed getting 200 responses and
> to respond to Anton's previous comments, firebug shows that nearly all
> the delay is spent waiting for the server. THere's no funny business
> in the apache config, it's quite standard.
> SiteController.cache_timeout is 300 seconds,
> Radiant::Config['dev.host'] is nil. The only config settings that show
> up when I call
>
> Radiant::Config.find(:all).each { |c| p c.key }
>
> Are:
>
> "admin.title"
> "admin.subtitle"
> "defaults.page.parts"
> "defaults.page.status"
> "defaults.page.filter"
> "session_timeout"
> "assets.additional_thumbnails"
> "assets.display_size"
> "assets.content_types"
> "assets.max_asset_size"
> "assets.skip_filetype_validation"
> "roles.settings"
> "SnS.stylesheet_directory"
> "SnS.javascript_directory"
> "SnS.cache_timeout"
> "SnS.javascript_mime_type"
>
> SnS.cache_timeout is 600.
>
> Page.find_by_parent_id(nil).cache? is true.
>
> So it looks like most of my settings are normal so far. Any more
> ideas? Thanks again for your troubleshooting tips.
>
> Cheers,
> Carl
--
DI Edmund Haselwanter, edm...@haselwanter.com, http://edmund.haselwanter.com/
http://www.iteh.at | http://facebook.com/iTeh.solutions |
http://at.linkedin.com/in/haselwanteredmund
Re: [Radiant] Slow page load times
On Mon, Jan 24, 2011 at 2:27 PM, William Ross wrote:
> 4. Now you can be fairly sure that the problem is in a radiant extension.
> You will need to poke around in the console (in production mode on the live
> site) to find out more. Here are some values you could check:
> SiteController.cache_timeout (should be a duration, most likely 5.minutes)
> Radiant::Config['dev.host'] (make sure this isn't the same as your
> production host)
> Page.find_by_parent_id(nil).cache? (that's the home page. cached? should be
> true)
> That last will most likely return false, and you will need to track down
> why.
> best,
> will
THanks for all these tips Will. I am indeed getting 200 responses and
to respond to Anton's previous comments, firebug shows that nearly all
the delay is spent waiting for the server. THere's no funny business
in the apache config, it's quite standard.
SiteController.cache_timeout is 300 seconds,
Radiant::Config['dev.host'] is nil. The only config settings that show
up when I call
Radiant::Config.find(:all).each { |c| p c.key }
Are:
"admin.title"
"admin.subtitle"
"defaults.page.parts"
"defaults.page.status"
"defaults.page.filter"
"session_timeout"
"assets.additional_thumbnails"
"assets.display_size"
"assets.content_types"
"assets.max_asset_size"
"assets.skip_filetype_validation"
"roles.settings"
"SnS.stylesheet_directory"
"SnS.javascript_directory"
"SnS.cache_timeout"
"SnS.javascript_mime_type"
SnS.cache_timeout is 600.
Page.find_by_parent_id(nil).cache? is true.
So it looks like most of my settings are normal so far. Any more
ideas? Thanks again for your troubleshooting tips.
Cheers,
Carl
Re: [Radiant] Slow page load times
On 24 Jan 2011, at 07:29, Carl Youngblood wrote: > On Sun, Jan 23, 2011 at 8:50 PM, William Ross wrote: > From a brief look it seems that the rss-reader extension disables all page > caching. It does this in a rather inadvisable way (by amending the Page class > itself rather than defining a specialist subclass) but since your site is > primarily an RSS-aggregator, fixing that wouldn't help much. > > The extension is quite old and I expect the decision dates back to the old > radiant cache: with Rack::Cache in front of the whole thing it doesn't make > sense any more. Simply commenting out these lines in lib/rss_reader.rb: > > > def cache? > > > false > > > end > > > > should make a big difference. > > William, I just thought of something. We're running Radiant 0.8.0. Would this > affect your advice at all? No, that shouldn't make any difference. But the advice is more diagnostic than curative, if you see what I mean. There are lots of factors that could be slowing you down and that was only one. Here are some more diagnostics: 1. Make sure you don't force-reload the page. On a normal radiant installation I find that a hard refresh in firefox will always pass through the cache. It shouldn't: the cache is configured to ignore reload requests from the client, but for some reason they still work. Hitting refresh without any modifier keys will ignore the browser cache but should still allow you to test the server cache. 2. Reload the page (without forcing it) twice in quick succession and on the second attempt examine the response headers. The easy way to do that is to install Firebug and watch the 'Net' panel. If the cache is working you should get '304 Not Modified' for everything. If you get '200 OK' responses for the radiant page in that test then your site is not cached. 3. Have a quick look at the webserver configuration. If there are proxies or redirects involved, for example, it's easy to mangle cache headers. 4. Now you can be fairly sure that the problem is in a radiant extension. You will need to poke around in the console (in production mode on the live site) to find out more. Here are some values you could check: SiteController.cache_timeout(should be a duration, most likely 5.minutes) Radiant::Config['dev.host'] (make sure this isn't the same as your production host) Page.find_by_parent_id(nil).cache? (that's the home page. cached? should be true) That last will most likely return false, and you will need to track down why. best, will
Re: [Radiant] Slow page load times
Carl Youngblood said the following on 01/24/2011 01:28 AM: > Thanks for the advice Anton. Pages are coming up fast when they are > accessed with a referer header and when they have recently been > generated by Radiant. But they are taking 30s when I type the URL in > directly and when I click refresh in a browser. This is not what I observe when I visit your site. I find it very responsive. It is for this reason I suggested the browser-side tools for analysis. When I was developing my own sites I found that the biggest hit I had was that I had broken my CSS up "logically" to include parts that dealt with colour, fonts, the sidebar, the footer. I had none of them, all in the database. When I merged them into one external file the load on radiant decreased dramatically - Apache was now serving the CSS. When I turned on browser caching (of CSS and images) the server was no longer hit for the background and logo. I later change the background to a plain colour rather than an image since that rendered faster, and make the sizes of the images exact so that the browser did not have to resize them. Again, an improvement in responsiveness. I repeat, I see no initial delay in accessing your site and I see no delay when I do a refresh or when I click for another page. > All this 30s is spent by > Radiant generating the page. No content has yet been sent to the browser > at that point, so I am certain that it is not a browser rendering issue. As I said, I see no initial delay when I access your site. If this were a problem with Radiant rendering the page then it would surely apply for every page that had be be newly brought out of the database and rendered? Have you considered this, which I met with one site: It ran on a shared service that wasn't terribly Rails-friendly and the time to start-up Passenger, Ruby then Rails and connect to the (remote) SQL database was... yes, about 30 seconds. Once it was up and running things ran better. > As soon as the radiant process finished parsing and compiling the > content, the CPU load drops to 1% and the page is displayed immediately > in my browser. All of which points to an issue on the server side. As I said, as a remote user who can only monitor what my browser is doing, I'm observing something very different. > The 30 second load time is a secondary problem, but far worse is the > fact that direct hits and browser refreshes cause Radiant to go through > all that unnecessary parsing again. I would like them to merely check > when the page was generated last and not do anything if it's been less > than an hour. Its the browser refresh that seems odd to me, and that is why I suggested the browser tools. Let me emphasise one again; as a remote user I an *not* seeing the kind of poor start-up and poor refresh that you are describing. Let me also emphasise that if it is a problem with Radiant rendering and not caching then that would apply to all pages every time under all circumstances, initial, directly entered URL or stepping though links on other pages. Before working on the server I would eliminate the possibility that what you are observing is due to a browser configuration. When I'm testing my sites I use a number of different browsers, different versions of firefox, different version of IE, Chromium, Opera and Konqueror, to make sure that even if the results are not quite the same then they are acceptable. I run those browsers on different hardware, though I do high speed cable to eliminate that kind of problem. On the server side, I'd look to your logs to see how much *work* the server has to do for each page; how many database hits, file references; what the database latency is. The browser tools will tell you the latency of the various components being served up. Some browsers will only fetch components (CSS, graphics, text) one-by-one, others support parallel fetch if your server can handle that. THIS IS IMPORTANT. One point which emerges from that is how your HTML Layout is done. If the browser knows the size and position of everything beforehand then it can lay them out and fill them in as they arrive. If not, it has to recalculate - and hence delay - until all the parts are available. I found that with one of my layouts having the sidebar and its embedded graphics specified BEFORE the main central body eliminated a 3 second delay. It allowed the browser to request the components that made up the sidebar earlier. That alteration in the scheduling of the parallel download made a big difference. Although the "hit" was on the server and server 'latency', the problem was uncovered by looking at things from the POV of the browser, NOT the server. -- "The greatest of all faults is to be conscious of none. Recognizing our limitations & imperfections is the first requisite of progress. Those who believe they have "arrived" believe they have nowhere to go. Some not only have closed their minds to new truth, but they sit on the
Re: [Radiant] Slow page load times
On Sun, Jan 23, 2011 at 8:50 PM, William Ross wrote: > > From a brief look it seems that the rss-reader extension disables all page > caching. It does this in a rather inadvisable way (by amending the Page > class itself rather than defining a specialist subclass) but since your site > is primarily an RSS-aggregator, fixing that wouldn't help much. > > The extension is quite old and I expect the decision dates back to the old > radiant cache: with Rack::Cache in front of the whole thing it doesn't make > sense any more. Simply commenting out these lines in lib/rss_reader.rb: > > def cache? > false > end > > should make a big difference. > William, I just thought of something. We're running Radiant 0.8.0. Would this affect your advice at all? Thanks, Carl
Re: [Radiant] Slow page load times
Thanks for the advice Anton. Pages are coming up fast when they are accessed with a referer header and when they have recently been generated by Radiant. But they are taking 30s when I type the URL in directly and when I click refresh in a browser. All this 30s is spent by Radiant generating the page. No content has yet been sent to the browser at that point, so I am certain that it is not a browser rendering issue. As soon as the radiant process finished parsing and compiling the content, the CPU load drops to 1% and the page is displayed immediately in my browser. All of which points to an issue on the server side. The 30 second load time is a secondary problem, but far worse is the fact that direct hits and browser refreshes cause Radiant to go through all that unnecessary parsing again. I would like them to merely check when the page was generated last and not do anything if it's been less than an hour. On Mon, Jan 24, 2011 at 1:44 AM, Anton J Aylward wrote: > Carl Youngblood said the following on 01/23/2011 02:04 PM: > > When I click around in the site after going through these slow load > > times for all the pages, they come up very suddenly, leading me to > > believe that they are being served from the radiant cache at that > > point. > > I'd suggest looking into the various developer tool plugins for Firefox. > What you described here may be an artefact of the browser, not the > server, caching things like graphics and CSS. > > I found that having the CSS served up as a page, while convenient for > development, meant more work on the server and database. Similarly with > graphics if they needed resizing. > > Once they are cached *in the browser* they don't need to be fetched again. > > You should ask yourself why a page that hasn't been accessed before > would be in the radiant cache? When I try your site, hitting a page > I've not visited before, it comes up fast. > > > -- > Quality is free, but only for those who are willing to pay heavily for > it.- Tom Demarco, "Peopleware" >
Re: [Radiant] Slow page load times
Carl Youngblood said the following on 01/23/2011 02:04 PM: > When I click around in the site after going through these slow load > times for all the pages, they come up very suddenly, leading me to > believe that they are being served from the radiant cache at that > point. I'd suggest looking into the various developer tool plugins for Firefox. What you described here may be an artefact of the browser, not the server, caching things like graphics and CSS. I found that having the CSS served up as a page, while convenient for development, meant more work on the server and database. Similarly with graphics if they needed resizing. Once they are cached *in the browser* they don't need to be fetched again. You should ask yourself why a page that hasn't been accessed before would be in the radiant cache? When I try your site, hitting a page I've not visited before, it comes up fast. -- Quality is free, but only for those who are willing to pay heavily for it.- Tom Demarco, "Peopleware"
Re: [Radiant] Slow page load times
Thanks a lot for taking the time to respond, Will. I have commented out those lines and restarted Apache, but I'm still experiencing the same exact thing. I have confirmed that the ruby processes I see spiking in htop are new ones, so I'm certain that my changes are being read. Thanks, Carl On Sun, Jan 23, 2011 at 8:50 PM, William Ross wrote: > On 23 Jan 2011, at 19:04, Carl Youngblood wrote: > > Hello, we're running a Radiant installation and we're experiencing > very slow load times. Here's our site: > > http://transfigurism.org > > We're running it on a linode 512 vm. The site uses the > radiant_rss_reader extension to construct many different snippets. I > was wondering if this was the problem, but have checked the RSS cache > and verified that it is not retrieving the RSS feeds again if they've > been cached within the set interval (currently 1 hr). > > > From a brief look it seems that the rss-reader extension disables all page > caching. It does this in a rather inadvisable way (by amending the Page > class itself rather than defining a specialist subclass) but since your site > is primarily an RSS-aggregator, fixing that wouldn't help much. > > The extension is quite old and I expect the decision dates back to the old > radiant cache: with Rack::Cache in front of the whole thing it doesn't make > sense any more. Simply commenting out these lines in lib/rss_reader.rb: > > def cache? > false > end > > should make a big difference. > > best, > > will > > > > > > > But it seems that whenever I initiate a refresh from the browser, > pages take about 30 seconds to load. Almost all of this time appears > to be spent in ruby. When I click refresh, the CPU load spikes to > about 60% for the duration of this 30s period and then at the end the > page appears suddenly, implying that bandwidth and browser rendering > are not the bottlenecks. > > When I click around in the site after going through these slow load > times for all the pages, they come up very suddenly, leading me to > believe that they are being served from the radiant cache at that > point. > > My first question is, is there a way to make it so that a browser > refresh does not cause the radiant cache to be invalidated? I would > like to set up an hourly cron job that spiders the web site to > basically "prime" the cache and make it so that real-world requests > are served up quickly from the cache instead of having to be re-parsed > and rendered by radiant. > > Second question is, why is my site taking so long for radiant to > render? It doesn't strike me as an especially complex layout? Is my > radiant process running out of memory or something? Any > recommendations on apache and/or db configurations that would help > here? > > Thanks, > Carl > > >
Re: [Radiant] Slow page load times
On 23 Jan 2011, at 19:04, Carl Youngblood wrote: > Hello, we're running a Radiant installation and we're experiencing > very slow load times. Here's our site: > > http://transfigurism.org > > We're running it on a linode 512 vm. The site uses the > radiant_rss_reader extension to construct many different snippets. I > was wondering if this was the problem, but have checked the RSS cache > and verified that it is not retrieving the RSS feeds again if they've > been cached within the set interval (currently 1 hr). From a brief look it seems that the rss-reader extension disables all page caching. It does this in a rather inadvisable way (by amending the Page class itself rather than defining a specialist subclass) but since your site is primarily an RSS-aggregator, fixing that wouldn't help much. The extension is quite old and I expect the decision dates back to the old radiant cache: with Rack::Cache in front of the whole thing it doesn't make sense any more. Simply commenting out these lines in lib/rss_reader.rb: def cache? false end should make a big difference. best, will > > But it seems that whenever I initiate a refresh from the browser, > pages take about 30 seconds to load. Almost all of this time appears > to be spent in ruby. When I click refresh, the CPU load spikes to > about 60% for the duration of this 30s period and then at the end the > page appears suddenly, implying that bandwidth and browser rendering > are not the bottlenecks. > > When I click around in the site after going through these slow load > times for all the pages, they come up very suddenly, leading me to > believe that they are being served from the radiant cache at that > point. > > My first question is, is there a way to make it so that a browser > refresh does not cause the radiant cache to be invalidated? I would > like to set up an hourly cron job that spiders the web site to > basically "prime" the cache and make it so that real-world requests > are served up quickly from the cache instead of having to be re-parsed > and rendered by radiant. > > Second question is, why is my site taking so long for radiant to > render? It doesn't strike me as an especially complex layout? Is my > radiant process running out of memory or something? Any > recommendations on apache and/or db configurations that would help > here? > > Thanks, > Carl
[Radiant] Slow page load times
Hello, we're running a Radiant installation and we're experiencing very slow load times. Here's our site: http://transfigurism.org We're running it on a linode 512 vm. The site uses the radiant_rss_reader extension to construct many different snippets. I was wondering if this was the problem, but have checked the RSS cache and verified that it is not retrieving the RSS feeds again if they've been cached within the set interval (currently 1 hr). But it seems that whenever I initiate a refresh from the browser, pages take about 30 seconds to load. Almost all of this time appears to be spent in ruby. When I click refresh, the CPU load spikes to about 60% for the duration of this 30s period and then at the end the page appears suddenly, implying that bandwidth and browser rendering are not the bottlenecks. When I click around in the site after going through these slow load times for all the pages, they come up very suddenly, leading me to believe that they are being served from the radiant cache at that point. My first question is, is there a way to make it so that a browser refresh does not cause the radiant cache to be invalidated? I would like to set up an hourly cron job that spiders the web site to basically "prime" the cache and make it so that real-world requests are served up quickly from the cache instead of having to be re-parsed and rendered by radiant. Second question is, why is my site taking so long for radiant to render? It doesn't strike me as an especially complex layout? Is my radiant process running out of memory or something? Any recommendations on apache and/or db configurations that would help here? Thanks, Carl
