Re: [web2py] Web2py - Front-end performance optimization tutorial

[Louis Amon]

Indeed I should look into contributing the book at some point. Still feel a 
bit too noobish for that just yet ^^

As for Heroku & Gzip, here's the official source to my info:
https://devcenter.heroku.com/articles/http-routing#gzipped-responses

On Monday, September 14, 2015 at 5:26:07 AM UTC+2, Kiran Subbaraman wrote:
>
> Louis,
> Thanks for putting this together. There are a bunch of performance tips in 
> the web2py book too. Wondering if your compilation can find a home there. 
> Also, in another thread, you had mentioned about Heroku deployments, and 
> how it requires the web application to perform compression of assets, and 
> not the web-server. That is a good thing to know, and one which needs to be 
> captured some place accessible (web2py book, again?). 
>
> 
> Kiran Subbaramanhttp://subbaraman.wordpress.com/about/
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] Web2py - Front-end performance optimization tutorial

[Carlos Cesar Caballero Díaz]

Hi Louis, good tutorial. I usually try serve my static (and uploaded) 
files directly, without pass for the web2py layer.


About the gzip, maybe this Stackoverflow threads can help:

http://stackoverflow.com/questions/8506897/how-do-i-gzip-compress-a-string-in-python
http://stackoverflow.com/questions/1543652/python-gzip-is-there-a-way-to-decompress-from-a-string

The zlib library seems to work fine with strings:

https://docs.python.org/2/library/zlib.html


Cheers.


El 13/09/15 a las 14:03, Louis Amon escribió:
I'm opening a thread dedicated to website performance issues in 
web2py, giving the answers I've got so far so that other may 
contribute as well.



Static assets (fonts, vendor CSS & JS)

How do you manage & distribute these ?

Fonts are easiest to manage using Google Fonts 
. This tool will generate an optimized 
file load management through Google's CDNs.


If at all possible, your static files need to be compiled into one big 
bundle per page (main.css & main.js) so that you can optimize its 
delivery. This can be done using Bower to download the source code and 
Gulp or Grunt to concatenate it into one file.


Otherwise, I'd recommend using the minified version of your vendors 
and using your own CDN for delivery, as publicly available ones aren't 
very reliable in terms of speed.



Dynamic assets (fast-changing CSS & JS, e.g : your main.css & main.js)

The challenge here is minification, variable replacement for JS (if 
you're not using AngularJS), compression & versioning.


Grunt & Gulp can take care of all this through a deployment task 
system, and you can get a manifest in JSON format giving you the 
versioned filename for each of your generated files.
See gulp-rev-all  for the 
versioning / manifest generation part.


Personally, I'm using Gulp to generate the minified bundles and then 
I'm using Django's collectstatic combined with Whitenoise to generate 
a gzipped & versioned file for all of my assets (I can go into details 
on how to combine Django's collectstatic with Web2py in another thread 
if someone is interested)



Static images

Google PageSpeed Insights 
 recommends 
that you resize & compress your images as much as can be done.


Compressing is a matter of using higher compression indices when 
generating your output files in .png or .jpg formats (using Photoshop, 
GIMP...), as these algorithmically include a compression.
In case you would need to use a format different than these, I'd 
recommend gzipping... but really you don't wanna stray too much from 
PNG & JPG.


As far as I know, resizing has to be done manually since no machine 
can know what the maximum viewable size of your content may be 
(especially with responsive websites). It is a sensible step that 
requires a lot of human interaction from your webdesigner.


Compression, on the other hand, can be done automatically with a gulp 
task .



Dynamic images (uploads)

Again, they need to be optimized in terms of size & compression.

Resizing can be easilly done in web2py with a computed field (i.e. 
something like Field('image_resized', 'upload', 
compute=THUMB(200,200), uploadfolder='uploads/resized')).


Compression can be done in the compute fonction I guess, but I confess 
I haven't spent much time on this yet.
The Python Imaging Library, forked as Pillow 
 these days, has all sorts of 
functions to achieve that.



HTML responses

Once you've compressed static assets, Google PageSpeed will 
(rightfully) nag you about compressing your HTML response itself.


Web2py includes a contribution for html minification 
(contrib/minify/html_minify.py) which can be used easilly with a 
decorator.


I've created a small decorator than handles compression:

|

importzlib


defdeflate(func):


def_f():


out=func()


render =response.render(out)ifisinstance(out,dict)elseout


if'deflate'inrequest.env.http_accept_encoding:


response.headers['Content-Encoding']='deflate'


returnzlib.compress(render)


else:


returnrender


return_f

|


Using these decorators has to be done in the right order 
(minification, then compression) otherwise you'll run into trouble :)


Here's how to do it :

|
@deflate
@minify
defindex():
returndict()
|


I've managed to reduce the size of my homepage's HTML code from 36.6KB 
to 7.6KB using both minification & compression.


Please note that these decorators do mean a CPU overhead for your 
webserver.
It is up to you to use one or both depending on what kind of hardware 
your server runs on.
There is also the possibility of using a caching strategy to reduce 
processing time. I'd recommend web2py's @cache.action(...) decorator 
for that.



HTML response headers

Now for the most important part : settings the right headers on your 
responses.


Most of the 

Re: [web2py] Web2py - Front-end performance optimization tutorial

[António Ramos]

speaking about optimization , here is my homework
https://github.com/ramstein74/Coffee_Jade_Stylus_inside_web2py

2015-09-14 15:11 GMT+01:00 Louis Amon :

> Indeed I should look into contributing the book at some point. Still feel
> a bit too noobish for that just yet ^^
>
> As for Heroku & Gzip, here's the official source to my info:
> https://devcenter.heroku.com/articles/http-routing#gzipped-responses
>
> On Monday, September 14, 2015 at 5:26:07 AM UTC+2, Kiran Subbaraman wrote:
>>
>> Louis,
>> Thanks for putting this together. There are a bunch of performance tips
>> in the web2py book too. Wondering if your compilation can find a home
>> there. Also, in another thread, you had mentioned about Heroku deployments,
>> and how it requires the web application to perform compression of assets,
>> and not the web-server. That is a good thing to know, and one which needs
>> to be captured some place accessible (web2py book, again?).
>>
>> 
>> Kiran Subbaramanhttp://subbaraman.wordpress.com/about/
>>
>> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] Web2py - Front-end performance optimization tutorial

[Louis Amon]

While working with caching, I noticed what seems to me like a caveat in 
web2py's current design.

When serving static files from the webserver, you can either :

- use the "static" pseudo-controller (it's actually a route built directly 
above the WSGI), in which case you can't set your own caching strategy.
Worse, "static" by default sets this value "headers.setdefault(
'Cache-Control', 'private')", which in turn becomes a major hindrance when 
building a CDN proxy on top of the webserver.
Let me explain myself here : your may want to simply add a CDN layer above 
your webserver so that it handles the caching strategy (Cloudfront does 
that just fine with behaviour settings). Problem is : Cloudfront will 
refuse to cache any data if the origin specifies "private" as Cache-Control 
(because "private" basically means it shouldn't be cached on a 
server-scale), so not only does web2py not allow for a custom caching 
strategy, but it also prevents your from using an alternative means

- build your own controller for serving your static files with whichever 
headers you see fit. The issue there would be that your controller would be 
run after models, which is a waste of resources, practically resulting in a 
high TTFB (Time to First Byte). This can be balanced back with a CDN layer 
though.

 - add a MiddleWare that handles static files more efficiently that web2py 
does (haven't had any luck so far, so I can't say wether it's doable or not)


I'll post about building a custom static controller soon, but I'm curious 
if anyone has a better solution :)

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Web2py - Front-end performance optimization tutorial

[Louis Amon]

I'm opening a thread dedicated to website performance issues in web2py, 
giving the answers I've got so far so that other may contribute as well.


Static assets (fonts, vendor CSS & JS)

How do you manage & distribute these ?

Fonts are easiest to manage using Google Fonts 
. This tool will generate an optimized file 
load management through Google's CDNs.

If at all possible, your static files need to be compiled into one big 
bundle per page (main.css & main.js) so that you can optimize its delivery. 
This can be done using Bower to download the source code and Gulp or Grunt 
to concatenate it into one file.

Otherwise, I'd recommend using the minified version of your vendors and 
using your own CDN for delivery, as publicly available ones aren't very 
reliable in terms of speed.


Dynamic assets (fast-changing CSS & JS, e.g : your main.css & main.js)

The challenge here is minification, variable replacement for JS (if you're 
not using AngularJS), compression & versioning.

Grunt & Gulp can take care of all this through a deployment task system, 
and you can get a manifest in JSON format giving you the versioned filename 
for each of your generated files.
See gulp-rev-all  for the 
versioning / manifest generation part.

Personally, I'm using Gulp to generate the minified bundles and then I'm 
using Django's collectstatic combined with Whitenoise to generate a gzipped 
& versioned file for all of my assets (I can go into details on how to 
combine Django's collectstatic with Web2py in another thread if someone is 
interested)


Static images

Google PageSpeed Insights 
 recommends that 
you resize & compress your images as much as can be done.

Compressing is a matter of using higher compression indices when generating 
your output files in .png or .jpg formats (using Photoshop, GIMP...), as 
these algorithmically include a compression.
In case you would need to use a format different than these, I'd recommend 
gzipping... but really you don't wanna stray too much from PNG & JPG.

As far as I know, resizing has to be done manually since no machine can 
know what the maximum viewable size of your content may be (especially with 
responsive websites). It is a sensible step that requires a lot of human 
interaction from your webdesigner.

Compression, on the other hand, can be done automatically with a gulp task 
.


Dynamic images (uploads)

Again, they need to be optimized in terms of size & compression.

Resizing can be easilly done in web2py with a computed field (i.e. 
something like Field('image_resized', 'upload', compute=THUMB(200,200), 
uploadfolder='uploads/resized')).

Compression can be done in the compute fonction I guess, but I confess I 
haven't spent much time on this yet.
The Python Imaging Library, forked as Pillow 
 these days, has all sorts of functions 
to achieve that.


HTML responses

Once you've compressed static assets, Google PageSpeed will (rightfully) 
nag you about compressing your HTML response itself.

Web2py includes a contribution for html minification 
(contrib/minify/html_minify.py) which can be used easilly with a decorator.

I've created a small decorator than handles compression:

import zlib

def deflate(func):

def _f():

out = func()

render = response.render(out) if isinstance(out, dict) else out

if 'deflate' in request.env.http_accept_encoding:

response.headers['Content-Encoding'] = 'deflate'

return zlib.compress(render)

else:

return render

return _f


Using these decorators has to be done in the right order (minification, 
then compression) otherwise you'll run into trouble :)

Here's how to do it :

@deflate
@minify
def index():
   return dict()


I've managed to reduce the size of my homepage's HTML code from 36.6KB to 
7.6KB using both minification & compression.

Please note that these decorators do mean a CPU overhead for your webserver.
It is up to you to use one or both depending on what kind of hardware your 
server runs on.
There is also the possibility of using a caching strategy to reduce 
processing time. I'd recommend web2py's @cache.action(...) decorator for 
that.


HTML response headers

Now for the most important part : settings the right headers on your 
responses.

Most of the performance-oriented websites nowadays will rely on CDNs to act 
as proxies and cache resources.
This is especially useful for static assets, as these can be served through 
your webserver and then cached in your CDN.

If you use Amazon Cloudfront for instance, you can set your website 
(http://your-website.com) as origin of the distribution, which means 
Cloudfront will look for the resource on your webserver once and then 
basically cache it "forever", serving it to your 

Re: [web2py] Web2py - Front-end performance optimization tutorial

[Kiran Subbaraman]

Louis,
Thanks for putting this together. There are a bunch of performance tips 
in the web2py book too. Wondering if your compilation can find a home 
there. Also, in another thread, you had mentioned about Heroku 
deployments, and how it requires the web application to perform 
compression of assets, and not the web-server. That is a good thing to 
know, and one which needs to be captured some place accessible (web2py 
book, again?).



Kiran Subbaraman
http://subbaraman.wordpress.com/about/

On Sun, 13-09-2015 11:33 PM, Louis Amon wrote:
I'm opening a thread dedicated to website performance issues in 
web2py, giving the answers I've got so far so that other may 
contribute as well.



Static assets (fonts, vendor CSS & JS)

How do you manage & distribute these ?

Fonts are easiest to manage using Google Fonts 
. This tool will generate an optimized 
file load management through Google's CDNs.


If at all possible, your static files need to be compiled into one big 
bundle per page (main.css & main.js) so that you can optimize its 
delivery. This can be done using Bower to download the source code and 
Gulp or Grunt to concatenate it into one file.


Otherwise, I'd recommend using the minified version of your vendors 
and using your own CDN for delivery, as publicly available ones aren't 
very reliable in terms of speed.



Dynamic assets (fast-changing CSS & JS, e.g : your main.css & main.js)

The challenge here is minification, variable replacement for JS (if 
you're not using AngularJS), compression & versioning.


Grunt & Gulp can take care of all this through a deployment task 
system, and you can get a manifest in JSON format giving you the 
versioned filename for each of your generated files.
See gulp-rev-all  for the 
versioning / manifest generation part.


Personally, I'm using Gulp to generate the minified bundles and then 
I'm using Django's collectstatic combined with Whitenoise to generate 
a gzipped & versioned file for all of my assets (I can go into details 
on how to combine Django's collectstatic with Web2py in another thread 
if someone is interested)



Static images

Google PageSpeed Insights 
 recommends 
that you resize & compress your images as much as can be done.


Compressing is a matter of using higher compression indices when 
generating your output files in .png or .jpg formats (using Photoshop, 
GIMP...), as these algorithmically include a compression.
In case you would need to use a format different than these, I'd 
recommend gzipping... but really you don't wanna stray too much from 
PNG & JPG.


As far as I know, resizing has to be done manually since no machine 
can know what the maximum viewable size of your content may be 
(especially with responsive websites). It is a sensible step that 
requires a lot of human interaction from your webdesigner.


Compression, on the other hand, can be done automatically with a gulp 
task .



Dynamic images (uploads)

Again, they need to be optimized in terms of size & compression.

Resizing can be easilly done in web2py with a computed field (i.e. 
something like Field('image_resized', 'upload', 
compute=THUMB(200,200), uploadfolder='uploads/resized')).


Compression can be done in the compute fonction I guess, but I confess 
I haven't spent much time on this yet.
The Python Imaging Library, forked as Pillow 
 these days, has all sorts of 
functions to achieve that.



HTML responses

Once you've compressed static assets, Google PageSpeed will 
(rightfully) nag you about compressing your HTML response itself.


Web2py includes a contribution for html minification 
(contrib/minify/html_minify.py) which can be used easilly with a 
decorator.


I've created a small decorator than handles compression:

|

importzlib


defdeflate(func):


def_f():


out=func()


render =response.render(out)ifisinstance(out,dict)elseout


if'deflate'inrequest.env.http_accept_encoding:


response.headers['Content-Encoding']='deflate'


returnzlib.compress(render)


else:


returnrender


return_f

|


Using these decorators has to be done in the right order 
(minification, then compression) otherwise you'll run into trouble :)


Here's how to do it :

|
@deflate
@minify
defindex():
returndict()
|


I've managed to reduce the size of my homepage's HTML code from 36.6KB 
to 7.6KB using both minification & compression.


Please note that these decorators do mean a CPU overhead for your 
webserver.
It is up to you to use one or both depending on what kind of hardware 
your server runs on.
There is also the possibility of using a caching strategy to reduce 
processing time. I'd recommend web2py's @cache.action(...) decorator 
for that.



HTML response headers

Now for the most important part :