Please see below...
Shanti Subramanyam - PAE wrote:
Although, OLIO-108 (Application performance is dominated by gd library)
was filed against the PHP application, this issue will plague both the
rails and java implementation as well.
As such it makes sense to explore a solution that will work for any
implementation.
On searching the internet, I did not find much references to gd library
performance. This makes me think that :
a) People don't use gd when they care about performance OR
b) Those who do use gd are not using it in performance-sensitive areas. OR
c) Folks have just not realized how cpu-intensive gd/libjpeg is.
I did however find many products specifically producing thumbnails and
they were making all sorts of performance claims. So it is likely that
those in the a) category above probably use a proprietary product.
Here are several possible solutions to fixing this and I'd like to hear
others opinions to decide which one we should adopt.
1. Use 'fastimagecopyresampled' as documented in
http://us.php.net/manual/pt_BR/function.imagecopyresampled.php
with a reasonable quality level. Tests did improve latency significantly
(by about 80%) but the system profile is still dominated by gd/jpeg.
Of course this fix only works for the PHP app and we will have to
figure out different optimizations for the others.
I'm quite OK with platform-specific improvements. While this proposal is
not the perfect solution, it seems to be a big step in the right
direction. Also, the impact to the code and the Olio deployment
architecture is minimal. So this is a +1 for me.
2. The jpeg library offers many optimizations (e.g. scale_denom
parameter for decompression). Search for scale_denom in :
http://ou800doc.caldera.com/en/jpeg/libjpeg.txt
However, the php gd extension does not provide direct access to these
tunables. By setting scale_denom to 4 (tested by using an interposing
library), we can get a dramatic improvement in performance - almost 10
fold.
One idea proposed by Richard is to enhance the PHP gd extension by
adding a new function that will take as arguments the thumbnail size,
quality etc. and set the appropriate parameters before calling into
libjpeg.
This of course is a big hack - it is not clean and once again only
improves the PHP app.
Being an Apache project, we do believe in open source. I'd have no
objection if this change is contributed to the gd extension and then
used by Olio. But I'd be against bundling a hacked up gd library. The gd
library itself is off the scope for Olio, although it is a dependency
for the PHP implementation. Thumbs down on this one for me, at least as
it stands right now. But if you contribute it to gd and then use it, I'd
vote for it.
3. If we are particular that the bulk of the load remain in core php (or
rails or jappserver) the better solution seems to be to send the thumb
request to another server. This server could even have a bare-bones
(read 'C') based implementation using libjpeg directly, by-passing all
the language-level libraries/wrappers etc. Perhaps we could then get
back the latency we lose by doing yet another network hop. The other
advantage is that this code becomes common across implementations
(except for the webserver we use to front-end the request) and
thumb-nail generation performance becomes common across Olio and will
not affect core web/app server functionality.
The dis-advantage of this approach is that we complicate deployment -
we will now have yet another server to deploy on yet another system
(although of course we could deploy this on the node serving the
filestore or the database so as not to require another physical system).
Neutral. While I like the solution and would opt for this in a
production site, it is a non-trivial change to the Olio deployment
architecture. It complicates deployment and also adds the need for
special handling of thumbnails pending processing (which are usually
used by the add operations). If there is strong support for this option,
I'd go with it.
-Akara
