On Jul 3, 2007, at 4:02 AM, Clinton Gormley wrote:
This last statement I have a quibble about : using mod_perl to handle
file uploads. I may be wrong here, so I'd welcome reasoned
disagreement,
but the way I understand it:
not all file uploads, but large ones. anything over 100k i won't put
onto mp.
- My image uploads require processing, so I need some form of Perl
- I
already have that available in mod_perl.
i'm doing that now. i'm migrating all the image processing to python
though. the image resizing generated by python is way better ( speed
is slightly better in python, but i'm actually worried about
quality. i'm not too happy with the way Imager resizes stuff, too
many artifacts in resize+jpeg compression. gd+image magick worked
poorly re: resources.)
to handle it, i'm doing thumbnails in perl as-created and flagging
the image in the db with a 'not processed' bool. a persistent
process queries the db and resizes the images in python.
- The mod_perl processes aren't being kept tied up for a long period,
because the proxy is handling the slow bit: the transfer.
i've had proxies break send off chunks when i've updated daemons.
its annoying.
- The only downside is that a large image upload plus processing
could cause the size of the process to grow a lot, reducing the
number of processes that you can run at once.
=snip
- Keep your upload processing mod_perls on one box, so that if you
are flooded with uploads, just the upload functionality on your
site
is taken out, rather than your entire site
thats my biggest concern. you could stay on the same box and just
create a sep. pool of mp servers for the upload proxies though... any
sort of processing done within mp is at the expense of other mp
servers. if your page generation time is an avg .08 seconds , and
you just did an image upload + resize that took 3 seconds to process,
you cost yourself a bunch of resources that mp could have handled.
so let me clarify my earlier statement-- you don't want your main mod-
perl app handling the uploads. if you can push it off onto another
mp process/server, thats ok. but if its on your main server, you're
just creating eventual problems- your authorization and general
content generation will be competing for resources with processes
that can take several seconds. bad idea.
// Jonathan Vanasco
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
| CEO/Founder SyndiClick Networks
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
| Founder/CTO/CVO
| FindMeOn.com - The cure for Multiple Web Personality Disorder
| Web Identity Management and 3D Social Networking
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
| RoadSound.com - Tools For Bands, Stuff For Fans
| Collaborative Online Management And Syndication Tools
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - -
