Re: [Image-SIG] PIL Consuming All System Resources

2010-05-31 Thread Andy McCurdy
Thanks for the explanation. As your and others have suggested, I think limiting the size of the source file seems like a reasonable idea. -andy On Sun, May 30, 2010 at 5:59 AM, Fredrik Lundh wrote: > The file has mode=RGB size=20400x28079, so you'd need about 2291246400 > bytes to load it all in

Re: [Image-SIG] PIL Consuming All System Resources

2010-05-30 Thread Yury V. Zaytsev
On Sun, 2010-05-30 at 14:59 -0700, Edward Cannon wrote: > Another method used by many websites is to put a limit on uploaded > file size. This has the double benifit of saving on bandwidth as well. > Facebook uses 5MB This is no magic bullet, though. As with ZIP bombs, you can craft a maliciou

Re: [Image-SIG] PIL Consuming All System Resources

2010-05-30 Thread Edward Cannon
Another method used by many websites is to put a limit on uploaded file size. This has the double benifit of saving on bandwidth as well. Facebook uses 5MB if I recall correctly, and that seems to work really well in practice, large enough to handle most camera files, but small enough to

Re: [Image-SIG] PIL Consuming All System Resources

2010-05-30 Thread Fredrik Lundh
I spent about a decade working on libraries with paging, tiling, and streaming processing architectures. PIL's simple storage model is pretty much a reaction against that. It has worked reasonably well this far :-) However, note that for resizing and some color conversions, the draft & thumbnail

Re: [Image-SIG] PIL Consuming All System Resources

2010-05-30 Thread Yury V. Zaytsev
Hi! On Sun, 2010-05-30 at 14:59 +0200, Fredrik Lundh wrote: > The file has mode=RGB size=20400x28079, so you'd need about 2291246400 > bytes to load it all into memory at once, and twice that to do e.g. > color conversion (which creates a second image memory), so it's a bit > on the big side, at l

Re: [Image-SIG] PIL Consuming All System Resources

2010-05-30 Thread Fredrik Lundh
The file has mode=RGB size=20400x28079, so you'd need about 2291246400 bytes to load it all into memory at once, and twice that to do e.g. color conversion (which creates a second image memory), so it's a bit on the big side, at least for a 32-bit environment. Note that PIL doesn't actually read t

[Image-SIG] PIL Consuming All System Resources

2010-05-30 Thread Andy McCurdy
Hi, We've been using PIL for the last 2 years to resize our users' uploaded images and have been extremely pleased. Thanks for all your effort. I noticed earlier today that we were experiencing issues of servers running out of memory. It looks like the problem might be within PIL. I've put togeth