Perhaps it has nothing to do with web2py, however it is your install of linux has the max-open-file-descripters setting set at a really low value, you might want to try increasing this setting.
Heres a bunch of links for you to take a look at. http://www.cs.wisc.edu/condor/condorg/linux_scalability.html http://hausheer.osola.com/docs/11 http://www.netadmintools.com/art295.html http://www.xenoclast.org/doc/benchmark/HTTP-benchmarking-HOWTO/node7.html http://thedaneshproject.com/posts/how-to-increase-total-file-descriptors-count-on-linux/ http://stackoverflow.com/questions/34588/how-do-i-change-the-number-of-open-files-limit-in-linux http://www.google.com/search?q=linux+open+file+limit&ie=utf-8&oe=utf-8&aq=t&rls=com.ubuntu:en-US:unofficial&client=firefox-a -Thadeus On Tue, Sep 22, 2009 at 5:05 PM, Paul <[email protected]> wrote: > > using lsof i was able to determine that each time I do a refresh, > web2py is opening several new file handles to [app]/cache/cache.lock > (it seems one open file id per image) so after 20 refreshes or so I > reach the limit for open files.. Any idea why this is happening and > how to fix it? > > Paul > > > > On Sep 22, 10:39 am, Paul <[email protected]> wrote: > > I still have the problem with the new trunk, but I did notice that I > > have been using a different implementation of download that the > > default one. mine is essentially > > return response.stream(open(os.path.join > > (request.folder,'uploads',request.args[0]),'rb')) > > > > this was an old implementation of download right? (because I don't > > remember changing this, but I may have). Does response.stream close > > the file when it is done? this could be my problem... can I just > > enclose it in a try .. finally block and close the stream in the > > finally block when it is done? > > > > Also, I tried to use the new download method ( return response.download > > (request,db) ) but it doesn't seem to allow access to files in the > > uploads directory unless they originated from an 'upload' db field. ie > > if I just stick a file (say img1.jpg in the uploads directory) and > > try /default/download/img1.jpg I get a 404 error. > > I can even generate internal errors if I name the file img.1.jpg (an > > extra . ), which causes a key error (listed below). Am I using > > 'download' incorrectly? because I do need a method to directly access > > files in the uploads directory. > > > > Thanks, > > > > Paul > > > > Traceback (most recent call last): > > File "/Users/green/work/CollaborativePhoto/Code/Server/web2py-read- > > only/gluon/restricted.py", line 178, in restricted > > exec ccode in environment > > File "/Users/green/work/CollaborativePhoto/Code/Server/web2py-read- > > only/applications/imagetest/controllers/default.py", line 61, in > > <module> > > File "/Users/green/work/CollaborativePhoto/Code/Server/web2py-read- > > only/gluon/globals.py", line 102, in <lambda> > > self._caller = lambda f: f() > > File "/Users/green/work/CollaborativePhoto/Code/Server/web2py-read- > > only/applications/imagetest/controllers/default.py", line 44, in > > download > > return response.download(request,db) > > File "/Users/green/work/CollaborativePhoto/Code/Server/web2py-read- > > only/gluon/globals.py", line 199, in download > > field = db[t][f] > > File "/Users/green/work/CollaborativePhoto/Code/Server/web2py-read- > > only/gluon/sql.py", line 531, in __getitem__ > > return dict.__getitem__(self, str(key)) > > KeyError: 'img' > > > > On Sep 21, 9:10 pm, mdipierro <[email protected]> wrote: > > > > > Although I cannot reproduce the problem I noticed web2py is opening > > > more tempfiles files that it needs. > > > I fixed this and the change should also make it faster on GET > > > requests. > > > In trunk now. > > > > > Please check if it fixes the problem. > > > > > Massimo > > > > > On Sep 21, 4:34 pm, Paul <[email protected]> wrote: > > > > > > I don't know if you were referring to my problem or Iceberg's... > > > > regardless, I checked out the latest trunk (rev 1240) and tried my > > > > application again and was able to reproduce my bug (with essentially > > > > the same error trace as above). > > > > > > the view is essentially this: > > > > > > {{for img in imgs:}} > > > > {{u=URL(r=request,c='default',f='download', args= img.img_name)}} > > > > {{=A(IMG(_src=u),_href=u)}} > > > > {{pass}} > > > > > > and there are about 10 png's in the 'uploads' directory / in the > > > > images table. refresh the page ~20 times and it crashes. > > > > > > Paul > > > > > > On Sep 21, 5:06 pm, mdipierro <[email protected]> wrote: > > > > > > > I cannot reproduce this. Can you try the latest trunk and let me > know > > > > > if you still get before we spend time in debugging? > > > > > > > On Sep 21, 10:02 am, Iceberg <[email protected]> wrote: > > > > > > > > Thanks for your attention, but the new experimental server might > not > > > > > > be the suspect this time, because my production server is still > using > > > > > > SVN rev1222, which is earlier than the experimental server > (sneaky > > > > > > server) 's birth. > > > > > > > > On Sep21, 9:02pm, mdipierro <[email protected]> wrote: > > > > > > > > > The trunk uses a new experimental server. It still has come > problems. > > > > > > > I will revert to cherrypy for now > > > > > > > > > Massimo > > > > > > > > > On Sep 21, 4:16 am, Iceberg <[email protected]> wrote: > > > > > > > > > > On Sep11, 5:13pm, zahariash <[email protected]> wrote: > > > > > > > > > > > On 11 Wrz, 07:13, mdipierro <[email protected]> > wrote: > > > > > > > > > > > > please check if it is fixed. I had to change your fix > because it was > > > > > > > > > > not python 2.4 compliant. > > > > > > > > > > > Revision 1124 works ok. It's fixed. > > > > > > > > > > I never saw web2py breaks in my 17 months experience, but I > just > > > > > > > > encountered an web2py out-of-service on my Linux production > server. > > > > > > > > The process is still running, but every request results in an > error > > > > > > > > ticket page, which are all > > > > > > > > IOError: [Errno 24] Too many open files: > '/home/me/web2py_trunk/ > > > > > > > > blahblah' > > > > > > > > > > I am using source code version from google svn rev1222. Is > the leak > > > > > > > > found by Zahariash already fixed in this revision? > > > > > > > > > > After kill and restart, now everything seems ok. Anyway, I > will keep > > > > > > > > close look to my production server, and come back to you if > anything > > > > > > > > new happened. > > > > > > > > > > Iceberg. > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/web2py?hl=en -~----------~----~----~----~------~----~------~--~---
