On Sun, Mar 02, 2008 at 04:17:53PM -0800, Ramon Felciano wrote: > Hi -- > > I'm running into a "Too Many Open Files" IOError 24 when looping through > several hundred calls to a website. I'm calling twill via the Python API > directly, and it looks like it is getting hung up inside mechanize and > subprocess calls. Is there an upper bound on how many times you can call > twill from within the same interpreter? Or are there explicit calls to make > to close or otherwise clean up after a particular twill session / subprocess > call?
I've encountered something like this in the past, and for me it was twill trying to call tidy, which I didn't have. IIRC, the subprocess call was leaking file descriptors. You *may* be seeing something similar. Try turning the config option to use tidy off to see if it helps. config use_tidy 0 Robert > > Thanks! > > Ramon > > Traceback (most recent call last): > File "upload.py", line 306, in <module> > File "upload.py", line 278, in cleanUpPubmedUrlsAndSave > File "upload.py", line 210, in saveEntries > File "upload.py", line 240, in cleanUpPubmedUrls > File "upload.py", line 63, in getPubmedIdsForTitle > File "upload.py", line 45, in getPubmedIdsForTitle_twill > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\commands.py", line > 108, in go > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\browser.py", line > 113, in go > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\browser.py", line > 516, in _journey > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\other_packages\mechanize\_mechanize.py", > line 156, in open > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\other_packages\mechanize\_mechanize.py", > line 201, in _mech_open > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\other_packages\mechanize\_mechanize.py", > line 255, in _set_response > File "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\utils.py", > line 336, in set_response > File "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\utils.py", > line 372, in _cleanup_html > File "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\utils.py", > line 288, in run_tidy > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\other_packages\subprocess.py", > line 536, in __init__ > File > "c:\python25\lib\site-packages\twill-0.9b1-py2.5.egg\twill\other_packages\subprocess.py", > line 588, in _get_handles > IOError: [Errno 24] Too many open files > > > > > ----+---- > This email message (and any attached document) contains information from > Ingenuity Systems Inc. which may be considered confidential by Ingenuity, or > which may be privileged or otherwise exempt from disclosure under law, and is > for the sole use of the individual or entity to whom it is addressed. Any > other dissemination, distribution or copying of this message is strictly > prohibited. If you receive this message in error, please notify me and > destroy the attached message (and all attached documents) immediately. > > _______________________________________________ > twill mailing list > [email protected] > http://lists.idyll.org/listinfo/twill > > !DSPAM:4021,47cb43c8186955219720167! > _______________________________________________ twill mailing list [email protected] http://lists.idyll.org/listinfo/twill
