On Wed, 6 Oct 2010, Peter Langfelder wrote:
you should close files that you do not use anymore. the maximum number
of open files is likely 4000 or so. Use close(file) before you open
the next one.
He/she said he did that, and as the limit on open connections is 128,
that cannot be the sole cause.
Without even the 'at a mimimum information' requested in the posting
guide let alone a reproducible example it is not possible to know the
cause. But most OSes have utilities available to trace the
opening/closing of files. (We weren't told but this looks like
Windows -- there I have used the monitors from sysinternals, now part
of Microsoft.)
And BTW, the subject line omitted the associated warning which gives
the underlying OS-level error -- that would have been a much better
choice of subject.
Peter
On Wed, Oct 6, 2010 at 4:55 PM, Yong Wang <wangyo...@gmail.com> wrote:
Dear List
I am running a loop downloading web pages and save the html to a
temporary file (use download.file() )
then read (using readLines) it in for processing;
finally write useful info from each processed page to a unique file
the problem is once the loop runs up to somewhere near 5000, it will
throw out an err like below and won't go further.
----------------------------------------------------------------
Error in file(file, ifelse(append, "a", "w")) :
cannot open the connection
-----------------------------------------------------------------
In the meantime, a request for new connection won't be successful, for
example, a request for the help page of "file" will trigger err below
-----------------------------------------------------------------------
?file
Error in gzfile(file, "rb") : cannot open the connection
In addition: Warning message:
In gzfile(file, "rb") :
cannot open compressed file
'C:/PROGRA~1/R/R-211~1.1/library/stats/help/aliases.rds', probable
reason 'Too many open files'
-----------------------------------------------------------------------
I am not sure if the problem is too many connections not closed. since
I close the file connection after each readLines.
checking with showConnections(all=T) does not show excessive
connections and closeAllConnections() does not help.
Can any one help me on this?
Any answer highly appreciated.
yong
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
