On 17/07/2009 7:57 PM, Marilyn & Rich Short wrote:
> Hello,
>
> I'm having a problem in R with repeated use of the "load", "save", and
> "close" commands. I'm getting an error message that reads, "Too many
> open files". I'm opening files and closing them (and unlinking them),
> but when I go through that process 509 times, the program halts and I
> get this error message: "cannot open the connection" with warning
> messages: "Too many open files". I've been working on this problem for
> a couple of weeks and have gleaned a bit of info from different internet
> threads, but no solutions yet.
>
> I'm using Windows XP, SP3 and R 2.9.1.
> Here is my session info:
>
> R version 2.9.1 (2009-06-26)
> i286-pc-mingw32
>
> locale:
> LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
> States. 1252;LC_MONETARY=English_United
> States. 1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> I'm using the vanilla load of R, with nothing added after booting up.
> The problem also occurs on my Vista machine as well.
>
> The program below will induce the problem.
> ------------------------------------------------------------
> junk = 1
> PathA = tempdir()
> conX = paste(PathA,"junk",sep="\\")
> outJunk = file(conX, open="wb")
> save(junk, file=outJunk)
> close(outJunk)
> for(i in 1:4000){
> outMIA = file(conX, open="rb")
> load(file=outMIA)
> close(outMIA)
> closeAllConnections()
> unlink(conX)
> rm(outMIA)
> rm(junk)
> cat(" i = ",i,sep=" ")
> gc()
> zzz = showConnections(all=FALSE)
> cat(" zzz = ",zzz,"\n",sep=" ")
> }
> --------------------------------------------------------------------
> There is some talk on the internet that some windows systems have a
> limit of 512 files that can be open at one time. Even though I'm closing
> my files each time, something is keeping track of how many times I've
> opened and closed a file in a session. I've talked to Microsoft and run
> a test program in Visual Studio C#, and, at the moment, it looks like
> the problem does not lie in the Microsoft arena. The C# program
> performed a similar task 10,000 times without a problem. I'm not totally
> convinced, but the current evidence says to look elsewhere.
>
> I've attached a script that will induce the problem. However, be warned
> that, if you use it, you will have to get out of R after you run it. R
> will no longer be able to access files such as help or sessionInfo().
> This can be solved by getting out of the R GUI and back in.
>
> R E-mails from as far back as 2006 ask for help on the issue. There have
> been several suggestions, but no confirmed solution that I can find. You
> will see my attempt at these suggestions in the script [ rm(outMIA);
> rm(junk); closeAllConnections(); and gc(); after close(outMIA); and
> unlink(conX);]. For me, this becomes important because it limits the
> total number if iterations in nested do-loops. This is where I ran into
> the problem. The program above will allow you to reproduce the problem.
>
> Any suggestion would be greatly appreciated.
This looks like a bug to me, and so I've posted it to the bugs list, but
your script doesn't make sense for normal use. Before the loop you
create a file named according to conX containing the junk variable, and
then within the loop you load that file, *then delete it*, using
unlink(). However, if you examine the value of unlink(), you'll see
that it fails every time.
Moreover, the use of connections is unnecessary: load() can work with a
filename. So you can skip the file() and close() calls.
If I simplify your script by removing the unnecessary stuff, then it
works. That is, this works:
junk = 1
PathA = tempdir()
conX = paste(PathA,"junk",sep="\\")
save(junk, file=conX)
for(i in 1:4000){
load(file=conX)
rm(junk)
cat(" i = ",i,sep=" ")
gc()
zzz = showConnections(all=FALSE)
cat(" zzz = ",zzz,"\n",sep=" ")
}
So a suggestion for a workaround is simply to follow the pattern above,
rather than that of your demonstration code.
I'll spend a bit more time and see if I can find what is causing the
open file leak.
Duncan Murdoch
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
