Hello Simon,
The gc() solution after loading each sheet works.
Thanks!
Luca
Il giorno 13/ott/2011, alle ore 17.05, Simon Urbanek ha scritto:
> Luca,
>
> On Oct 13, 2011, at 5:45 AM, Luca Meyer wrote:
>
>> Hi,
>>
>> I have to upload data from more than 200 separated excel pages and I am
>> using the read.xlsx function in the xlsx package. Each sheet is an
>> articulated page (made of more than one table plus extra data) and I need to
>> load into a R data frame the different elements I find in each page. This
>> procedure needs to be repeated several times - i.e. on more than one excel
>> file.
>>
>> Initially I did not have problems and the script run just fine providing the
>> desired data frame. After running a few hundreds sheets now I get this error
>> after I upload each page:
>>
>> Errore in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl, :
>> java.lang.OutOfMemoryError: Java heap space
>>
>> Has this to do with some sort of cache that is being filled up? Can anyone
>> suggest a solution to this error?
>>
>
> Well, you're running out of memory on the Java side. First, did you run gc()?
> You should make sure that you don't keep any references around unnecessarily,
> since memory cannot be released until they are collected.
>
> I did run a quick test with rJava memory profiling enabled and I see no leaks
> in rJava:
>
> ginaz:~$ R -e 'library(xlsx); d=read.xlsx("/Users/urbanek/Downloads/Acme
> Coffin Company.xls",1); gc()' | ./mem.match
> Loading required package: xlsxjars
> Loading required package: rJava
>
> SUM[Leaked objects]: 0
> SUM[Used objects]: 253
>
> So if you see any issues even after running gc(), it will be hard to trace.
> It could be in the Java code used by xlsx (which I don't know how to trace)
> or in theory some local places in the infrastructure that are not covered by
> the memory traces (the above counts Java objects referenced from R). You may
> have some luck using java.lang.management to look into Java usage. Also you
> can force Java to run its own garbage collector using
> .jcall("java/lang/System",,"gc")
> but I doubt that it will help, since Java would have run it before running
> out of memory.
>
> Finally, you could increase the Java heap if you have enough memory - for
> example to use 2Gb:
>
> library(rJava)
> .jinit(parameters="-Xmx2g")
>
> but that's just delaying the problem.
>
> So, again, make sure you run gc() after you're done with each worksheet and
> see if the problem persists.
>
> Cheers,
> Simon
>
>
_______________________________________________
R-SIG-Mac mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
