Re: [R] Rprof causing R to crash
Hi there, Thanks for the quick feedback. There is no further info on the reason for the crash - R just stops. The for loop is there just to generate the error. I'm following a previous example that others couldn't reproduce. I need Rprof to work on a different set of functions that I haven't posted. Those function do involve some large arrays, but have never given any trouble away from Rprof. I'm pretty sure Rprof is broken. Is there an alternative? Cheers Steven Sent from phone. On Feb 4, 2013 5:03 PM, jim holtman jholt...@gmail.com wrote: Could you be a little more explicit of what the error message is. On my Windows system 32-bit, I get the following: Rprof() z - 1 system.time(for (i in 1:1e8) z - z + 1/i) Error: cannot allocate vector of size 381.5 Mb Which is probably due to trying to allocate 1e8 elements of integers (400MB). Could you have a similar problem? Now you could restructure your code to not allocate the large vector by: Rprof() z - 1 i - 0 system.time(while((i - i + 1) 1e8) z - z + 1/i) z Rprof(NULL) On Mon, Feb 4, 2013 at 7:48 AM, c97sr s...@stevenriley.net wrote: Am also having trouble with Rprof. I am using on R 2.15.2 on the Mac (Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)). The example above without profiling takes around a minute to run: z = 1 for (i in 1:1e8) z = z+1/i z [1] 19.9979 With profiling: Rprof() z = 1 for (i in 1:1e8) z = z+1/i [INFO] Feb 4, 2013 12:45:20 PM - R stopped. Crashes within a second or so. This is consistent with the problem I'm having in my own code in which there is a reasonable amount of memory used (but not a massive amount). Any help much appreciated. Am happy to post this as a bug if that is now appropriate. regards Steven -- View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657482.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Jim Holtman Data Munger Guru What is the problem that you are trying to solve? Tell me what you want to do, not how you want to do it. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Rprof causing R to crash
Hi Jim, Could you be a little more explicit of what the error message is. On my Windows system 32-bit, I get the following: Sorry - it just stops as per the message posted. The text is from eclipse. Rprof() z - 1 system.time(for (i in 1:1e8) z - z + 1/i) Error: cannot allocate vector of size 381.5 Mb Which is probably due to trying to allocate 1e8 elements of integers (400MB). Could you have a similar problem? Shouldn't be. 400MB shouldn't be a problem on a 4GB machine? Just tried this with no problem on 2.15.0 on Platform: x86_64-redhat-linux-gnu (64-bit), so it looks to be particular to my setup on the mac. I also just tried 2.15.2 on the mac using the GUI and _not_ eclipse, It worked with no problem. Therefore, it seems that my issues are specific to R 2.15.2 (and R 2.14.0 prior to upgrade) running via StatEt's rj package (with no additional arguments.) I have a workaround now, and I guess this is no longer an issue for the wider group. Will post a link to the StatEt group. cheers Steven Now you could restructure your code to not allocate the large vector by: Rprof() z - 1 i - 0 system.time(while((i - i + 1) 1e8) z - z + 1/i) z Rprof(NULL) Yes - the example is not related to my code, its just a way of recreating a similar error. I haven't posted my code that causes the error. Is there a reason that 400 MB should be a problem for RProf? My previous experience is that Rprof is stable with processes with much larger memory footprints. Hence why I think the for loop error might be related to the problems I'm having. Is there an alternative to Rprof? This is fairly crucial for me right now. Will try to get setup on cheers Steven On Mon, Feb 4, 2013 at 7:48 AM, c97sr lt;sr@gt; wrote: Am also having trouble with Rprof. I am using on R 2.15.2 on the Mac (Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)). The example above without profiling takes around a minute to run: z = 1 for (i in 1:1e8) z = z+1/i z [1] 19.9979 With profiling: Rprof() z = 1 for (i in 1:1e8) z = z+1/i [INFO] Feb 4, 2013 12:45:20 PM - R stopped. Crashes within a second or so. This is consistent with the problem I'm having in my own code in which there is a reasonable amount of memory used (but not a massive amount). Any help much appreciated. Am happy to post this as a bug if that is now appropriate. regards Steven -- View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657482.html Sent from the R help mailing list archive at Nabble.com. __ R-help@ mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Jim Holtman Data Munger Guru What is the problem that you are trying to solve? Tell me what you want to do, not how you want to do it. __ R-help@ mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657545.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Rprof causing R to crash
Am also having trouble with Rprof. I am using on R 2.15.2 on the Mac (Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)). The example above without profiling takes around a minute to run: z = 1 for (i in 1:1e8) z = z+1/i z [1] 19.9979 With profiling: Rprof() z = 1 for (i in 1:1e8) z = z+1/i [INFO] Feb 4, 2013 12:45:20 PM - R stopped. Crashes within a second or so. This is consistent with the problem I'm having in my own code in which there is a reasonable amount of memory used (but not a massive amount). Any help much appreciated. Am happy to post this as a bug if that is now appropriate. regards Steven -- View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657482.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Rprof causing R to crash
Could you be a little more explicit of what the error message is. On my Windows system 32-bit, I get the following: Rprof() z - 1 system.time(for (i in 1:1e8) z - z + 1/i) Error: cannot allocate vector of size 381.5 Mb Which is probably due to trying to allocate 1e8 elements of integers (400MB). Could you have a similar problem? Now you could restructure your code to not allocate the large vector by: Rprof() z - 1 i - 0 system.time(while((i - i + 1) 1e8) z - z + 1/i) z Rprof(NULL) On Mon, Feb 4, 2013 at 7:48 AM, c97sr s...@stevenriley.net wrote: Am also having trouble with Rprof. I am using on R 2.15.2 on the Mac (Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)). The example above without profiling takes around a minute to run: z = 1 for (i in 1:1e8) z = z+1/i z [1] 19.9979 With profiling: Rprof() z = 1 for (i in 1:1e8) z = z+1/i [INFO] Feb 4, 2013 12:45:20 PM - R stopped. Crashes within a second or so. This is consistent with the problem I'm having in my own code in which there is a reasonable amount of memory used (but not a massive amount). Any help much appreciated. Am happy to post this as a bug if that is now appropriate. regards Steven -- View this message in context: http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846p4657482.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Jim Holtman Data Munger Guru What is the problem that you are trying to solve? Tell me what you want to do, not how you want to do it. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Rprof causing R to crash
Uwe,
I am unfortunately not able to upgrade to R 2.15.2 right now, but I have
seen a similar problem with several older R versions. If you want to
test with a shorter script, you can try the lines below. These provoke a
crash from a fresh R session on my machine (R 2.15.1 Windows 7):
Rprof()
z = 1
for (i in 1:1e8) z = z+1/i
This runs without problems without the call to Rprof(). I was far from
using all my memory with this call, but you could also check with a
higher number of loops in case you can run these lines and it is somehow
memory related. Just for comparison, I tried 1e9 loops after a call to
Rprof() on one of our Linux servers (R 2.14.0) without any problems.
Jon
On 12-Dec-12 17:06, Uwe Ligges wrote:
On 12.12.2012 00:05, Marian Talbert wrote:
I'm trying to use Rprof() to identify bottlenecks and speed up a
particullary
slow section of code which reads in a portion of a tif file and compares
each of the values to values of predictors used for model fitting. I've
written up an example that anyone can run. Generally temp would be a
section of a tif read into a data.frame and used later for other
processing.
The first portion which just records the time works in about 6
seconds the
second part causes RGui to immediately close with no error or
warning. Any
advice on how to get Rprof to work or how to speed up this code would be
greatly appreciated. I'm using Windows 7 (which might be my problem)
and R
version 2.15.0.
The problem is rather the R version: I cannot reproduce errors with a
recent R.
Uwe Ligges
CalcMESS-function(tiff.entry,pred.vect){
f-sum(pred.vecttiff.entry)/length(pred.vect)*100
if(is.na(f)) return(NA)
if(f==0)
return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
if(0f f=50) return(2*f)
if(50=f f100) return(2*(100-f))
if(f==100)
return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
else return(NA)
}
train.dat -
data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
temp -
data.frame(a=runif(13),b=runif(13),c=runif(13),d=runif(13))
pred.rng-temp
vnames.final.mod - names(train.dat)
nvars.final - length(vnames.final.mod)
start.time-Sys.time()
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Sys.time()-start.time
Rprof(C:\\temp\\mapply.out)
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Rprof(NULL)
--
View this message in context:
http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Rprof causing R to crash
On 14.12.2012 14:22, Jon Olav Skoien wrote:
Uwe,
I am unfortunately not able to upgrade to R 2.15.2 right now, but I have
seen a similar problem with several older R versions. If you want to
test with a shorter script, you can try the lines below. These provoke a
crash from a fresh R session on my machine (R 2.15.1 Windows 7):
Rprof()
z = 1
for (i in 1:1e8) z = z+1/i
This runs without problems without the call to Rprof(). I was far from
using all my memory with this call, but you could also check with a
higher number of loops in case you can run these lines and it is somehow
memory related. Just for comparison, I tried 1e9 loops after a call to
Rprof() on one of our Linux servers (R 2.14.0) without any problems.
Works for me.
Uwe
Jon
On 12-Dec-12 17:06, Uwe Ligges wrote:
On 12.12.2012 00:05, Marian Talbert wrote:
I'm trying to use Rprof() to identify bottlenecks and speed up a
particullary
slow section of code which reads in a portion of a tif file and compares
each of the values to values of predictors used for model fitting. I've
written up an example that anyone can run. Generally temp would be a
section of a tif read into a data.frame and used later for other
processing.
The first portion which just records the time works in about 6
seconds the
second part causes RGui to immediately close with no error or
warning. Any
advice on how to get Rprof to work or how to speed up this code would be
greatly appreciated. I'm using Windows 7 (which might be my problem)
and R
version 2.15.0.
The problem is rather the R version: I cannot reproduce errors with a
recent R.
Uwe Ligges
CalcMESS-function(tiff.entry,pred.vect){
f-sum(pred.vecttiff.entry)/length(pred.vect)*100
if(is.na(f)) return(NA)
if(f==0)
return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
if(0f f=50) return(2*f)
if(50=f f100) return(2*(100-f))
if(f==100)
return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
else return(NA)
}
train.dat -
data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
temp -
data.frame(a=runif(13),b=runif(13),c=runif(13),d=runif(13))
pred.rng-temp
vnames.final.mod - names(train.dat)
nvars.final - length(vnames.final.mod)
start.time-Sys.time()
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Sys.time()-start.time
Rprof(C:\\temp\\mapply.out)
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Rprof(NULL)
--
View this message in context:
http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Rprof causing R to crash
On 14/12/2012 13:22, Jon Olav Skoien wrote:
Uwe,
I am unfortunately not able to upgrade to R 2.15.2 right now, but I have
Why not? Note that is part of the R-help contract: we only offer any
support for the current version of R (see the posting guide).
The posting guide also asked for 'at a minimum' information: we do not
know (amongst other things) if this is 32- or 64-bit R. (Your example
works for me in both.)
seen a similar problem with several older R versions. If you want to
test with a shorter script, you can try the lines below. These provoke a
crash from a fresh R session on my machine (R 2.15.1 Windows 7):
Rprof()
z = 1
for (i in 1:1e8) z = z+1/i
This runs without problems without the call to Rprof(). I was far from
using all my memory with this call, but you could also check with a
higher number of loops in case you can run these lines and it is somehow
memory related. Just for comparison, I tried 1e9 loops after a call to
Rprof() on one of our Linux servers (R 2.14.0) without any problems.
Which is irrelevant: a different version of R and a completely different
implementation of Rprof.
Jon
On 12-Dec-12 17:06, Uwe Ligges wrote:
On 12.12.2012 00:05, Marian Talbert wrote:
I'm trying to use Rprof() to identify bottlenecks and speed up a
particullary
slow section of code which reads in a portion of a tif file and compares
each of the values to values of predictors used for model fitting. I've
written up an example that anyone can run. Generally temp would be a
section of a tif read into a data.frame and used later for other
processing.
The first portion which just records the time works in about 6
seconds the
second part causes RGui to immediately close with no error or
warning. Any
advice on how to get Rprof to work or how to speed up this code would be
greatly appreciated. I'm using Windows 7 (which might be my problem)
and R
version 2.15.0.
The problem is rather the R version: I cannot reproduce errors with a
recent R.
Uwe Ligges
CalcMESS-function(tiff.entry,pred.vect){
f-sum(pred.vecttiff.entry)/length(pred.vect)*100
if(is.na(f)) return(NA)
if(f==0)
return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
if(0f f=50) return(2*f)
if(50=f f100) return(2*(100-f))
if(f==100)
return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
else return(NA)
}
train.dat -
data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
temp -
data.frame(a=runif(13),b=runif(13),c=runif(13),d=runif(13))
pred.rng-temp
vnames.final.mod - names(train.dat)
nvars.final - length(vnames.final.mod)
start.time-Sys.time()
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Sys.time()-start.time
Rprof(C:\\temp\\mapply.out)
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Rprof(NULL)
--
View this message in context:
http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Rprof causing R to crash
On 12.12.2012 00:05, Marian Talbert wrote:
I'm trying to use Rprof() to identify bottlenecks and speed up a particullary
slow section of code which reads in a portion of a tif file and compares
each of the values to values of predictors used for model fitting. I've
written up an example that anyone can run. Generally temp would be a
section of a tif read into a data.frame and used later for other processing.
The first portion which just records the time works in about 6 seconds the
second part causes RGui to immediately close with no error or warning. Any
advice on how to get Rprof to work or how to speed up this code would be
greatly appreciated. I'm using Windows 7 (which might be my problem) and R
version 2.15.0.
The problem is rather the R version: I cannot reproduce errors with a
recent R.
Uwe Ligges
CalcMESS-function(tiff.entry,pred.vect){
f-sum(pred.vecttiff.entry)/length(pred.vect)*100
if(is.na(f)) return(NA)
if(f==0)
return((tiff.entry-min(pred.vect))/(max(pred.vect)-min(pred.vect))*100)
if(0f f=50) return(2*f)
if(50=f f100) return(2*(100-f))
if(f==100)
return((max(pred.vect)-tiff.entry)/(max(pred.vect)-min(pred.vect))*100)
else return(NA)
}
train.dat - data.frame(a=runif(200),b=runif(200),c=runif(200),d=runif(200))
temp -
data.frame(a=runif(13),b=runif(13),c=runif(13),d=runif(13))
pred.rng-temp
vnames.final.mod - names(train.dat)
nvars.final - length(vnames.final.mod)
start.time-Sys.time()
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Sys.time()-start.time
Rprof(C:\\temp\\mapply.out)
for(k in 1:nvars.final){
pred.range-train.dat[,match(vnames.final.mod[k],names(train.dat))]
pred.rng[,k]-mapply(CalcMESS,tiff.entry=temp[,match(vnames.final.mod[k],names(temp))],MoreArgs=list(pred.vect=pred.range))
}
Rprof(NULL)
--
View this message in context:
http://r.789695.n4.nabble.com/Rprof-causing-R-to-crash-tp4652846.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
