Hi Robert, Many thanks for the answer. I tried this again using "median" rather than "Median", and got the following results on my example:
setOptions(todisk = TRUE) # I used this option to rule out memory issues system.time(med <- calc(stk, median, filename = "med.tif", datatype = "INT2S", overwrite = T)) # user system elapsed #108.856 1.158 115.998 system.time(med <- calc(stk, mean, filename = "mean.tif", datatype = "INT2S", overwrite = T)) setOptions(todisk = FALSE) # user system elapsed # 1.232 0.336 1.663 So slower than mean, of course, but I can now get a result on my larger rasters. Thanks again, Lyndon On Fri, Jul 22, 2011 at 12:50 PM, Robert Hijmans <[email protected]> wrote: >> I am trying to extract the per-pixel median from a raster stack, and >> am finding that the process takes a very long time relative to taking >> the per-pixel mean. It is long enough that I have never actually seen >> whether it successfully completes on my data, but this dummy example >> shows the code structure I am using and the time differences between >> the two functions: >> >> stk <- stack(lapply(1:10, function(x) { >> r <- raster(nrow = 1000, ncol = 1000) >> r2 <- setValues(r, sample(1:100, size = ncell(r), replace = T, prob = >> NULL)) >> })) >> >> setOptions(todisk = TRUE) # I used this option to rule out memory issues >> t1 <- Sys.time() >> med <- calc(stk, Median, filename = "med.tif", datatype = "INT2S", >> overwrite = T) >> Sys.time() - t1 # 7.26 mins >> >> t2 <- Sys.time() >> med <- calc(stk, mean, filename = "mean.tif", datatype = "INT2S", >> overwrite = T) >> Sys.time() - t2 # 2.34 secs >> setOptions(todisk = FALSE) >> >> >> Am I doing something wrong with my code, or is the speed difference I >> am seeing expected? >> > > Lyndon, > > Median (big M) was intended for use like Median(s) not for use in calc > (which did not exist yet); but I am surprised how slow it is in calc. Now we > have calc, you can use median (small m) and that is much quicker; and about > the same as Median(s); I think I will remove Median as it is no longer > needed. > > Either way, median will be slower than mean, which is expected: > >> v = runif(1e+7) >> system.time(mean(v)) > user system elapsed > 0.09 0.00 0.09 >> system.time(median(v)) # 7 times slower > user system elapsed > 0.56 0.06 0.63 >> >> s <- stack(sapply(1:10, function(x) setValues(r, runif(ncell(r))))) >> r <- raster() >> >> system.time(mean(s)) > user system elapsed > 4.75 0.02 4.79 >> system.time(calc(s, mean)) > user system elapsed > 0.25 0.00 0.26 > >> system.time(Median(s)) > user system elapsed > 7.73 0.00 7.75 >> system.time(calc(s, median)) > user system elapsed > 8.48 0.00 8.54 >> system.time(calc(s, Median)) # slow! > user system elapsed > 28.15 0.03 28.27 >> >> > > > Robert > > > > > -- > View this message in context: > http://r-sig-geo.2731867.n2.nabble.com/raster-Median-function-very-slow-on-stack-compared-to-mean-expected-behavior-tp6609202p6611150.html > Sent from the R-sig-geo mailing list archive at Nabble.com. > > _______________________________________________ > R-sig-Geo mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > _______________________________________________ R-sig-Geo mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
