Jan, It's ok, calc works fine. I don't have problem when the raster data is small and I use the calc function. But with large raster data and multilayers (310), like the MOD13Q1 product, I need process the data in chunks, and for that I need use the function 'writeStart'. My problem is I need load 310 files into a "rasterStack" (no problem) and then I need to transform to "rasterBrick" that takes lot of time. I need to know if that is the best way to do it, or exist another way, more efficient in time and memory.
2013/5/8 Jan Verbesselt <[email protected]> > Dear Francisco, > > Have a look at the calc() function for raster bricks (save to disk), it > works well for accessing all layers per pixel. > > Best, > Jan > > > > On Wed, May 8, 2013 at 5:07 PM, Francisco Zambrano <[email protected]>wrote: > >> I need apply the function "lowess" and that (from what I understand) it >> take values from all layers of the stack at the same time. But, "lapply or >> loop" run function for each layer not for one cell and all the layers, And >> is for that I'm think need a rasterBrick instead a rasterStack >> >> The code is something like: >> >> out<-stack(list.files(pattern='*.tiff$) >> filename<-rasterTmpFile() >> >> >> out<-writeStart(out,filename,overwrite=TRUE) >> bs<-blockSize(out) >> pb<-pbCreate(bs$n) >> for (i in 1:bs$n){ >> v<-getValues(out,row=bs$row[i],nrows=bs$nrows[i]) >> * s<-t(sapply(apply(v,1,lowess,f=7/n,iter=10),"[[","y"))* >> >> out<-writeValues(out,s,bs$row[i]) >> pbStep(pb,i) >> } >> >> >> >> 2013/5/8 Nagle, Nicholas <[email protected]> >> >> > Stacks can be treated as lists of RasterLayers. So use writeStart on >> > lists of rasters. >> > >> > >> > Options to consider: >> > 1) You could either loop (or lapply) through list, modifying the >> original >> > TIFs with new data. >> > 2) Create a new stack (not in memory) of RasterLayers with empty values. >> > You can use one of the MODIS stack layers as a template for these new >> > layers, and then loop (lapply) through the stack, using writeStart on >> each >> > Layer, or >> > 3) create a new brick (not in memory!) with the necessary dimensions, >> and >> > then use writeStart on this with the calculated values. Again, you >> should >> > be able to use one of the stack layers as a template to create this >> brick. >> > >> > >> > Only method 1) will modify the original TIFs and put the new values >> > alongside the original MODIS values. I don't know how important it is >> to >> > duplicate the original MODIS pixel values along with whatever values you >> > are creating. >> > Method 2) will create lots of individual, manageable files. You could >> > modify the naming convention of the original MODIS files. >> > Method 3) will keep all of the new values together in one big file. >> > >> > Sometimes I am doing analysis for myself, and don't need to pass the >> > data along to someone else. In these instances, I often just save the >> new >> > values as R data.frames, and just inject subsets of the data.frame into >> > RasterLayers at the time of plotting. >> > >> > Nicholas >> > >> > >> > >> > >> > On May 8, 2013, at 9:23 AM, Francisco Zambrano <[email protected]> >> > wrote: >> > >> > Howdy Nicholas, >> > >> > Thanks for your answer. In the beginning I did the stack, but to make a >> > function using "writeStart", this function works with "rasterBrick" >> objects >> > and not with 'rasterStack'. That is the reason why I need to make a >> brick. >> > >> > I don't know what could be the most efficent solution. >> > >> > My solution in this moment is as follow: >> > >> > 1.- Create the 'rasterStack' >> > 2.- Save with 'writeRaster(rasterStack,filename) >> > 3.- Open it with "brick(filenanme)" >> > 4.- Use the function writeStart >> > >> > Thats is the better way or exist another option? >> > >> > >> > >> > 2013/5/8 Nagle, Nicholas <[email protected]> >> > >> >> Dear Francisco, >> >> >> >> I am working with the same time series of MOD09 data, but I leave it >> as a >> >> stack, rather than create the brick. >> >> Why do you need it as a brick? The problem with brick is that it needs >> >> the data to be in one file. That's a big file in this instance (Tb >> >> scale?). And, since you didn't specify a filename, R is going to try >> and >> >> do this in memory, and will need to create virtual memory for this. 11 >> >> hours doesn't sound unreasonable for this. >> >> >> >> Why not just leave it as a stack? I've found it to work pretty well. >> >> Otherwise, be sure to specify the filename so that R doesn't try to to >> >> this in memory. But that still might take time, and is probably a >> needless >> >> recopy of a large amount of data. >> >> >> >> Nicholas >> >> >> >> >> >> >> >> Nicholas Nagle, Assistant Professor >> >> Department of Geography >> >> Burchfiel Geography Building 307 >> >> University of Tennessee >> >> Knoxville, TN 37996 >> >> >> >> >> >> > Date: Tue, 7 May 2013 10:35:00 -0400 >> >> > From: Francisco Zambrano <[email protected]> >> >> > To: [email protected] >> >> > Subject: [R-sig-Geo] How to write functions with mutiple large raster >> >> > data file (MOD13Q1, 2000-2012) >> >> > Message-ID: >> >> > <CANE=RDO9N-WmnjKTbqnzkeFN= >> >> [email protected]> >> >> > Content-Type: text/plain >> >> > >> >> > Hi everybody, >> >> > >> >> > I'm working with MODIS data MOD13Q1 and the NDVI index between 2000- >> >> today. >> >> > I have been trying to do a function, but I have a problem when stack >> the >> >> > 310 files and then I tried to use the writeStart function. I had an >> >> error >> >> > because the startFuction work with rasterLayer or RasterBrick. But, >> >> when I >> >> > tried to make the brick with the rasterStack (>brick(rasterStack)) >> the >> >> > operation took too much time (11 hours) >> >> > >> >> > My question, is what is the most efficiently way to make a >> rasterBrick >> >> from >> >> > multiple raster (geoTIFF) files? >> >> > >> >> > save the rasterStack and then open it with brick? >> >> > >> >> > My code is something like: >> >> > >> >> >> out<-stack(list.files(pattern=*..tif$)) >> >> >> out<-brick(mod) #too much time (11 hours appox) >> >> > >> >> >> out<-writeStart(out,filename,overwrite=TRUE) >> >> >> bs<-blockSize(out) >> >> >> pb<-pbCreate(bs$n) >> >> >> for (i in 1:bs$n){ >> >> > v<-getValues(out,row=bs$row[i],nrows=bs$nrows[i]) >> >> > s<-t(sapply(apply(v,1,lowess,f=7/n,iter=10),"[[","y")) >> >> > out<-writeValues(out,s,bs$row[i]) >> >> > pbStep(pb,i) >> >> > } >> >> > >> >> > Kind Regards >> >> > >> >> > Francisco Zambrano Bigiarini >> >> > >> >> > from Chile, latin america >> >> >> >> >> > >> > >> > -- >> > El que te roba una idea se queda con un pedazo de tí, pero pierde dos >> > pedazos de sí: la imaginación y el honor >> > >> > >> > >> >> >> -- >> El que te roba una idea se queda con un pedazo de tí, pero pierde dos >> pedazos de sí: la imaginación y el honor >> >> [[alternative HTML version deleted]] >> >> >> _______________________________________________ >> R-sig-Geo mailing list >> [email protected] >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo >> >> > -- El que te roba una idea se queda con un pedazo de tí, pero pierde dos pedazos de sí: la imaginación y el honor [[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
