Jan, It's ok, calc works fine. I don't have problem when the raster data is
small and I use the calc function. But with large raster data and
multilayers (310), like the MOD13Q1 product, I need process the data in
chunks, and for that I need use the function 'writeStart'. My problem is I
need load 310 files into a "rasterStack" (no problem) and then I need to
transform to "rasterBrick" that takes lot of time. I need to know if that
is the best way to do it, or exist another way, more efficient in time and
memory.



2013/5/8 Jan Verbesselt <[email protected]>

> Dear Francisco,
>
> Have a look at the calc() function for raster bricks (save to disk), it
> works well for accessing all layers per pixel.
>
> Best,
> Jan
>
>
>
> On Wed, May 8, 2013 at 5:07 PM, Francisco Zambrano <[email protected]>wrote:
>
>> I need apply the function "lowess" and that (from what I understand) it
>> take values from all layers of the stack at the same time. But, "lapply or
>> loop" run function for each layer not for one cell and all the layers, And
>> is for that I'm think need a rasterBrick instead a rasterStack
>>
>> The code is something like:
>>
>>  out<-stack(list.files(pattern='*.tiff$)
>>  filename<-rasterTmpFile()
>>
>>
>>     out<-writeStart(out,filename,overwrite=TRUE)
>>     bs<-blockSize(out)
>>     pb<-pbCreate(bs$n)
>>     for (i in 1:bs$n){
>>       v<-getValues(out,row=bs$row[i],nrows=bs$nrows[i])
>> *      s<-t(sapply(apply(v,1,lowess,f=7/n,iter=10),"[[","y"))*
>>
>>       out<-writeValues(out,s,bs$row[i])
>>       pbStep(pb,i)
>>     }
>>
>>
>>
>> 2013/5/8 Nagle, Nicholas <[email protected]>
>>
>> >  Stacks can be treated as lists of RasterLayers.  So use writeStart on
>> > lists of rasters.
>> >
>> >
>> >  Options to consider:
>> > 1) You could either loop (or lapply) through list, modifying the
>> original
>> > TIFs with new data.
>> > 2) Create a new stack (not in memory) of RasterLayers with empty values.
>> >  You can use one of the MODIS stack layers as a template for these new
>> > layers, and then loop (lapply) through the stack, using writeStart on
>> each
>> > Layer, or
>> > 3) create a new brick (not in memory!) with the necessary dimensions,
>> and
>> > then use writeStart on this with the calculated values.  Again, you
>> should
>> > be able to use one of the stack layers as a template to create this
>> brick.
>> >
>> >
>> >  Only method 1) will modify the original TIFs and put the new values
>> > alongside the original MODIS values.  I don't know how important it is
>> to
>> > duplicate the original MODIS pixel values along with whatever values you
>> > are creating.
>> > Method 2) will create lots of individual, manageable files.  You could
>> > modify the naming convention of the original MODIS files.
>> > Method 3) will keep all of the new values together in one big file.
>> >
>> >  Sometimes I am doing analysis for myself, and don't need to pass the
>> > data along to someone else.  In these instances, I often just save the
>> new
>> > values as R data.frames, and just inject subsets of the data.frame into
>> > RasterLayers at the time of plotting.
>> >
>> >  Nicholas
>> >
>> >
>> >
>> >
>> >  On May 8, 2013, at 9:23 AM, Francisco Zambrano <[email protected]>
>> >  wrote:
>> >
>> >  Howdy Nicholas,
>> >
>> >  Thanks for your answer. In the beginning I did the stack, but to make a
>> > function using "writeStart", this function works with "rasterBrick"
>> objects
>> > and not with 'rasterStack'. That is the reason why I need to make a
>> brick.
>> >
>> >  I don't know what could be the most efficent solution.
>> >
>> >  My solution in this moment is as follow:
>> >
>> >  1.- Create the 'rasterStack'
>> > 2.- Save with 'writeRaster(rasterStack,filename)
>> > 3.- Open it with "brick(filenanme)"
>> > 4.- Use the function writeStart
>> >
>> >  Thats is the better way or exist another option?
>> >
>> >
>> >
>> > 2013/5/8 Nagle, Nicholas <[email protected]>
>> >
>> >> Dear Francisco,
>> >>
>> >> I am working with the same time series of MOD09 data, but I leave it
>> as a
>> >> stack, rather than create the brick.
>> >> Why do you need it as a brick?  The problem with brick is that it needs
>> >> the data to be in one file.  That's a big file in this instance (Tb
>> >> scale?).  And, since you didn't specify a filename, R is going to try
>> and
>> >> do this in memory, and will need to create virtual memory for this.  11
>> >> hours doesn't sound unreasonable for this.
>> >>
>> >> Why not just leave it as a stack?  I've found it to work pretty well.
>> >>  Otherwise, be sure to specify the filename so that R doesn't try to to
>> >> this in memory.  But that still might take time, and is probably a
>> needless
>> >> recopy of a large amount of data.
>> >>
>> >> Nicholas
>> >>
>> >>
>> >>
>> >> Nicholas Nagle, Assistant Professor
>> >> Department of Geography
>> >> Burchfiel Geography Building 307
>> >> University of Tennessee
>> >> Knoxville, TN 37996
>> >>
>> >>
>> >> > Date: Tue, 7 May 2013 10:35:00 -0400
>> >> > From: Francisco Zambrano <[email protected]>
>> >> > To: [email protected]
>> >> > Subject: [R-sig-Geo] How to write functions with mutiple large raster
>> >> >       data    file (MOD13Q1, 2000-2012)
>> >> > Message-ID:
>> >> >       <CANE=RDO9N-WmnjKTbqnzkeFN=
>> >> [email protected]>
>> >> > Content-Type: text/plain
>> >> >
>> >> > Hi everybody,
>> >> >
>> >> > I'm working with MODIS data MOD13Q1 and the NDVI index between 2000-
>> >> today.
>> >> > I have been trying to do a function, but I have a problem when stack
>> the
>> >> > 310 files and then I tried to use the writeStart function. I had an
>> >> error
>> >> > because the startFuction work with rasterLayer or RasterBrick. But,
>> >> when I
>> >> > tried to make the brick with the rasterStack (>brick(rasterStack))
>> the
>> >> > operation took too much time (11 hours)
>> >> >
>> >> > My question, is what is the most efficiently way to make a
>> rasterBrick
>> >> from
>> >> > multiple raster (geoTIFF) files?
>> >> >
>> >> > save the rasterStack and then open it with brick?
>> >> >
>> >> > My code is something like:
>> >> >
>> >> >> out<-stack(list.files(pattern=*..tif$))
>> >> >> out<-brick(mod) #too much time (11 hours appox)
>> >> >
>> >> >> out<-writeStart(out,filename,overwrite=TRUE)
>> >> >> bs<-blockSize(out)
>> >> >> pb<-pbCreate(bs$n)
>> >> >> for (i in 1:bs$n){
>> >> >    v<-getValues(out,row=bs$row[i],nrows=bs$nrows[i])
>> >> >    s<-t(sapply(apply(v,1,lowess,f=7/n,iter=10),"[[","y"))
>> >> >    out<-writeValues(out,s,bs$row[i])
>> >> >   pbStep(pb,i)
>> >> >  }
>> >> >
>> >> > Kind Regards
>> >> >
>> >> > Francisco Zambrano Bigiarini
>> >> >
>> >> > from Chile, latin america
>> >>
>> >>
>> >
>> >
>> >  --
>> > “El que te roba una idea se queda con un pedazo de tí, pero pierde dos
>> > pedazos de sí: la imaginación y el honor”
>> >
>> >
>> >
>>
>>
>> --
>> “El que te roba una idea se queda con un pedazo de tí, pero pierde dos
>> pedazos de sí: la imaginación y el honor”
>>
>>         [[alternative HTML version deleted]]
>>
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> [email protected]
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
>


-- 
“El que te roba una idea se queda con un pedazo de tí, pero pierde dos
pedazos de sí: la imaginación y el honor”

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Reply via email to