Re: [R-sig-Geo] stack many files without loading into memory

2015-02-25 Thread dschneiderch
Just wanted to update this thread in case anyone else comes looking. Some of these things were not immediately clear to me. I ended up doing: library(raster) library(ncdf4) fn=list.files('serverpath') fnstack=stack(fn) layerdates=names(fnstack) #instead of writeRaster, use ncdf4 directly to get

Re: [R-sig-Geo] stack many files without loading into memory

2015-02-06 Thread dschneiderch
Ok - Looks like it worked this time for 112 files from 2012. The netcdf is 2.25 GB while the compressed multiband geotiff is 510MB. Does the netcdf have so much overhead- the 112 file at 10MB each are only 1.12 GB individually? I like the tidiness of 1 file per year so I'll have to play with how

Re: [R-sig-Geo] stack many files without loading into memory

2015-02-06 Thread dschneiderch
Hi Michael - Yes saving as GTiff with the compression options reduced the file size from ~10MB to ~2.5M for a single file but I am having a lot of trouble getting it to save the whole stack. I'm definitely running out of memory on my computer so maybe R is being slow and timing out? I've left it

[R-sig-Geo] stack many files without loading into memory

2015-02-04 Thread Dominik Schneider
Hi - I have some data on a server but would like to bring them local in a somewhat compressed format that is still easy to access. /Volumes/hD/2012 - 100 geotiffs ~/project/data/ - store those geotiffs here without needing server access. untested, I think I could do something like: s=stack()

Re: [R-sig-Geo] stack many files without loading into memory

2015-02-04 Thread Dominik Schneider
Wouldn't that keep the link to the server on which they are stored now? Dominik Schneider o 303.735.6296 | c 518.956.3978 On Wed, Feb 4, 2015 at 12:50 PM, Michael Sumner mdsum...@gmail.com wrote: Why not stack(fn) ? On Thu, 5 Feb 2015 06:41 Dominik Schneider dominik.schnei...@colorado.edu

Re: [R-sig-Geo] stack many files without loading into memory

2015-02-04 Thread Michael Sumner
Why not stack(fn) ? On Thu, 5 Feb 2015 06:41 Dominik Schneider dominik.schnei...@colorado.edu wrote: Hi - I have some data on a server but would like to bring them local in a somewhat compressed format that is still easy to access. /Volumes/hD/2012 - 100 geotiffs ~/project/data/ - store

Re: [R-sig-Geo] stack many files without loading into memory

2015-02-04 Thread Michael Sumner
On Thu Feb 05 2015 at 7:41:33 AM Dominik Schneider dominik.schnei...@colorado.edu wrote: I think you are correct. s=stack(fn,quick=T) writeRaster(s,'localpath/2012data') Ugh, sorry yes that's me reading too fast. I should have suggested the next step to writeRaster, I'm not sure why you