Re: [GRASS-user] raster exchange between GRASS and R with nodata
I think I discovered the problem. This was a mistake on my side: The summary() function in R applied on a raster does subsample the data in case of very large raster maps. However, this seems to cause that summary information about NA gets lost and no NA are mentioned in the summary output although there are NA included in the raster map. Here an example output of a map with lots of NA's which are ignored by summary(rastermap): summary(streams[[1]]) Elbe_distance_from_mouth Min. 0. 1st Qu. 488.7067 Median 789.8560 3rd Qu.1025.4501 Max. 1876.6429 NA's 0. Warning message: In .local(object, ...) : summary is an estimate based on a sample of 1e+05 cells (10.7% of all cells) length(bioclim[[1]][is.na(bioclim[[1]])]) [1] 12688 So there was actually no problem, just some confusion concerning the information about NA in the summary. Best regards, Johannes On Thu, Sep 4, 2014 at 6:26 PM, Robert J. Hijmans r.hijm...@gmail.com wrote: Johannes, raster uses GDAL to access GTiff data. So it seems that there is no, or at least GDAL does not see, a no data value in the file. I would suggest running GDALinfo to troubleshoot. Can you email me a small example file? By the way, the spgrass6 route has a shortcoming: it may not work with very large files as it attempts to load all values into RAM. Robert On Thu, Sep 4, 2014 at 6:47 AM, Johannes Radinger johannesradin...@gmail.com wrote: On Thu, Sep 4, 2014 at 3:38 PM, Nuno Sá nunocesard...@gmail.com wrote: Hello! Did you try this one? r.out.gdal etc nodata='NA' As mentioned in the manual of r.out.gdal, the no data parameter takes only float values and no strings like 'NA'. Without stating as specific value in GRASS, this nodata-value is automatically set to e.g. 65535 for DCELL rasters if I remember correctly and to 255 for BYTE rasters. However, this seems not to be recognized when imported into R with the package 'raster'. /Johannes On 4 September 2014 14:27, Johannes Radinger johannesradin...@gmail.com wrote: Hi all, of course it is possible to load the raster maps directly via spgrass6. However, we use this work flow also to exchange some of the maps between different users (e.g. via email) and to permanently store single files (geotiffs that contain the proj information within the file). So, I agree that using spgrass6 would be much more efficient, but I'll stick to exporting to geotiffs and so I need to solve the issues with NA's. /Johannes On Thu, Sep 4, 2014 at 2:31 PM, Thomas Adams tea...@gmail.com wrote: Johannes, If you want to read your file into R, there is no need to export your map from GRASS to do this. Simply install and use the R contributed package 'spgrass6' (spgrass6 has R dependencies that need to be installed first); it works wonderfully: Within GRASS, at the GRASS terminal prompt... library(spgrass6) Loading required package: sp Loading required package: XML GRASS GIS interface loaded with GRASS version: GRASS 7.0.0beta3 (2014) and location: ohrfc_mpe dat-readRAST6(xmrg0101200306z) image(dat) This is far more efficient. Tom On Thu, Sep 4, 2014 at 5:32 AM, Johannes Radinger johannesradin...@gmail.com wrote: Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org
[GRASS-user] raster exchange between GRASS and R with nodata
Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] raster exchange between GRASS and R with nodata
You probably have good reasons for your work flow, but just to make sure: is there a specific reason to export to geotiff first? It seems easier to import the raster layer directly from GRASS into R using the spgrass6 package (readRAST6)? You can subsequently convert the spatial raster to the raster format. On Thu, Sep 4, 2014 at 11:32 AM, Johannes Radinger johannesradin...@gmail.com wrote: Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] raster exchange between GRASS and R with nodata
Johannes, If you want to read your file into R, there is no need to export your map from GRASS to do this. Simply install and use the R contributed package 'spgrass6' (spgrass6 has R dependencies that need to be installed first); it works wonderfully: Within GRASS, at the GRASS terminal prompt... library(spgrass6) Loading required package: sp Loading required package: XML GRASS GIS interface loaded with GRASS version: GRASS 7.0.0beta3 (2014) and location: ohrfc_mpe dat-readRAST6(xmrg0101200306z) image(dat) This is far more efficient. Tom On Thu, Sep 4, 2014 at 5:32 AM, Johannes Radinger johannesradin...@gmail.com wrote: Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] raster exchange between GRASS and R with nodata
Hi all, of course it is possible to load the raster maps directly via spgrass6. However, we use this work flow also to exchange some of the maps between different users (e.g. via email) and to permanently store single files (geotiffs that contain the proj information within the file). So, I agree that using spgrass6 would be much more efficient, but I'll stick to exporting to geotiffs and so I need to solve the issues with NA's. /Johannes On Thu, Sep 4, 2014 at 2:31 PM, Thomas Adams tea...@gmail.com wrote: Johannes, If you want to read your file into R, there is no need to export your map from GRASS to do this. Simply install and use the R contributed package 'spgrass6' (spgrass6 has R dependencies that need to be installed first); it works wonderfully: Within GRASS, at the GRASS terminal prompt... library(spgrass6) Loading required package: sp Loading required package: XML GRASS GIS interface loaded with GRASS version: GRASS 7.0.0beta3 (2014) and location: ohrfc_mpe dat-readRAST6(xmrg0101200306z) image(dat) This is far more efficient. Tom On Thu, Sep 4, 2014 at 5:32 AM, Johannes Radinger johannesradin...@gmail.com wrote: Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] raster exchange between GRASS and R with nodata
Hello! Did you try this one? *r.out.gdal etc nodata*=*'NA'* On 4 September 2014 14:27, Johannes Radinger johannesradin...@gmail.com wrote: Hi all, of course it is possible to load the raster maps directly via spgrass6. However, we use this work flow also to exchange some of the maps between different users (e.g. via email) and to permanently store single files (geotiffs that contain the proj information within the file). So, I agree that using spgrass6 would be much more efficient, but I'll stick to exporting to geotiffs and so I need to solve the issues with NA's. /Johannes On Thu, Sep 4, 2014 at 2:31 PM, Thomas Adams tea...@gmail.com wrote: Johannes, If you want to read your file into R, there is no need to export your map from GRASS to do this. Simply install and use the R contributed package 'spgrass6' (spgrass6 has R dependencies that need to be installed first); it works wonderfully: Within GRASS, at the GRASS terminal prompt... library(spgrass6) Loading required package: sp Loading required package: XML GRASS GIS interface loaded with GRASS version: GRASS 7.0.0beta3 (2014) and location: ohrfc_mpe dat-readRAST6(xmrg0101200306z) image(dat) This is far more efficient. Tom On Thu, Sep 4, 2014 at 5:32 AM, Johannes Radinger johannesradin...@gmail.com wrote: Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user -- Nuno César de Sá +351 91 961 90 37 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] raster exchange between GRASS and R with nodata
On Thu, Sep 4, 2014 at 3:38 PM, Nuno Sá nunocesard...@gmail.com wrote: Hello! Did you try this one? *r.out.gdal etc nodata*=*'NA'* As mentioned in the manual of r.out.gdal, the no data parameter takes only float values and no strings like 'NA'. Without stating as specific value in GRASS, this nodata-value is automatically set to e.g. 65535 for DCELL rasters if I remember correctly and to 255 for BYTE rasters. However, this seems not to be recognized when imported into R with the package 'raster'. /Johannes On 4 September 2014 14:27, Johannes Radinger johannesradin...@gmail.com wrote: Hi all, of course it is possible to load the raster maps directly via spgrass6. However, we use this work flow also to exchange some of the maps between different users (e.g. via email) and to permanently store single files (geotiffs that contain the proj information within the file). So, I agree that using spgrass6 would be much more efficient, but I'll stick to exporting to geotiffs and so I need to solve the issues with NA's. /Johannes On Thu, Sep 4, 2014 at 2:31 PM, Thomas Adams tea...@gmail.com wrote: Johannes, If you want to read your file into R, there is no need to export your map from GRASS to do this. Simply install and use the R contributed package 'spgrass6' (spgrass6 has R dependencies that need to be installed first); it works wonderfully: Within GRASS, at the GRASS terminal prompt... library(spgrass6) Loading required package: sp Loading required package: XML GRASS GIS interface loaded with GRASS version: GRASS 7.0.0beta3 (2014) and location: ohrfc_mpe dat-readRAST6(xmrg0101200306z) image(dat) This is far more efficient. Tom On Thu, Sep 4, 2014 at 5:32 AM, Johannes Radinger johannesradin...@gmail.com wrote: Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user -- Nuno César de Sá +351 91 961 90 37 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] raster exchange between GRASS and R with nodata
Ok, didn't check that before since I didn't have the same problem before. Try exporting with a crazy value as NA such as in a 16bit unsigned r.out.gdal etc nodata=65535 Then on R use the mask function such as: mask(x, mask, filename=, inverse=FALSE, maskvalue=65535, updatevalue='NA', ...) might work but its very, you know, hurtful to see :P but I'm like you, I prefer .tif. On 4 September 2014 14:47, Johannes Radinger johannesradin...@gmail.com wrote: On Thu, Sep 4, 2014 at 3:38 PM, Nuno Sá nunocesard...@gmail.com wrote: Hello! Did you try this one? *r.out.gdal etc nodata*=*'NA'* As mentioned in the manual of r.out.gdal, the no data parameter takes only float values and no strings like 'NA'. Without stating as specific value in GRASS, this nodata-value is automatically set to e.g. 65535 for DCELL rasters if I remember correctly and to 255 for BYTE rasters. However, this seems not to be recognized when imported into R with the package 'raster'. /Johannes On 4 September 2014 14:27, Johannes Radinger johannesradin...@gmail.com wrote: Hi all, of course it is possible to load the raster maps directly via spgrass6. However, we use this work flow also to exchange some of the maps between different users (e.g. via email) and to permanently store single files (geotiffs that contain the proj information within the file). So, I agree that using spgrass6 would be much more efficient, but I'll stick to exporting to geotiffs and so I need to solve the issues with NA's. /Johannes On Thu, Sep 4, 2014 at 2:31 PM, Thomas Adams tea...@gmail.com wrote: Johannes, If you want to read your file into R, there is no need to export your map from GRASS to do this. Simply install and use the R contributed package 'spgrass6' (spgrass6 has R dependencies that need to be installed first); it works wonderfully: Within GRASS, at the GRASS terminal prompt... library(spgrass6) Loading required package: sp Loading required package: XML GRASS GIS interface loaded with GRASS version: GRASS 7.0.0beta3 (2014) and location: ohrfc_mpe dat-readRAST6(xmrg0101200306z) image(dat) This is far more efficient. Tom On Thu, Sep 4, 2014 at 5:32 AM, Johannes Radinger johannesradin...@gmail.com wrote: Hi, I want to export a raster map (FCELL) from GRASS70 to the geotiff format using r.out.gdal and to import it later on in R. The map contains many no data values. Here some details about the raster: Type of Map: raster Number of Categories: 0 Data Type:FCELL Rows: 750 Columns: 750 Total Cells: 562500 total null and non-null cells: 15105636 total null cells: 15105047 So when I export the map, r.out.gdal reports: Input raster map contains cells with NULL-value (no-data). The value -nan will be used to represent no-data values in the input map. You can specify a nodata value with the nodata option. When I subsequently try to import the geotiff into R (using the package 'Raster') the nodata values are not recognised as NA's: a - raster(*.tif) summary(a) Min. 0.5294496 1st Qu. 0.7171210 Median 0.7871540 3rd Qu. 1.1581826 Max. 1.5494517 NA's 0.000 So I am wondering if I need to set any specific parameter during the export (r.out.gdal) or import (raster()). As I am not only exporting FCELL (Float32) raster but also multiple (N=500) other rasters to R I would be interested in a solution also for DCELL (Float64). Of course I can export all of as Float64 as the file size should not be a problem. Any suggestions or experiences of handling NA's during raster exchange between GRASS and R? /Johannes ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user -- Nuno César de Sá +351 91 961 90 37 -- Nuno César de Sá +351 91 961 90 37 ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
