Re: [R-sig-Geo] NASA's Black Marble monthly data: Reprojection isn't accurate
The extent of this one is
70, 80, 10, 20
Later versions of GDAL determine this automatically (I'm not sure when)
rast("VNP46A3.A2018091.h25v07.001.2021125122857.h5")
class : SpatRaster
dimensions : 2400, 2400, 26 (nrow, ncol, nlyr)
resolution : 0.00417, 0.00417 (x, y)
extent : 70, 80, 10, 20 (xmin, xmax, ymin, ymax)
coord. ref. : lon/lat Unknown datum based upon the GRS 1980 Authalic Sphere
ellipsoid
sources :
VNP46A3.A2018091.h25v07.001.2021125122857.h5://AllAngle_Composite_Snow_Covered
VNP46A3.A2018091.h25v07.001.2021125122857.h5://AllAngle_Composite_Snow_Covered_Num
VNP46A3.A2018091.h25v07.001.2021125122857.h5://AllAngle_Composite_Snow_Covered_Quality
... and 23 more source(s)
varnames: AllAngle_Composite_Snow_Covered
AllAngle_Composite_Snow_Covered_Num
AllAngle_Composite_Snow_Covered_Quality
...
Be very careful with canned steps to "fix" georeferencing, make sure it
needs fixing and check that it's right. Ultimately the best way to "fix" it
is to go to the source of the code that interfaces the data, which here is
GDAL and report there - but clearly that's been updated from whatever
version was in use on stackoverflow.
Cheers, Mike
.
On Sun, Sep 10, 2023 at 1:17 AM Nikolaos Tziokas
wrote:
> I downloaded NASA's Black Marble monthly nighttime light NTL data, VNP46A3
> <
> https://ladsweb.modaps.eosdis.nasa.gov/missions-and-measurements/products/VNP46A3/
> >.
> In a previous question
> <
> https://gis.stackexchange.com/questions/466571/extent-not-found-on-nasas-black-marble-monthly-images-how-to-set-it/466574?noredirect=1#comment761916_466574
> >
> of mine the reprojection worked perfectly but now it seems that it doesn't.
> For example, I wanted to download NTL data for the city of Mumbai, India.
> After reprojecting the NTL (product 5 (All_Angles_Snow_Free) from the .h5)
> the result is attached here
> <https://drive.google.com/drive/folders/1V115zpdU2-5fXssI6iWv_F6aNu4E5qA7
> >.
>
> At the bottom if the image is a shp of Mumbai (downloaded from GADM) and
> the red circle in the top indicates where Mumbai is in the NTL image.
> Clearly something's not right.
>
> I downloaded the image from here
> <
> https://ladsweb.modaps.eosdis.nasa.gov/missions-and-measurements/products/VNP46A3/
> >
> (LAADS-DAAC, Level-1 and Atmosphere Archive & Distribution System
> Distributed Active Archive Center). The code I used to extract the NTL
> radiance image is:
>
> library(terra)
>
> wd <- "path/"
>
> r <- rast(paste0(wd, "VNP46A3.A2018091.h25v07.001.2021125122857.h5"))
> crs(r) <- "epsg:4326"
>
> 2400*(15/(60*60))
>
> h = 25
> v = 7
>
> ext(r) = c(-180+h*10,-180+(h+1)*10, (v-2)*10,(v-1)*10)
>
> ntl <- r[[5]]
>
> writeRaster(ntl, paste0(wd, "ntl.tif"), overwrite = TRUE)
>
> Why the code worked perfectly in the previous question and now it doesn't?
> From here
> <https://drive.google.com/drive/folders/1V115zpdU2-5fXssI6iWv_F6aNu4E5qA7>
> you can download the .h5 image if you don't want to use NASA's website. I
> am using R 4.3.1 and RStudio 2023.06.2+561.
>
> --
> Tziokas Nikolaos
> Cartographer
>
> Tel:(+44)07561120302
> LinkedIn <http://linkedin.com/in/nikolaos-tziokas-896081130>
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Did I correctly convert my data from UTM zone 12 to 11?
You should treat the different sets of coordinates (from Zone 11 and Zone 12) completely separately. You cannot have mixed crs (of projected coordinates) in a dataset. When you plot in R (generally, even with popular spatial formats) there is no account taken of crs, the graphics doesn't know that you plot Zone 11 and then (say) add coordinates from Zone 12, they will be "shifted" as you say. (There are exceptions to this plotting rule but apart from ggplot2::coord_sf they are in dusty corners not front and centre of plotting code in R). Some level of coordinate hackery (arithmetic shifting) *can work* in limited circumstances, but I would highly recommend against that. Separate your coordinates into two objects, set the source crs appropriately of the zone for each, transform each to a common crs. (UTM is generally a bad idea but comes with a very popular usage culture which is a shame, there's no single right choice but you always need to put thought into the overall region of your data, the properties of the coordinate space that make sense for your work, and whether you will need smaller or larger regions in the future for related work ... I would advise a common crs that is *not UTM* but advice there really depends on the details of your situation). It's hard to be more specific without details of your flow, e.g. code examples. HTH, Mike On Mon, Aug 7, 2023 at 6:23 AM Jason Edelkind via R-sig-Geo < r-sig-geo@r-project.org> wrote: > hello, first time user here and aspiring grad student. I have a set of > location data that I’ve been trying to import into google earth engine from > R as a CSV file. The problem is that about half of my data is from utm zone > 12, and the other half is from utm zone 11. When I import my original data > into google earth engine, the zone 11 data is shifted over to the right > because I use utm zone 12 as the crs in R. After some reading into the > definition of a utm zone, I tried to just subtract 6 from the zone 11 > latitude values after first converting them to a lon/lat format. This > appears to have worked as on first glance all of the zone 11 points are > where they should be, however it feels like too easy a fix for me after > struggling with this for several days. So my question is, is this an > acceptable way to convert my data, or am I doing something wrong that could > result in inaccurate location data? Thanks! > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Mapping my own polygon?
The OpenStreetMap package is using PseudoMercator (quite common for image
tile servers) which is EPSG:3857. Using your long/lat min max the numbers
give the right neighbourhood, perhaps a transcribe error in the original
'rw' data frame numbers for Y?
https://gist.github.com/mdsumner/e6997c2f4a54c743e078aca8401537a0?permalink_comment_id=4597644#gistcomment-4597644
On Mon, Jun 12, 2023 at 12:58 AM Kevin Zembower wrote:
> Hi, Mike, thanks for answering me from across the world!
>
> I created my polygon of my neighborhood the old fashion way: by printing
> a map with a grid in meters, and using a pair of navigational dividers
> to pick off the x and y coordinates of the borders of my neighborhood.
> You can see a map of my neighborhood, drawn by someone else, at
> https://www.radnorwinston.org/, if you're curious.
>
> Here's code to generate a map of this area, one with degrees lat/long,
> and the one I used, with a grid of meters:
>
> ===
> ## This gives degrees lat long:
> library(tidyverse)
> library(tigris)
> options(tigris_use_cache = TRUE)
> library(sf)
> library(OpenStreetMap)
>
> lat_max <- 39.3525
> long_max <- -76.617
> lat_min <- 39.3455
> long_min <- -76.6095
> nw <- c(lat_max, long_max)
> se <- c(lat_min, long_min)
>
> rw_map <- openmap(nw, se,
>type = "osm",
>mergeTiles = TRUE) %>%
> openproj() %>%
> OpenStreetMap::autoplot.OpenStreetMap() +
> xlab("long") + ylab("lat")
>
> rw_map
>
> ## This gives map with grid in meters:
> library(tidyverse)
> library(tidycensus)
> library(sf)
> library(tmap)
> library(tigris)
> options(tigris_use_cache = TRUE)
> library(tmaptools)
>
> rw_block_list <- c("3000", "3001", "3002", "3005", "3006", "3007",
> "3008", "3009", "3010", "3011", "3012")
>
> ## Get the RW blocks from the census:
> rw_blocks <- blocks(state = "MD",
> county = "Baltimore city",
> year = "2020") %>%
> filter(substr(GEOID20, 6, 11) == "271101" &
> substr(GEOID20, 12, 15) %in% rw_block_list)
>
> ## Create a map of just the RW blocks:
> rw_base_blocks <- read_osm(bb(rw_blocks, ext = 1.3))
>
> tmap_mode("plot")
>
> (RW_block_map <- tm_shape(rw_base_blocks) +
> tm_rgb() +
> tm_shape(rw_blocks) +
> tm_fill("MAP_COLORS", alpha = 0.2, palette = "Accent", n = 10) +
> tm_borders() +
> tm_scale_bar() +
> tm_grid() + tm_xlab("Long") + tm_ylab("Lat") +
> tm_layout(title = "Radnor-Winston Neighborhood")
> )
> ===
>
> You're right, that I did pick my example "1 POINT (-76.61246 39.35010)"
> just by picking a random point in my neighborhood, using Google Maps.
>
> In the second example that gives the grid in meters, I just assumed that
> this was CRS 6487, due to it being Maryland and in meters. Was this a
> bad assumption? How can I tell what the CRS of the return of the call to
> "read_osm(bb(rw_blocks, ext = 1.3))" is? When I display it, it says:
>
> > rw_base_blocks
> stars object with 3 dimensions and 1 attribute
> attribute(s), summary of first 1e+05 cells:
> Min. 1st Qu. MedianMean 3rd Qu. Max.
> X 0 217224 219.915 238 255
> dimension(s):
> from to offsetdelta refsys
> values x/y
> x 1 663 -8528854 1.19942 WGS 84 / Pseudo-Mercator
> NULL [x]
> y 1 907 4772338 -1.19981 WGS 84 / Pseudo-Mercator
> NULL [y]
> band1 3 NA NA NA red , green,
> blue
> >
>
> Thanks, again, for your work to help me. Thanks also for introducing me
> to Github Gist, which I had never heard of before. I'm going to go to
> that page next and see if I can put this response in there.
>
> Take care.
>
> -Kevin
>
> On 6/11/23 04:04, Michael Sumner wrote:
> > I took a guess that your coordinates are not in EPSG:6487 but in global
> > Mercator (EPSG:3857) which seems to give a reasonable region from online
> > image servers.
> >
> > https://gist.github.com/mdsumner/e6997c2f4a54c743e078aca8401537a0
> > <https://gist.github.com/mdsumner/e6997c2f4a54c743e078aca8401537a0>
> >
> > If that looks ok? Then, simply replace 6487 with 3857 in your code,
> > but also please take care t
Re: [R-sig-Geo] Mapping my own polygon?
I took a guess that your coordinates are not in EPSG:6487 but in global
Mercator (EPSG:3857) which seems to give a reasonable region from online
image servers.
https://gist.github.com/mdsumner/e6997c2f4a54c743e078aca8401537a0
If that looks ok? Then, simply replace 6487 with 3857 in your code, but
also please take care to track down where your neighbourhood coordinates
for the x,y range came from. 3857 is the infamous global google mercator,
and that might be what you're using or simply close to it, you should make
sure you know. :)
I'm using in-dev code in the example, it's not because I want you to use
that or advise you to - it's just to keep a record of what I did. I
experimented with the code and scale to guess at what might be the problem.
If the pic in my gist is not on the right track I'm happy to follow up,
best of luck!
HTH, Mike
On Sun, Jun 11, 2023 at 5:27 AM Kevin Zembower via R-sig-Geo <
r-sig-geo@r-project.org> wrote:
> In my continuing work on reporting on US Census data for my
> neighborhood, I'd like to draw a map of the boundaries of it. I was
> successful in creating and printing an OSM basemap, with the US Census
> blocks that make up my neighborhood on it.
>
> Now, I'd like to create my own polygon, of the boundaries of my
> neighborhood, because the census blocks don't line up exactly with the
> neighborhood boundaries. I need help creating a polygon that I can
> submit to read_osm() that will correctly return an OSM map of my area.
>
> Here's what I've tried so far:
> ## Reproducible simple example:
> library(tidyverse)
> library(sf)
>
> rw <- data.frame( ## Simplified neighborhood rectangle
> Longitude = c(-8528150, -8528500, -8528500, -8528150),
> Latitude = c( 4771475, 4771475, 4771880, 4771880)
> )
>
> rw ## Returns (as expected):
>
> > rw
>Longitude Latitude
> 1 -8528150 4771475
> 2 -8528500 4771475
> 3 -8528500 4771880
> 4 -8528150 4771880
> >
>
> rw %>%
> st_as_sf(coords = c("Longitude", "Latitude"), dim = "XY") %>%
> st_set_crs(6487) %>% ## CRS 6487 is NAD83 (2011) Maryland in meters
> st_transform(crs = 4269) ## CRS 4269 is NAD83
>
> ## Returns:
>
> Simple feature collection with 4 features and 0 fields
> Geometry type: POINT
> Dimension: XY
> Bounding box: xmin: 171.777 ymin: 24.65904 xmax: 171.7818 ymax: 24.66314
> Geodetic CRS: NAD83
> geometry
> 1 POINT (171.7818 24.66192)
> 2 POINT (171.7806 24.65904)
> 3 POINT (171.777 24.66026)
> 4 POINT (171.7781 24.66314)
> >
>
> I expected the POINTS to look like:
> 1 POINT (-76.61246 39.35010)
>
> Can anyone suggest what I'm doing wrong? Thanks so much in advance. I've
> worked on it all day today, without making much progress.
>
> -Kevin
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Combine two polygons
fwiw, a method with geodata/terra
cyp <- geodata::gadm(country = c("CYP", "XNC"), path = tempdir(), version =
"3.6")
terra::aggregate(cyp)
see ?terra::writeVector to write it out
(rgdal is very old and you should abandon it, it will be removed from
support by the end of this year)
(just as an aside, we can't hit the geodata vector urls directly with gdal
as we can with the rasters, as they are in an R specific format)
Cheers, Mike
On Fri, Apr 28, 2023 at 10:52 AM Ben Tupper wrote:
> Hi,
>
> You might try st_union() from the sf package
> https://r-spatial.github.io/sf/articles/sf3.html#geometrical-operations
>
>
>
> On Thu, Apr 27, 2023 at 5:37 PM Miluji Sb wrote:
>
> > Dear all,
> >
> > I am using the country-level shapefile from GADM. I would like to merge
> > Cyprus and North Cyprus into one polygon. Is this possible? I am using
> > rgdal to read the shapefile, subset, and then rewrite it. Any help will
> be
> > highly appreciated.
> >
> > ##
> > world <- readOGR("~/gadm36_levels_shp", layer = "gadm36_0")
> >
> > Best regards,
> >
> > Milu
> >
> > [[alternative HTML version deleted]]
> >
> > ___
> > R-sig-Geo mailing list
> > R-sig-Geo@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
[R-sig-Geo] general GDAL engagement
Hello,
I just want to send a general note of encouragement to R folks to engage
more closely with the GDAL development project.
GDAL is used heavily in terra and sf and vapour and gdalcubes. It provides
read and write access to many many file and url and database formats,
amongst many other facilities.
Sometimes, there are features in R that can be made redundant because of
changes in GDAL, an example is this file mentioned in the related github
issue:
If you've got useful information about files that could be used to improve
GDAL, please share it on the gdal-dev mailing list, or as bug reports on
the GDAL issues on github. If you need assistance or aren't sure I'm very
happy to help too.
Here's two examples.
1)
In R, terra knows this file is longitude/latitude because it has internal
heuristics (the data projection metadata is not set, but the data lives
with longitudes -180, 360 and latitudes -90, 90 (or near enough):
terra::crs(terra::rast("NetCDF:oisst-avhrr-v02r01.20220218.nc:sst"))
[1] "GEOGCRS[\"WGS 84 (CRS84)\",\nDATUM[\"World G
In sf and stars, it's not known because sf is more faithful to what GDAL
provides (this file is non compliant to whatever convention, it's a very
common situation because some domains assume "longlat" is just obvious).
sf::gdal_read("NetCDF:oisst-avhrr-v02r01.20220218.nc:sst", read_data =
F)$crs
Coordinate Reference System: NA
In GDAL 3.7.0 when that rolls out across R, both sf and terra will get the
same valid crs information from GDAL if the
option GDAL_NETCDF_ASSUME_LONGLAT=YES is set.
2) GRIB is a wild and wonderful format, and when it uses map projections
sometimes the metadata is completely screwed up - for too long I worked
around this in R, but a simple bug report on thegithub issues saves a whole
of of pain and extra code on the R side:
https://github.com/OSGeo/gdal/issues/7298
These examples represent one of many tiny details that users encounter in
many places. Please take the time to report these issues to GDAL, sometimes
effort that goes into downstream packages on CRAN won't make it further up
the chain, which is a shame - because, it might be fixed in R, but if it
were fixed in GDAL it'll also be fixed in Python, QGIS, and many other
applications.
There are many ways to contribute beyond simply sharing files and output of
course, improving the documentation, fixing bugs - if you're interested in
this and want to discuss I'm also happy to help.
The file I used in the example above is posted in the related GDAL PR (pull
request):
https://github.com/OSGeo/gdal/issues/6195
All the best!
Mike
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Spatial nested grid in R
Oh, with a COG you'd also want control over reading a particular level -
I'm not sure how well that's supported in the "R spatial" realm but I'll
include it in my example, you can always generate VRT to get exactly what
you want from terra and friends.
Cheers, Mike
On Sat, Feb 11, 2023 at 9:36 AM Michael Sumner wrote:
> One way is to write it to a (COG) GeoTIFF with overviews ("pyramid", or
> zoom levels) - these are pre-calculated copies of the highest resolution
> data saved as lower resolution versions.
>
> Do you want this to generate a set of nested data, or is it more about
> working with the nested logic?
>
> I'm not sure what version of GDAL is required to write one directly with
> terra or raster, but i'll explore. In terms of the nesting logic , you can
> read a particular zoom level (or model it with an data-empty object) and
> determine cell index using the cells tools in terra (or raster). I think
> you would need to generate xyFromCell and use that to cellFromXY between
> layers (the arithmetic is not onerous but doesn't exist for reuse anywhere
> in R afaik).
>
> I'm interested in this generally for workflows I'm using so might come
> back with an example, happy to follow up related questions.
>
> At the command line with COG format you can do
> gdalinfo in.tif overviews.if -of COG -co OVERVIEW_COUNT=3 ## 3, for
> example
>
> or
>
> gdal_translate in.tif overviews.tif
> gdaladdo overviews.tif 2 4 8
>
> but, of course there are implications with tile pattern and potential
> overlap for a given size. You'd probably also want tiling enabled and
> choose a particular tile size (perhaps to match the next resolution down).
> And I'd definitely want to make sure my extent was clean whole numbers and
> choose sensible sized zoom and tile levels.
>
> Cheers, Mike
>
>
> On Fri, Feb 10, 2023 at 10:22 AM Manuel Spínola
> wrote:
>
>> Dear list members,
>>
>> Is it possible to generate a spatial nested grid in R?
>>
>> For example, a grid of several 8km x 8km tiles, and within that grid, I
>> want 4 tiles of 4km x 4km, and in each of those I want 4 tiles of 2km x
>> 2km, and in each of those I want 4 tiles of 1km x 1km.
>>
>> Manuel
>>
>> --
>> *Manuel Spínola, Ph.D.*
>> Instituto Internacional en Conservación y Manejo de Vida Silvestre
>> Universidad Nacional
>> Apartado 1350-3000
>> Heredia
>> COSTA RICA
>> mspin...@una.cr
>> mspinol...@gmail.com
>> Teléfono: (506) 8706 - 4662
>> Institutional website: ICOMVIS
>> <http://www.icomvis.una.ac.cr/index.php/manuel>
>> Blog sobre Ciencia de Datos:
>> https://mspinola-ciencia-de-datos.netlify.app
>>
>> [[alternative HTML version deleted]]
>>
>> ___
>> R-sig-Geo mailing list
>> R-sig-Geo@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
> --
> Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> Hobart, Australia
> e-mail: mdsum...@gmail.com
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Spatial nested grid in R
One way is to write it to a (COG) GeoTIFF with overviews ("pyramid", or
zoom levels) - these are pre-calculated copies of the highest resolution
data saved as lower resolution versions.
Do you want this to generate a set of nested data, or is it more about
working with the nested logic?
I'm not sure what version of GDAL is required to write one directly with
terra or raster, but i'll explore. In terms of the nesting logic , you can
read a particular zoom level (or model it with an data-empty object) and
determine cell index using the cells tools in terra (or raster). I think
you would need to generate xyFromCell and use that to cellFromXY between
layers (the arithmetic is not onerous but doesn't exist for reuse anywhere
in R afaik).
I'm interested in this generally for workflows I'm using so might come back
with an example, happy to follow up related questions.
At the command line with COG format you can do
gdalinfo in.tif overviews.if -of COG -co OVERVIEW_COUNT=3 ## 3, for example
or
gdal_translate in.tif overviews.tif
gdaladdo overviews.tif 2 4 8
but, of course there are implications with tile pattern and potential
overlap for a given size. You'd probably also want tiling enabled and
choose a particular tile size (perhaps to match the next resolution down).
And I'd definitely want to make sure my extent was clean whole numbers and
choose sensible sized zoom and tile levels.
Cheers, Mike
On Fri, Feb 10, 2023 at 10:22 AM Manuel Spínola
wrote:
> Dear list members,
>
> Is it possible to generate a spatial nested grid in R?
>
> For example, a grid of several 8km x 8km tiles, and within that grid, I
> want 4 tiles of 4km x 4km, and in each of those I want 4 tiles of 2km x
> 2km, and in each of those I want 4 tiles of 1km x 1km.
>
> Manuel
>
> --
> *Manuel Spínola, Ph.D.*
> Instituto Internacional en Conservación y Manejo de Vida Silvestre
> Universidad Nacional
> Apartado 1350-3000
> Heredia
> COSTA RICA
> mspin...@una.cr
> mspinol...@gmail.com
> Teléfono: (506) 8706 - 4662
> Institutional website: ICOMVIS
> <http://www.icomvis.una.ac.cr/index.php/manuel>
> Blog sobre Ciencia de Datos: https://mspinola-ciencia-de-datos.netlify.app
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] getting data from an nc file
have a look at the output of printing nc_data, you are looking for
dimensions that belong to a 2D (or higher) variable of interest .
They can be called anything, but longitude, lon, x, etc are common. you can
use ncdf4 functions to find them (without reading the output), but it's all
stored in a nested list in nc_data).
See ncmeta package for an alternative way to access the metadata.
Hth, Mike
On Wed, 9 Nov 2022, 21:54 Nick Wray, wrote:
> Hello
>
> I am trying to get rainfall data from the UK chess-met site
>
>
> https://catalogue.ceh.ac.uk/datastore/eidchub/2ab15bf0-ad08-415c-ba64-831168be7293/precip/
>
> and here there are a large number of nc files eg
>
> "chess-met_precip_gb_1km_daily_20150101-20150131.nc"
>
> I’ve found various sources on the net for opening nc files, and getting the
> data but when I try instructions like
>
> nc_data<-nc_open("chess-met_precip_gb_1km_daily_20150101-20150131.nc")
>
> lon <- ncvar_get(nc_data, "lon")
>
> lon
>
>
>
> the code works but all I get is a series of element numbers and no data:
>
> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8]
> [,9]
> [,10][,11][,12]
>
> [,13][,14][,15][,16][,17][,18][,19]
> [,20][,21][,22][,23][,24]
>
>
>
> And similarly for latitude
>
> But if I open the nc file as a raster I get a raster precipitation plot of
> Great Britain, which rather suggests that the lat and long values are in
> there somewhere
>
> Can anyone help with getting the actual data sets out of an nc file?
>
> Thanks Nick Wray
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Raster Data Management Advice
I would set up a polygon of the bounding box (in the native projection) of each raster source, and use fields on those polygons to store the details of interest: xmin,xmax,ymin,ymax, dimension, resolution, crs, and your other details. Then hone in a areas of interest for different tasks to see what set of overlapping data you have for it. There's a lot of fanfare about STAC, but it's really just a JSON-format with some of the information you could store on a simple polygon dataset ...with STAC as with so many formats you'd have to shoehorn your data into that more restrictive form (you can always spit out STAC as a side product of your own rich summary for less sophisticated uses). The crux is keeping the details of the source's native projection independent from the representation you use to query it spatially IMO, just record what's there. Further, the GDAL warper app-lib (one level below the gdalwarp.exe) is the right tool for doing general reads, of any number of sources into one specific window of your choosing in any projection (you could use your dataset described above to limit which sources get included). You can easily see what you'd get by merging any number of sources together, and of course more nuanced situations like a sensible background with more detailed layers merged over that is very valuable. Cheers, Mike On Sat, Oct 8, 2022 at 3:35 AM Alexander Ilich wrote: > Hi, I was wondering if anyone has some advice on how to organize raster > data so that it is easily queryable by various attributes (e.g. find me all > the rasters of data type bathymetry, collected by this organization with > 10m resolution or finer ). Currently we have data on a server organized > often by when/where it was collected but that can make it difficult to find > specific rasters that meet a certain criteria. I've created a table as a > csv file on github <https://github.com/ailich/WFS_Multibeam_Metadata> > where > each row is a raster and it has various column attributes describing it > (e.g. who collected it, what sonar was used, resolution, coordinate system, > etc) and a path to the filename as a temporary solution, but I think some > type of spatial database that would allow for querying and then reading > into R as terra objects, as well as into QGIS and ArcGIS as layers for > visualization would be optimal as multiple project members use these data. > Tools I've come across that seem potentially useful include PostGIS and > Geopackage, but I'm not entirely sure how to properly set them up or if > they'd suit my needs. Any advice would be greatly appreciated. > > Thanks, > Alex > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Extract Lightning Flash Time series data
I don't understand this file structure yet, but you might try tidync:
library(tidync)
## open the file and get a summary of all vars and the dimensions they use
(nc <- tidync::tidync("TRMM_LIS.nc"))
See how viewtime_lat, viewtime_lon (and others) exist on the grid
identified by "D0" (other grids include multiple dims and so have
identifiers like "D1,D2" - this concept doesn't exist in NetCDF itself but
tidync wanted to group variables by the dims they use).
The grid "D0" is not active by default, so you can do
nc |> activate("D0")
(everything is lazy up to this point, no data has been read)
Convert to dataframe with
nc |> activate("D0") |> hyper_tibble()
You can use hyper_array() to get in native array format with some informal
metadata on the output.
HTH, otherwise package stars has another approach and some similar
facilities. I often use tidync just to understand what's there, but I
might craft a solution with a different tool after I've explored. RNetCDF
will give you direct low level access to this format, for example.
Cheers, Mike
On Sun, Sep 11, 2022 at 1:22 PM sownal chand
wrote:
> Hello Sir/Madam,
>
> I have downloaded a TRMM_LIS.nc File with lightning time series data and
> was trying to extract it into a CSV file but I am not sure on how to do it.
>
> I am also attaching the TRMM_LIS file via this link:
>
> https://drive.google.com/drive/u/0/folders/19nJNwPKlJaPlkHn_NGzmjMHMaoUevH04
>
> Request if someone can guide me and thanking you in advance
> sownalc
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Does stars::st_extract preserve records order
lol, as a matrix - does it just?
very clever
On Wed, 27 July 2022, 22:36 Ben Tupper, wrote:
> Hi,
>
> I have never observed st_extract() fail to preserve row order.
>
> Your workflow works well. The documentation for st_extract() mentions
> that if you might see better performance for large sets of points if
> you pass `at` locations as a matrix instead as sf. I don't think it
> makes it any more or less elegant. Such an approach might look like
> this...
>
> ### start
>
> # Set column names of matrix/data.frame to months
> set_column_names <- function(x) {
> colnames(x) <- month.abb
> x
> }
>
> # Extract monthly avg temperature for cities
> tavg_df <- st_extract(tavg, at = sf::st_coordinates(sfdf)) %>% # <- pass
> matrix
> set_column_names() %>%
># <- name the columns
> dplyr::as_tibble()
> # <-cast as data frame
>
> # Recombine with original sf dataframe
> full_df <- cbind(sfdf, tavg_df)
>
> # Simple feature collection with 3 features and 13 fields
> # Geometry type: POINT
> # Dimension: XY
> # Bounding box: xmin: 2.3522 ymin: 43.2965 xmax: 5.3698 ymax: 48.8566
> # Geodetic CRS: WGS 84
> # names Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
> # 1 Paris 4.8 5.4 8.2 10.6 14.8 17.6 20.0 20.0 16.5 12.3 7.8 5.7
> # 2 Marseille 8.6 9.2 11.1 13.3 17.3 20.7 23.6 23.4 20.3 16.4 11.7 9.5
> # 3 Lyon 3.6 5.1 8.1 10.8 15.4 18.7 21.7 21.2 17.2 12.5 7.2 4.5
> # geometry
> # 1 POINT (2.3522 48.8566)
> # 2 POINT (5.3698 43.2965)
> # 3 POINT (4.8357 45.764)
>
> ### end
>
> Cheers,
> Ben
>
> On Wed, Jul 27, 2022 at 5:54 AM DUTRIEUX Loic
> wrote:
> >
> > Hi everyone,
> >
> > I'm wondering whether st_extract() from stars package preserves the
> order of the input sf dataframe.
> > See the example below; is it a safe approach? Is there perhaps a more
> "elegant" way to do the same extraction?
> >
> > library(sf)
> > library(stars)
> > library(geodata)
> > library(magrittr)
> >
> > # Create sf dataframe of cities
> > cities <- data.frame(x=c(2.3522, 5.3698, 4.8357),
> > y=c(48.8566, 43.2965, 45.7640),
> > names=c('Paris', 'Marseille', 'Lyon'))
> > sfdf <- st_as_sf(cities,
> > coords=c('x','y'),
> > crs=st_crs(4326),
> > agr='identity')
> >
> > # Get raster of monthly average tempature
> > tavg <- worldclim_tile(var = 'tavg', lon = 15, lat = 45, path =
> tempdir()) %>%
> > st_as_stars()
> >
> > # Extract monthly avg temperature for cities
> > tavg_df <- st_extract(tavg, at = sfdf) %>%
> > st_as_sf() %>%
> > st_drop_geometry()
> >
> > # Recombine with original sf dataframe
> > full_df <- cbind(sfdf, tavg_df)
> >
> >
> > Kind regards,
> > Loïc
> >
> > ___
> > R-sig-Geo mailing list
> > R-sig-Geo@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
>
>
> --
> Ben Tupper (he/him)
> Bigelow Laboratory for Ocean Science
> East Boothbay, Maine
> http://www.bigelow.org/
> https://eco.bigelow.org
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] How to export a GeoTIFF 8 bit raster
this might work, your file link doesn't provide access to me so I created a guess example library(terra) x <- rast(matrix(sample(0:255, 12), 3)) file.tif <- tempfile(fileext = ".tif") #writeRaster(x, file.tif, gdal = "NBITS=8", datatype = "INT1U") #rast(file.tif) terra has its own controls over the options, some interact/overlap with the gdal ones I haven't explored that HTH, Mike On Mon, Jan 31, 2022 at 5:52 AM Rik Ferreira wrote: > > Hello! > > How a raster image can be exported as a 8 bit GeoTIFF image? > > I'm downloading a 8 bit image from this link: > http://geoinfo.cnps.embrapa.br/documents/2918/download, cropping it with a > vector mask, treating it, and exporting it with terra package (writeRaster > function). > > However, the original image is a GeoTIFF with 8 bit depth but the output > image (terra package, writeRaster function) is a 32 bit image. > > I can control it with `gdal = "NBITS=8"` argument to writeRaster but i get > such warning: > > Warning message: > > In x@ptr$writeRaster(opt) : > > GDAL Message 6: > > /mnt/HDD/STORAGE/r-projects/raster-bit-depth/output/recorte.tif: Only > > NBITS=16 is supported for data type Float32 > > > > How can I export this raster as a 8 bit GeoTIFF image with terra package? > > -- > Rik Ferreira Alves > Graduando em Geografia - Unimontes > https://rikferreira.netlify.app/ > > [[alternative HTML version deleted]] > > _______ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Question on hierarchical data format 5 .h5 file format
see if gdal understands the coordinate arrays already, you can just churn
it through the warper, otherwise construct a vrt that specifies them, which
is a bit obscure how you do it admittedly - but this is a gdal not an R
topic.
Best
On Mon, 15 Nov 2021, 00:22 Edzer Pebesma,
wrote:
>
> On 14/11/2021 13:59, Edzer Pebesma wrote:
> > Gabriel shared the data with me (off-list, 1Gb) and it contains two hdf5
> > datasets, one containing the raster of the longitude and latitude, the
> > other rasters with the attributres (LST) but no coordinates. I could get
> > a (rough: factor 50 downsampling) plot with stars using:
> >
> > library(stars)
> > lat =
> >
> read_stars("HDF5:ECOSTRESS_L1B_GEO_11712_012_20200730T061029_0601_01.h5://Geolocation/latitude")
>
> >
> > lon =
> >
> read_stars("HDF5:ECOSTRESS_L1B_GEO_11712_012_20200730T061029_0601_01.h5://Geolocation/longitude")
>
> >
> >
> > var =
> > "HDF5:ECOSTRESS_L2_LSTE_11712_012_20200730T061029_0601_01.h5://SDS/LST"
> > lst = read_stars(var, curvilinear = list(x = lon, y = lat))
> > plot(lst, downsample = 50, axes = TRUE, reset = FALSE)
> >
> > library(rnaturalearth)
> > ne = ne_countries(returnclass = "sf")
> > plot(st_geometry(ne), add = TRUE, border = 'yellow')
> > # california border seems to make sense
> >
> >
> > but I'm not sure how to make a regular raster (in some CRS) out of this
> > for exporting to GeoTIFF.
>
> This might be useful: https://github.com/r-spatial/stars/issues/386
> but converting to polygons will be incredibly slow and memory hungry.
>
>
> >
> > On 14/11/2021 11:00, Edzer Pebesma wrote:
> >> This might be because the grid is not regular but possibly
> >> curvilinear, having two raster layers with the lon & lat values for
> >> each pixel. Can you share a sample data set?
> >>
> >> On 14/11/2021 10:47, Gabriel Cotlier wrote:
> >>> Dear Michael,
> >>>
> >>> Following your advice I have tired :
> >>>
> >>> library(terra)
> >>> library(raster)
> >>> library(dplyr)
> >>>
> >>> r = terra::rast(file.choose())
> >>> plot(r$LST)
> >>> r_lst = r$LST
> >>> r_lst %>% raster()
> >>> plot(r_lst)
> >>>
> >>> Then I effectively get a raster layer format from the raster package
> >>> as I
> >>> wanted, but looking at the plot and the raster layer information--see
> >>> bellow-- the scene appears to need both latitude and longitude for
> >>> correct
> >>> geolocation, how can I do this procedure?
> >>>
> >>>> r_lst %>% raster()
> >>> class : RasterLayer
> >>> dimensions : 5632, 5400, 30412800 (nrow, ncol, ncell)
> >>> resolution : 1, 1 (x, y)
> >>> extent : 0, 5400, 0, 5632 (xmin, xmax, ymin, ymax)
> >>> crs: NA
> >>> source : LST
> >>> names : LST
> >>> values : 0, 65535 (min, max)
> >>>
> >>> The extent and the resolution are not the correct ones.
> >>> Maybe in the same .h5 is all the data for georeferencing it although I
> >>> could not see how to get to it and apply it to the layer.
> >>> There is also another file with complementary data related to
> >>> geolocation
> >>> but how can I query it, and get the data for geolocation and apply it
> to
> >>> the raster layer?.
> >>> The objective is to save it as a GeoTIFF file to be opened in QGIS with
> >>> correct geolocation.
> >>>
> >>> What do you suggest for converting the .h5 file to a georeferenced
> >>> Raster
> >>> Layer in GeoTIFF format ?
> >>>
> >>> Thanks a lot.
> >>> Kind regards,
> >>> Gabriel
> >>>
> >>> On Sat, Nov 13, 2021 at 12:22 PM Gabriel Cotlier
> >>> wrote:
> >>>
> >>>> Dear Michael Summer,
> >>>> Thanks a lot for your swift reply.
> >>>> I will give a try to the procedure you mentioned.
> >>>> Kind regards,
> >>>> Gabriel
> >>>>
> >>>>
> >>>> On Sat, Nov 13, 2021 at 12:14 PM Michael Sumner
> >>>> wrote:
> >>>>
> >>>>> try terra::rast() or stars::read_stars() on the file, both use gdal
> to
> >>>>> interrogate and read. What you get depends on
Re: [R-sig-Geo] Question on hierarchical data format 5 .h5 file format
try terra::rast() or stars::read_stars() on the file, both use gdal to interrogate and read. What you get depends on the structure of the files, variables interpreted as subdatasets. stars is more general but sees variables as 2D arrays with bands, terra calls these bands layers. Alternatively try raster::raster() or RNetCDF or tidync which might be more straightforward than rhdf5 but all are quite different to each other and to the gdal view. hth On Sat, 13 Nov 2021, 18:56 Gabriel Cotlier, wrote: > Hello, > > I would like to be able to visualize / plot a specific data layer within > the structure a hierarchical data file format .h5 in R. I could observe > the structure as follows : > > library(rhdf5) > r= h5ls(file.choose()) > r > > and also in this way identify the Group and Name of the data to be plotted, > however I've tried I could not access the data itself, nor plot it as a > raster layer with its corresponding geographic coordinates. > I would like to be able to select such data, plot it as a raster layer and > if possible save it as GeoTIFF file format to be further processed with the > raster package as a common raster layer. > > If any small guidance example, reprex, of this procedure is possible > it would be appreciated. > > Thanks a lot in advance > Kind regards, > Gabriel Cotlier > -- > Gabriel Cotlier, PhD > Haifa Research Center for Theoretical Physics and Astrophysics (HCTPA) > University of Haifa > Israel > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] rgdal retirement
On Tue, Sep 21, 2021 at 8:04 PM Roger Bivand wrote:
> On Sun, 19 Sep 2021, Michael Sumner wrote:
>
> > Hi Jean-Luc, if the packages mentioned don't yet cover it, you can get
> > ogrInfo and ogrFIDs with a combination of vapour's vapour_read_names()
> > (for FID), vapour_geom_summary() (for FID and type) and
> > vapour_layer_info().
>
> In sf you can use the fid_column_name= argument to st_read() to retreive
> the FIDs with the geometries, but I think not the FIDs alone. orgInfo grew
> over time and became unwieldy, are there specific components that you use?
> Might sf::st_read() be modified to report these (and return FIDs) without
> reading the data and geometries? Using vapour is of course a good way to
> proceed, and very useful for cross-checking (Do rgdal and vapour return
> the same values, I hope so, vapour is much cleaner).
>
>
They do where I've checked! I've put some notes here running the code
below: https://gist.github.com/mdsumner/6e756ded5f73c19bfb5935b12e4cec16
file <- "list_locality_postcode_meander_valley.tab"
## MapInfo file is 1-based FID
mvfile <- system.file(file.path("extdata/tab", file), package="vapour")
range(vapour::vapour_read_names(mvfile))
range(rgdal::ogrFIDs(mvfile, "list_locality_postcode_meander_valley"))
## shapefile is 0-based FID
pfile <- system.file("extdata/point.shp", package="vapour")
range(vapour::vapour_read_names(pfile))
range(rgdal::ogrFIDs(pfile, "point"))
## geopackage is 1-based
gfile <- system.file("gpkg/nc.gpkg", package = "sf")
range(vapour::vapour_read_names(gfile))
range(rgdal::ogrFIDs(gfile, "nc.gpkg")) ## quirky layer name, it's not
always predictable
## real DB used not to have FIDS accessible by SQL (so I thought ... but
now it seems to work
vapour::vapour_read_attributes(mvfile, sql = "SELECT FID FROM
list_locality_postcode_meander_valley")
sf::read_sf(mvfile, query = "SELECT FID FROM
list_locality_postcode_meander_valley")
## some formats are arbitrary FIDS (not sequential by-1 or starting from
0/1)
osmfile <- system.file("extdata/osm/osm-ways.osm", package = "vapour")
vapour::vapour_read_names(osmfile)
rgdal::ogrFIDs(osmfile, "points")
My major concern with the vapour version is it won't handle 64-bit FIDs, I
tried and failed to get that to work but I've also never needed it, I don't
use databases much. I'm not sure if there's a broader variety of FIDS/names
to be concerned with.
> >
> > For make_EPSG() it's probably time to hit the proj.db more directly and
> I'd
> > be happy to have a look at that.
>
> Right, RSQLite can handle this quite well, but sometimes the names of
> tables and fields in tables change.
>
>
Indeed, note that you can read it with gdal itself i.e.
## on my linux (I don't know how to generalize this proj.db find step)
f <- "/usr/share/proj/proj.db"
## or CRAN built windows/macos
#f <- system.file("proj/proj.db", package = "sf", mustWork = TRUE)
## table of authority codes, but I don't generally know my way around this
projected_crs <- sf::read_sf(f, "projected_crs")
Cheers, Mike
Roger
>
> >
> > Cheers, Mike
> >
> > On Sun, Sep 19, 2021 at 9:35 AM Jean-Luc Dupouey <
> jean-luc.dupo...@inrae.fr>
> > wrote:
> >
> >> Dear Roger Bivand,
> >>
> >> I read your message about the rgdal retirement. It was a nice package
> >> and an important step in the development of R-spatial.
> >>
> >> I looked for alternatives for the functions I was using, but I didn't
> >> find them all. Those I am missing:
> >>
> >> . ogrInfo
> >> . ogrFIDs
> >> . make_EPSG
> >>
> >> Do you know of any (more or less) equivalent functions in the packages
> >> you mentioned that have taken over from rgdal?
> >>
> >> Thanks in advance,
> >>
> >> Jean-Luc Dupouey
> >>
> >> --
> >> INRAE
> >> UMR Silva
> >> F-54280 Champenoux
> >> France
> >>
> >> ___
> >> R-sig-Geo mailing list
> >> R-sig-Geo@r-project.org
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>
> >
> >
> >
>
> --
> Roger Bivand
> Emeritus Professor
> Department of Economics, Norwegian School of Economics,
> Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway.
> e-mail: roger.biv...@nhh.no
> https://orcid.org/-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0J=en
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] rgdal retirement
Hi Jean-Luc, if the packages mentioned don't yet cover it, you can get ogrInfo and ogrFIDs with a combination of vapour's vapour_read_names() (for FID), vapour_geom_summary() (for FID and type) and vapour_layer_info(). For make_EPSG() it's probably time to hit the proj.db more directly and I'd be happy to have a look at that. Cheers, Mike On Sun, Sep 19, 2021 at 9:35 AM Jean-Luc Dupouey wrote: > Dear Roger Bivand, > > I read your message about the rgdal retirement. It was a nice package > and an important step in the development of R-spatial. > > I looked for alternatives for the functions I was using, but I didn't > find them all. Those I am missing: > > . ogrInfo > . ogrFIDs > . make_EPSG > > Do you know of any (more or less) equivalent functions in the packages > you mentioned that have taken over from rgdal? > > Thanks in advance, > > Jean-Luc Dupouey > > -- > INRAE > UMR Silva > F-54280 Champenoux > France > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Create a ppp object in UTM with mixture of zones
On Sat, Jul 31, 2021 at 6:09 AM Alexandre Santos <
alexandresanto...@yahoo.com.br> wrote:
>
> Thanks, Mike
>
> Problem solved with a transformation of to Lambert azimuthal equal-area
projection centred on longitude and latitude of 0:
>
> # Packages
> library(spatstat)
> library(sf)
> library(sp)
> library(raster)
>
>
> # Open spatial data set in GitHub
> sp_ds<-read.csv("
https://raw.githubusercontent.com/Leprechault/trash/master/myspds.csv;)
> str(sp_ds)
> #'data.frame': 4458 obs. of 2 variables:
> # $ Lat : num 9.17 9.71 9.12 9.12 9.71 ...
> # $ Long: num 35.8 35.5 35.8 35.8 35.5 ...
>
>
> sp_ds <- st_read("
https://raw.githubusercontent.com/Leprechault/trash/master/myspds.csv;)
> sp_ds_sf <- st_as_sf(sp_ds, coords = c("Lat", "Long"), crs = 4326)
>
>
> # Here the Mike tip!!
> sp_ds_laea = st_transform(sp_ds_sf,
>crs = "+proj=laea +x_0=0 +y_0=0 +lon_0=0
+lat_0=0")
great, but you should centre the projection for your data - +lon_0 and
+lat_0 specify a central-ish point for your longitude,latitude values. Plot
the data on a map in long lat, and then plot them once projected - it looks
pretty weird with centre lon,lat at 0,0 . - you want more like "+proj=laea
+lon_0=29.9 +lat_0=13.4 +datum=WGS84" - the x_0 and y_0 may be specified,
they just move the 0,0 of the projection away from the centre of your map
(nice to not have negative values). It's likely you could find a local
authority projection that is defined for general use (laea, lcc, aeqd are
common).
Also, you are using Lat,Long order which is technically correct for
EPSG:4326, other variants of longitude,latitude specifications use the more
straightforward coordinate order, it's configurable and sf handles it etc.
but you should definitely check :)
Finally, it's probably worth saying in this case for these data you could
probably choose the central (of the 3) UTM zones involved and get just as
good properties ( I never see that said in the wild). But, it's not really
possible to have overarching advice here because your data and your context
and your distances can be a very broad set of cases, that's why different
families of projections exist, for options. You can calculate distance and
area in long,lat with geodist, geosphere, sf and many other packages - and
sf is incorporating s2 to enable that always for data in long,lat - but I
assume you need an actual coordinate system for spatstat to operate in its
own context (so your approach is exactly right). The choice of projection
can be subtle and require expertise, but a local central laea for smallish
regions is pretty fail safe. Over a small area like this, most projections
will be exactly the same in terms of distance, area, shape - but not
knowing the broader case of data you might want to deal with means such
advice needs to be issued with care. :)
HTH
> # Create boudaries using sf
> ch_ds <- st_convex_hull(st_union(sp_ds_laea))
> W<- as.owin(ch_ds)
>
>
> # Create a ppp Point Pattern with sf object
> out.ppp <- as.ppp(X=st_coordinates(sp_ds_laea),W=W)
> plot(out.ppp)
>
>
>
>
> # Make a CRS test
> f1 <- ppm(out.ppp~1)
> E <-envelope(f1, Kinhom, nsim = 19, global = TRUE, correction = "border")
> plot(E)
>
> Best wishes,
>
> Alexandre
>
>
>
>
>
>
>
>
> Em sexta-feira, 30 de julho de 2021 15:41:08 BRT, Michael Sumner <
mdsum...@gmail.com> escreveu:
>
>
>
>
>
> Don't use UTM, it's a legacy of a bygone era. Use a local custom
projection such as laea:
>
> https://twitter.com/mdsumner/status/1136794870113218561?s=19
>
> I know it's common advice, utm for projection choice but it's trash
propagated by folks who should know better. Traversing zones for projection
choice for basic distance properties is unnecessary.
>
> Best, Mike
>
> On Fri, 30 Jul 2021, 08:21 Alexandre Santos via R-sig-Geo, <
r-sig-geo@r-project.org> wrote:
> > Dear Members,
> >
> >
> > I'd like to use my spatial data set ("sp_ds") as ppp (spatstat Point
Pattern object), an I try to do:
> >
> >
> >
> > # Open spatial data set in GitHub
> > library(spatstat)
> > library(sf)
> > library(sp)
> >
> >
> > sp_ds<-read.csv("
https://raw.githubusercontent.com/Leprechault/trash/master/myspds.csv;)
> > str(sp_ds)
> > #'data.frame': 4458 obs. of 2 variables:
> > # $ Lat : num 9.17 9.71 9.12 9.12 9.71 ...
> > # $ Long: num 35.8 35.5 35.8 35.8 35.5 ...
> >
> >
> > # Create boudaries using sf
> > sfds = st_as_sf(sp_ds, coords=c("Long","Lat"), crs=4326)
> > traps<-sp_ds
> > ch <- chull(traps[,c(2,1)])
> > c
Re: [R-sig-Geo] Create a ppp object in UTM with mixture of zones
Don't use UTM, it's a legacy of a bygone era. Use a local custom projection
such as laea:
https://twitter.com/mdsumner/status/1136794870113218561?s=19
I know it's common advice, utm for projection choice but it's trash
propagated by folks who should know better. Traversing zones for projection
choice for basic distance properties is unnecessary.
Best, Mike
On Fri, 30 Jul 2021, 08:21 Alexandre Santos via R-sig-Geo, <
r-sig-geo@r-project.org> wrote:
> Dear Members,
>
>
>
> I'd like to use my spatial data set ("sp_ds") as ppp (spatstat Point Pattern
> object), an I try to do:
>
>
>
> # Open spatial data set in GitHub
> library(spatstat)
> library(sf)
> library(sp)
>
>
> sp_ds<-read.csv("
> https://raw.githubusercontent.com/Leprechault/trash/master/myspds.csv;)
> str(sp_ds)
> #'data.frame': 4458 obs. of 2 variables:
> # $ Lat : num 9.17 9.71 9.12 9.12 9.71 ...
> # $ Long: num 35.8 35.5 35.8 35.8 35.5 ...
>
>
> # Create boudaries using sf
> sfds = st_as_sf(sp_ds, coords=c("Long","Lat"), crs=4326)
> traps<-sp_ds
> ch <- chull(traps[,c(2,1)])
> coords <- traps[c(ch, ch[1]), ]
> coordinates(coords) <- c("Long","Lat")
> proj4string(coords) <- CRS("+init=epsg:4326")
> W <- owin(poly=cbind(coordinates(coords)[,2],coordinates(coords)[,1]))
>
>
>
> # Create a ppp Point Pattern object
> out.ppp<-ppp(x=sp_ds$Lat,y=sp_ds$Long,window=W)
> plot(out.ppp)
>
>
> f1 <- ppm(out.ppp~1)
> E <-envelope(f1, Kinhom, nsim = 19, global = TRUE, correction = "border")
> plot(E)
>
>
> #But I'd like to r distance (x axis)
> in kilometers and for this I need to convert the coordinate reference system
> of 4326 to
> UTM, a have 3 UTM zones:
> #34N bounds: (18.0, 0.0, 24.0, 84.0)
> #35N bounds: (24.0, 0.0, 30.0, 84.0)
> #36N bounds: (30.0, 0.0, 36.0, 84.0)
>
>
> Please the area a simple way to create a ppp object in UTM with a
> mixture of zones?
>
>
> Thanks in advance,
>
>
> Alexandre
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] How to project a SpatExtent object?
I think you can project the empty rast() object, using the same idiom as
{raster} to create an empty version:
f <- system.file("ex/elev.tif", package="terra", mustWork = TRUE)
r <- rast(f)
terra::project(terra::rast(r), "+proj=laea")
that seems analogous to what raster::projectExtent does.
HTH, Mike
On Tue, Jul 6, 2021 at 6:14 AM Tim Elrick wrote:
> Hi all,
>
> Does anyone know how to (re-)project a SpatExtent object from the terra
> package?
>
> project() only works on SpatVector or SpatRaster, but vect() doesn't
> work on a SpatExtent either to convert a SpatExtent to a SpatVector.
>
> Thanks,
>
> Tim
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] How to define a local equidistant CRS
Hi again, at that kind of range - 15km - you could choose almost any
projection family and a custom centre. I don't think distance accuracy fors
those distances would be an issue even over a continent like USA or
Australia. AEQD or LAEA or LCC , doesn't matter. What you might consider is
a single custom projection (or an actual authority one, such as Abers for
USA, or LCC for Australia (they exist)) that is suitable for all of your
sites to avoid complexity.
(I avoid UTM because it's rarely suitable, sometimes its actual properties
are of use but mostly it's a legacy from a previous age, it's only in wide
use because authorities did deploy it for local use in many many
jurisdictions, and it's easy to calculate.)
For custom use I find it simpler to pick a long-lat centre for a LAEA, it
requires less effort and is more robust *globally* that any UTM choice.
In terms of your locality, the actual region over which your sites might
be, maybe you do need a non-WGS84 datum, and perhaps a non-temporal or an
actual specific-temporal datum family - but I don't get that impression
from your description.
Cheers, Mike
On Mon, Jun 14, 2021 at 11:54 PM Micha Silver wrote:
> Hello Mike:
>
>
> Thanks for addressing this issue.
>
>
> A few more details might make the overall goal clearer. The RAINLINK
> package converts cellular signal attenuation between microwave towers to
> rain rate (path averaged depending on the distance between the towers).
> Typically the tower locations will be in longitude/latitude, but RAINLINK
> needs Cartesian coordinates in three functions:
>
>
> 1- An important first step in RAINLINK determines if the microwave antenna
> itself is wet (causing attenuation even before hitting any rain). The
> approach adopted in the package is to find other antennas in the time and
> space vicinity that also are experiencing attenuation. Here we need to get
> those antennas that are within, say, 15 km. This issue can be easily
> addressed with the distVincenty* functions from geosphere, or (as Roger
> suggested) using the new s2 package for calculating distances on a sphere.
> No need really for any reprojection (submeter accuracy is not an issue).
>
>
> 2- RAINLINK also produces some informative plots of the topology of the
> microwave link network. Among them bar plots of distances between towers
> vs frequency, etc. Here, cartesian distance is needed, but again, we could
> probably get away with either of the distance on sphere functions above.
>
>
> 3- The more tricky problem (I think) is the RAINLINK interpolation
> function. Here, both IDW and kriging are used to produce precipitation
> distribution maps. A Cartesian CRS is needed/preferred, I believe, both to
> construct the variogram model, and to create the target interpolation grid.
>
>
> So, my intuition is to choose right at the start a "proper" projected CRS
> depending on the location on the earth, and use that throughout. In the
> current version of RAINLINK a hand-crafted aeqd CRS was defined from the
> middle of the distribution of antennas. But now, with proj >=6.x
> definitions like:
>
> projstring <- paste("+proj=aeqd +R_A +lat_0=", YMiddle,
> " +lon_0=", XMiddle,
> " +x_0=0 +y_0=0 +ellps=WGS84",sep="")
>
> might not be future-proof. The second option could be a local UTM based
> CRS, getting the zone from the XMiddle, Ymiddle values.
>
>
> Any other suggestions are welcome!
>
>
> Best,
>
> Micha
>
>
>
>
>
>
>
> On 6/11/21 5:16 AM, Michael Sumner wrote:
>
> I'd do
>
> local_aeqd <- function(lon, lat) {
> sprintf("+proj=aeqd +lat_0=%f +lon_0=%f +datum=WGS84", lat, lon)
> }
>
> local_aeqd(147, -42)
>
> just note that the very high accuracy of distances only applies from
> points to that central point - but for a fair region around probably suits
> your purposes well also and definitely a 15km region, the equidistant
> property is unlikely to matter at that scale, for general use I go for laea
> or if I have a definite extent and for larger regions >=1000km lcc
> with +lat_1 and +lat_2 set to the north/south extent.
>
> But, do you really need a projection or just distance between points of
> long-lat? (Can't tell from your description). I use geosphere or geodist
> routinely, the same code (Karney) exists in the sf stack as well.
>
> Cheers, Mike.
>
> On Sun, Jan 24, 2021 at 2:56 AM Micha Silver wrote:
>
>>
>> On 1/23/21 1:05 PM, Roger Bivand wrote:
>>
>> On Sat, 23 Jan 2021, Micha Silver wrote:
>>
>>
>> Roger:
>>
>>
>>
>>
>> Yes, but the exceptions for WGS84, NAD83 and NAD27
Re: [R-sig-Geo] How to define a local equidistant CRS
I'd do
local_aeqd <- function(lon, lat) {
sprintf("+proj=aeqd +lat_0=%f +lon_0=%f +datum=WGS84", lat, lon)
}
local_aeqd(147, -42)
just note that the very high accuracy of distances only applies from points
to that central point - but for a fair region around probably suits your
purposes well also and definitely a 15km region, the equidistant property
is unlikely to matter at that scale, for general use I go for laea or if I
have a definite extent and for larger regions >=1000km lcc with +lat_1
and +lat_2 set to the north/south extent.
But, do you really need a projection or just distance between points of
long-lat? (Can't tell from your description). I use geosphere or geodist
routinely, the same code (Karney) exists in the sf stack as well.
Cheers, Mike.
On Sun, Jan 24, 2021 at 2:56 AM Micha Silver wrote:
>
> On 1/23/21 1:05 PM, Roger Bivand wrote:
>
> On Sat, 23 Jan 2021, Micha Silver wrote:
>
>
> Roger:
>
>
>
>
> Yes, but the exceptions for WGS84, NAD83 and NAD27 are just that,
> exceptions from the general rule of +datum= being dropped. The next road
> bump will occcur soon(ish) because WGS84 is an ensemble of datum
> specifications, not a unique specification, and PROJ 7.2.0 is starting to
> address this: https://proj.org/news.html (Update to EPSG 10.003 and make
> code base robust to dealing with WKT CRS with DatumEnsemble (#2370)). The
> link to https://github.com/OSGeo/PROJ/pull/2370 gives details.
>
> Unless you need sub-metre accuracy, this may not matter, but if different
> parts of the world use different WGS84 definitions (at different times),
> the s2 package may be more stable though a little less accurate.
>
> Sub meter accuracy is definitely not an issue in our use case. The
> microwave antennas are from 1-12 km apart. We need the list of antennas
> that are within a radius of about 15 km. Missing one antenna that is, on a
> sphere, a few 10's of meters too far is of no consequence here.
>
>
> This is more detail than you expected, I think, but at least it is a way
> of documenting things.
>
> Well, like you said, you can't get the sweet potatoes back after cooking
> orange vegetable soup.
>
> :-)
>
>
> Roger
>
>
>
> If this isn't a reprex, please extend briefly.
>
> Roger
>
>
> Thanks again,
>
> Regards, Micha
>
>
>
>
> --
> Micha Silver
> Ben Gurion Univ.
> Sde Boker, Remote Sensing Lab
> cell: +972-523-665918
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Reprojection doubt
It probably doesn't matter at all. Your data is longitude,latitude on datum WGS84? Those strings are equivalent (some softwares will flip the order of lon,lat as that's what the standard dictates but easy to spot that usually). raster::projection(), raster::crs(), sp::proj4string() all are fairly equivalent functions, they just get the projection metadata - they are just overlapping and inconsistent because of historical vagarie, they differ in ideological stance and how much checking and churn they do to the metadata and in efforts to interpret your intentions. But on the face of it I don't think you need to worry about your code here. Just keep an eye on whether your data looks right, before and after reprojection. It takes care, but the details are yours alone for checking unless you can pin down a reproducible example that raises questions. Good luck. fwiw you seem to not be doing anything with ncdf4 (or ncdf4::nc_open) here. the raster package *uses* ncdf4 on your behalf via raster()/stack()/brick() functions. Cheers, Mike. On Fri, Nov 6, 2020 at 12:11 PM Pietro Andre Telatin Paschoalino < pietro_tel...@hotmail.com> wrote: > Dear, > > I have a question that is possibly extremely easy for you. > > I'm extracting data from layers of grid to a shape file. > > To do this, first, I know that I need to reproject the coordinates > reference system of my shape to the same CRS as my grid. The code for this > is: > > library(ncdf4) > > fn<-file.path(path of my data) > > ncfile <- nc_open(fn) > > rasbrick <- stack(fn) > > rasbrick <- brick[[70:81]] (the layers that I want) > > shape2<- spTransform(shape, crs(rasbrick)) in lowercase, right? It is my > first question. > > The rest of the extraction code works without any problems. If I open this > file in qgis the CRS is: > > OGC:CRS84 - WGS 84 (CRS84) - Geographic > > The problem is that if I write a raster of a layer using: > > rasbrick <- brick[[70]] > > writeRaster(rasbrick, "test.tif", format="GTiff") > > and open the raster in qgis the CRS is: > > EPSG:4326 - WGS 84 - Geographic > > So, my second question is why on the raster it looks like EPSG and on the > shape it looks like OGC? And could this cause any problems in extracting > data from layers for the shape? If yes, what I need to change to reproject > the shape for the correct CRS? > > Thanks. > > Pietro Andre Telatin Paschoalino > Doutorando em Ciências Econômicas da Universidade Estadual de Maringá - > PCE. > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] subsetting terra::raster
you could do this for a single layer object:
p <- which(values(r) > 0)# Possibly faster to use values(r > 0) > 0
cbind(xyFromCell(r,p),extract(r,p))
as long as getting all values is memory-safe (I'm not sure if there's a
Which analog in terra).
HTH
On Tue, Jul 14, 2020 at 4:02 PM Herr, Alexander (L, Black Mountain)
wrote:
> You are right,
>
> What I really want is extract cell coordinates and their values for cells
> with values >0
>
>
>
> So I could use in raster
>
> xyz <- rasterToPoints(r, fun=function(x){x>0}, xy=T)
>
> or
>
> Which(r>0,cells=T)->p
>
> cbind(xyFromCell(r,p),extract(r,p))
>
>
>
> what’s the terra alternative?
>
>
>
>
>
>
>
>
>
> *From:* Michael Sumner
> *Sent:* Tuesday, 14 July 2020 2:41 PM
> *To:* Herr, Alexander (L, Black Mountain)
> *Cc:* r-sig-geo@r-project.org
> *Subject:* Re: [R-sig-Geo] subsetting terra::raster
>
>
>
> The second argument to xyFromCell is a *cell number*, but in `r[r>0]` you
> have a *cell value*, which may not be what you want.
>
>
>
> It's the r[r > 0] part that's not working in terra, so you could do this
> as a workaround ...
>
>
>
> xy <- xyFromCell(r, values(r)[values(r) >0])
>
>
>
> (but are your cell values really intended for use as cell numbers? It's
> certainly useful for that but probably uncommon).
>
>
>
> Your code also did not set the extent in the terra object, or set.seed so
> there's several steps that lack compatibility.
>
>
>
> Cheers, Mike
>
>
>
>
>
> On Tue, Jul 14, 2020 at 2:18 PM Herr, Alexander (L, Black Mountain) <
> alexander.h...@csiro.au> wrote:
>
> Hi,
>
> I am moving from raster to terra. I get an error when I adapt the
> following raster code:
> require(raster)
> set.seed(666)
> reps=10
> r <- raster( matrix(nrow=reps,ncol=reps, round(rnorm(reps^2),1)))
> extent(r)<- c(1,101,1,101)
> plot(r)
> xy<-xyFromCell(r,r[r>0])
> z<-r[r>0]
> cbind(xy,z)
>
> Under terra code:
> require(terra)
> r <- rast( matrix(nrow=reps,ncol=reps, round(rnorm(reps^2),1)))
> plot(r)
> xy<-xyFromCell(r,r[r>0])
> Error in r[r > 0] : object of type 'S4' is not subsettable
>
> What am I doing wrong?
>
> Thanks
> herry
>
>
> in terra
>
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
>
>
>
> --
>
> Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> Hobart, Australia
> e-mail: mdsum...@gmail.com
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] subsetting terra::raster
The second argument to xyFromCell is a *cell number*, but in `r[r>0]` you have a *cell value*, which may not be what you want. It's the r[r > 0] part that's not working in terra, so you could do this as a workaround ... xy <- xyFromCell(r, values(r)[values(r) >0]) (but are your cell values really intended for use as cell numbers? It's certainly useful for that but probably uncommon). Your code also did not set the extent in the terra object, or set.seed so there's several steps that lack compatibility. Cheers, Mike On Tue, Jul 14, 2020 at 2:18 PM Herr, Alexander (L, Black Mountain) wrote: > Hi, > > I am moving from raster to terra. I get an error when I adapt the > following raster code: > require(raster) > set.seed(666) > reps=10 > r <- raster( matrix(nrow=reps,ncol=reps, round(rnorm(reps^2),1))) > extent(r)<- c(1,101,1,101) > plot(r) > xy<-xyFromCell(r,r[r>0]) > z<-r[r>0] > cbind(xy,z) > > Under terra code: > require(terra) > r <- rast( matrix(nrow=reps,ncol=reps, round(rnorm(reps^2),1))) > plot(r) > xy<-xyFromCell(r,r[r>0]) > Error in r[r > 0] : object of type 'S4' is not subsettable > > What am I doing wrong? > > Thanks > herry > > > in terra > > > ___________ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Help with spTransform() function and final plot colors
Hi, your longlat coordinates are in the western hemisphere (-119, 39) but
EPSG 27700 is designed for (and appropriate for some scenarios) somewhere
in the UK. You need a local projection suitable for the area you are in.
Look for an EPSG for your region, or maybe Albers conic or Lambert
Azimuthal local authority (look on a paper map if you have one). Or define
your own with projstrings (or wkt, it's a bit complicated atm to advise
simple construction). There's some discussion here:
https://geocompr.robinlovelace.net/reproj-geo-data.html#which-crs-to-use
Some will say use UTM, but you should not unless it's a particular local
one used commonly in your work and the region the data is from.
HTH, Mike
On Mon., 18 May 2020, 19:46 Poling, William via R-sig-Geo, <
r-sig-geo@r-project.org> wrote:
> #RStudio Version Version 1.2.1335
> sessionInfo()
> # R version 4.0.0 Patched (2020-05-03 r78349)
> #Platform: x86_64-w64-mingw32/x64 (64-bit) #Running under: Windows 10 x64
> (build 17763)
>
> Hello. I am running my data through a routine I found that finds clusters
> of data points based on distance rule.
>
> https://gis.stackexchange.com/questions/64392/finding-clusters-of-points-based-distance-rule-using-r
>
> 1. I get this error when I get to this point in the routine, see complete
> routine below?
> xy <- spTransform(xy, CRS("+init=epsg:27700 +datum=WGS84")) non finite
> transformation detected:
>
> I tried converting columns from numeric to Integer but did not help.
>
> However, I continue to run the rest of the routine and the final sequence,
> the plot itself, seems to work Can this error be corrected somehow, despite
> the fact that it continues to work, just curious what it is I guess"
>
> 2. However in the plot at the end of the routine the color key appears
> with the colors but the clusters in the plot are black, see plot call at
> the end of the routine below?
>
> Thank you for any advice.
>
> WHP
>
> ERROR:
> non finite transformation detected:
> coords.x1 coords.x2
> [1,] -119.7339 39.53939 Inf Inf
> [2,] -119.7665 39.39630 Inf Inf
> [3,] -119.7794 39.28768 Inf Inf
> [4,] -121.0234 39.20503 Inf Inf
> [5,] -122.0047 47.19262 Inf Inf
> [6,] -122.0135 47.18883 Inf Inf
> [7,] -122.0379 47.52190 Inf Inf
> [8,] -122.0578 47.60975 Inf Inf
> [9,] -122.1330 47.13669 Inf Inf
> [10,] -122.1509 47.55962 Inf Inf
> [11,] -122.1706 47.15546 Inf Inf
> [12,] -122.1846 47.23485 Inf Inf
> [13,] -122.1846 48.15307 Inf Inf
> [14,] -122.1851 47.44870 Inf Inf
> [15,] -122.1954 47.68485 Inf Inf
> [16,] -122.1990 47.51610 Inf Inf
> [17,] -122.2014 47.44772 Inf Inf
> [18,] -122.2025 47.69815 Inf Inf
> [19,] -122.2037 47.67190 Inf Inf
> [20,] -122.2090 47.40378 Inf Inf
> [21,] -122.2108 47.25336 Inf Inf
> [22,] -122.2291 47.63880 Inf Inf
> [23,] -122.2419 47.76870 Inf Inf
> [24,] -122.2722 48.04803 Inf Inf
> [25,] -122.2732 47.87700 Inf Inf
> [26,] -122.2804 47.77620 Inf Inf
> [27,] -122.2839 47.82103 Inf Inf
> [28,] -122.2890 47.86899 Inf Inf
> [29,] -122.2993 47.67306 Inf Inf
> [30,] -122.3180 47.38217 Inf Inf
> [31,] -122.3270 47.40378 Inf Inf
> [32,] -122.3474 47.43884 Inf Inf
> [33,] -122.3484 47.53083 Inf Inf
> [34,] -122.3581 47.27678 Inf Inf
> [35,] -122.3618 47.76735 Inf Inf
> [36,] -122.3700 47.56567 Inf Inf
> [37,] -122.3908 47.54938 Inf Inf
> [38,] -122.4128 47.64622 Inf Inf
> [39,] -122.4293 47.17660 Inf Inf
> [40,] -122.4621 47.44505 Inf Inf
> [41,] -122.4904 47.27460 Inf Inf
> [42,] -122.5515 46.93979 Inf Inf
> [43,] -122.7348 42.37320 Inf Inf
> [44,] -122.7827 47.31059 Inf Inf
> [45,] -122.7987 47.23475 Inf Inf
> [46,] -122.8385 42.35119 Inf Inf
> [47,] -122.8537 42.34495 Inf Inf
> [48,] -122.8904 42.39555 Inf Inf
> [49,] -122.8927 42.33022 Inf Inf
> [50,] -122.9451 47.37574 Inf Inf
> [51,] -122.9594 42.30376 Inf Inf
> [52,] -123.0641 47.16428 Inf Inf
> [53,] -123.3413 42.44117 Inf Inf
> Error in spTransform(xSP, CRSobj, ...) :
> failure in points
> 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20:21:22:23:24:25:26:27:28:29:30:31:32:33:34:35:36:37:38:39:40:41:42:43:44:45:46:47:48:49:50:51:52:53
> In addition: Warning message:
> In spTransform(xSP, CRSobj, ...) : 53 projected point(s) not finite
>
> Here is the data:
> dput(sample)
> structure(list(ID = 1:53, Longitude = c(-119.733899, -119.766493,
> -119.779416, -121.0234, -122.004736, -122.013456, -122.0379, -122.0578,
> -122.132971, -122.150901, -122.170608, -122.18462, -122.184639,
> -122.185079, -122.195398, -122.198994, -122.201356, -122.202507,
> -122.20371, -122.209047, -122.210797, -122.229095, -122.2419, -122.27216,
> -122.273164, -122.280355, -122.28389, -122.289043, -122.299261,
> -122.318009, -122.326987, -122.347382, -122.34844, -122.358115,
> -122.361839, -122.37003, -122.390815, -122.41282, -122.429323, -122.462136,
> -122.490417, -122.551483, -122.734847, -122.782736, -122.798669,
> -122.838498, -122.853683, -122.8904, -122.89271, -122.94511, -122.959407,
>
Re: [R-sig-Geo] Help: assign crs to RasterBrick
You need to also setExtent() of the raster. That's the coordinates that the crs refers to xmin,xmax,ymin,ymax values in metres. Sometimes visual cues can be used to derive these from an image, sometimes the grid is a commonly used convention (a certain dimension, I.e. extent +resolution in a region). It really needs the file source itself or some info from the provider where the data was already a map. Not much can be done from a picture alone though. I'll have a look around sometimes wild guesses work because projection-use is rarely bespoke. Best On Mon., 27 Apr. 2020, 01:21 Jesús Rojo, wrote: > Hello everyone > I have a flipped RasterBrick without crs assigned (crs = NA): > plot(a[[100]] > I would like to fill the map and assign the correct coordinate systems, > that it has been said to me to be specifically: > > "+init=EPSG:3034 +proj=lcc +lat_1=35 +lat_2=65 +lat_0=50 +lon_0=10 > +x_0=400 +y_0=280 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m > +no_defs” > > I try to solve with the next commands for the raster library: > > In a first step I have flipped the map, and it has been right with: > > a <- t(flip(a, direction='y' )) > > But I have problems with the assignments of the suitable crs. > > crs(a) <- "+init=EPSG:3034 +proj=lcc +lat_1=35 +lat_2=65 +lat_0=50 > +lon_0=10 +x_0=400 +y_0=280 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 > +units=m +no_defs” > > Because although the map assigns crs = “+init = EPSG:3034”, the map > doesn’t seem georreferenced, because the starting point of the map is > (0,0). My final objective is to transform the crs to "+proj=longlat > +datum=WGS84” > > A <- projectRaster(a, crs = "+proj=longlat +datum=WGS84") > I will be grateful some kind of help. Do I need more additional > information? Any step is wrong? > Thank you so much > Jesús Rojo > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] rgdal: problem reading a bigger raster dataset (R 4.0.0/3.6.3, Ubuntu 20.04)
I reckon that's unlikely, it's also unlikely to scale or be fast enough ever given what rgdal is - raster will happily crop sections of it out but I guess your workflow is to write the geotiff out in one go? You want rgdal because you want raster? Have you tried terra? (I forgot to mention that one, and I haven't tried on that size GeoTIFF with it. It has its own internal connection to GDAL). Possibly you can instantiate a GeoTIFF with rgdal of the right size, then write to it piecewise - but if the convenience of raster is what you're after that's obviously no good. I think I'm missing some workflow constraint or motivation you have, but I'm interested (it's an awkward space). Cheers, Mike On Tue, Apr 28, 2020 at 7:14 PM Thorsten Behrens < thorsten.m.behr...@gmail.com> wrote: > Michael, > > Thanks for the hint, it seems to work! Real-world tests will follow in > the next few days... > > So it definitely seems to be a problem of rgdal. It would be great if it > could still be solved. > > Best, > > Thorsten > > > > Am 27.04.2020 um 15:58 schrieb Michael Sumner: > > Try stars it worked for me on a test > > > > On Mon., 27 Apr. 2020, 23:54 Thorsten Behrens, > > mailto:thorsten.m.behr...@gmail.com>> > > wrote: > > > > Roger, > > > > thanks a lot for your reply! > > > > I have 256GB RAM installed (mentioned it somewhere). And there, > > all is > > fine when I run: > > > > rDemTest = raster(nrow = 48000, ncol = 72000, ext = extent(c(0, > 72000, > > > > values(rDemTest) = 1 > > > > When limiting the memory to about 8GB with > > ulimit::memory_limit(8000), > > the max size which can be allocated seems to be around 1 x > > 1px. > > In this case all tests run fine. Unfortunately it seems to be > > related to > > the size of the grid (48000 x 72000) and therefore the problem > > can't be > > reproduced on machines with 8GB RAM. For some processing steps I need > > grids of that size in the memory, which is why I have 256 GB > > installed. > > > > Normally, I use the raster package and not rgdal::readGDAL(). This > > was > > just a desperate attempt to find the source of the problem. > > > > This is what I use in my code: > > > > rDem = raster(sFileNameTiff) > > mDem = raster::as.matrix(rDem) > > > > But maybe this is the same... > > > > Any further suggestions are much appreciated! > > > > Thanks again! > > > > Best, > > > > Thorsten > > > > > > > > > > Am 27.04.2020 um 14:50 schrieb Roger Bivand: > > > On Mon, 27 Apr 2020, Thorsten Behrens wrote: > > > > > >> Dear all, > > >> > > >> my problem is that I want to read a big geotiff raster dataset > > into R > > >> and convert it to a matrix, which does not work. > > >> The file is big but there is sufficient memory. I need all the > > data > > >> in the memory at the same time. > > >> > > >> The error occurs under R 3.6.3 as well as 4.0.0 using Ubuntu 20.04 > > >> LTS with the latest version of the packages (see session info > > below) > > >> and 256GB RAM installed. > > >> > > >> When loading the raster dataset using rgdal (via readGDAL or > > >> raster::readAll) I get the follwoing error in R 4.0.0: > > >> > > >> ``` > > >> Error in rgdal::getRasterData(con, offset = offs, region.dim = > > reg, > > >> band = object@data@band) : > > >> long vectors not supported yet: memory.c:3782 > > >> ``` > > > > > > On a 16GB Fedora platform: > > > > > >> library(raster) # 3.1-5 > > >> rDemTest = raster(nrow = 48000, ncol = 72000, ext = extent(c(0, > > 72000, > > > 0, > > > + 48000))) # all fine > > >> rDemTest > > > class : RasterLayer > > > dimensions : 48000, 72000, 3.456e+09 (nrow, ncol, ncell) > > > resolution : 1, 1 (x, y) > > > extent : 0, 72000, 0, 48000 (xmin, xmax, ymin, ymax) > > > crs: NA > > > > > >> values(rDemTest) = 1 > > > Error: cannot allocate vector of size 25.7 Gb > > > > > > So you are deceiving yourself into thinking t
Re: [R-sig-Geo] rgdal: problem reading a bigger raster dataset (R 4.0.0/3.6.3, Ubuntu 20.04)
Oh nice one, Roger I'd lost sight of all that churn there.
Cheers, Mike
On Tue, Apr 28, 2020 at 8:11 PM Roger Bivand wrote:
> On Tue, 28 Apr 2020, Thorsten Behrens wrote:
>
> > Michael,
> >
> > Thanks for the hint, it seems to work! Real-world tests will follow in
> > the next few days...
> >
> > So it definitely seems to be a problem of rgdal. It would be great if it
> > could still be solved.
>
> Not rgdal, but your use of it. Try looking at a sequence of
>
> file <- system.file("pictures/SP27GTIF.TIF", package="rgdal")
> obj <- GDAL.open(file)
> (dims <- dim(obj))
> band <- 1
> data_vector <- getRasterData(obj, band)
> GDAL.close(obj)
> str(data_vector)
>
> This does not create any more complicated objects, just a matrix. In some
> cases, the rows in the matrix are ordered S -> N, so it may appear the
> wrong way up.
>
> rgdal::getRasterData() is lightweight, and has many arguments which may be
> helpful. rgdal::readGDAL() is heavyweight, creating a SpatialGridDataFrame
> object. This involves much copying of data, but the output object can be
> used for example in mapping or analysis without further conversion. My
> guess is that rgdal::getRasterData() gives you your matrix directly. Look
> at the R code to see how as.is= etc. work (files may include scale and
> offset values - recently a user was confused that scale and offset were
> "magically" applied to convert Uint16 values to degrees Kelvin on
> reading). For example, if as.is == TRUE or scale == 1 and offset == 0, no
> copying of the input matrix occurs because it is not converted. If you
> could check this route, others following this thread could also benefit;
> if I'm wrong, that would also be good to know.
>
> Roger
>
>
> >
> > Best,
> >
> > Thorsten
> >
> >
> >
> > Am 27.04.2020 um 15:58 schrieb Michael Sumner:
> >> Try stars it worked for me on a test
> >>
> >> On Mon., 27 Apr. 2020, 23:54 Thorsten Behrens,
> >> mailto:thorsten.m.behr...@gmail.com>>
> >> wrote:
> >>
> >> Roger,
> >>
> >> thanks a lot for your reply!
> >>
> >> I have 256GB RAM installed (mentioned it somewhere). And there,
> >> all is
> >> fine when I run:
> >>
> >> rDemTest = raster(nrow = 48000, ncol = 72000, ext = extent(c(0,
> 72000,
> >>
> >> values(rDemTest) = 1
> >>
> >> When limiting the memory to about 8GB with
> >> ulimit::memory_limit(8000),
> >> the max size which can be allocated seems to be around 1 x
> >> 1px.
> >> In this case all tests run fine. Unfortunately it seems to be
> >> related to
> >> the size of the grid (48000 x 72000) and therefore the problem
> >> can't be
> >> reproduced on machines with 8GB RAM. For some processing steps I
> need
> >> grids of that size in the memory, which is why I have 256 GB
> >> installed.
> >>
> >> Normally, I use the raster package and not rgdal::readGDAL(). This
> >> was
> >> just a desperate attempt to find the source of the problem.
> >>
> >> This is what I use in my code:
> >>
> >> rDem = raster(sFileNameTiff)
> >> mDem = raster::as.matrix(rDem)
> >>
> >> But maybe this is the same...
> >>
> >> Any further suggestions are much appreciated!
> >>
> >> Thanks again!
> >>
> >> Best,
> >>
> >> Thorsten
> >>
> >>
> >>
> >>
> >> Am 27.04.2020 um 14:50 schrieb Roger Bivand:
> >>> On Mon, 27 Apr 2020, Thorsten Behrens wrote:
> >>>
> >>>> Dear all,
> >>>>
> >>>> my problem is that I want to read a big geotiff raster dataset
> >> into R
> >>>> and convert it to a matrix, which does not work.
> >>>> The file is big but there is sufficient memory. I need all the
> >> data
> >>>> in the memory at the same time.
> >>>>
> >>>> The error occurs under R 3.6.3 as well as 4.0.0 using Ubuntu 20.04
> >>>> LTS with the latest version of the packages (see session info
> >> below)
> >>>> and 256GB RAM installed.
> >>>>
> >>>> When loading the raster dataset using rgdal (via readGD
Re: [R-sig-Geo] rgdal: problem reading a bigger raster dataset (R 4.0.0/3.6.3, Ubuntu 20.04)
Try stars it worked for me on a test On Mon., 27 Apr. 2020, 23:54 Thorsten Behrens, wrote: > Roger, > > thanks a lot for your reply! > > I have 256GB RAM installed (mentioned it somewhere). And there, all is > fine when I run: > > rDemTest = raster(nrow = 48000, ncol = 72000, ext = extent(c(0, 72000, > > values(rDemTest) = 1 > > When limiting the memory to about 8GB with ulimit::memory_limit(8000), > the max size which can be allocated seems to be around 1 x 1px. > In this case all tests run fine. Unfortunately it seems to be related to > the size of the grid (48000 x 72000) and therefore the problem can't be > reproduced on machines with 8GB RAM. For some processing steps I need > grids of that size in the memory, which is why I have 256 GB installed. > > Normally, I use the raster package and not rgdal::readGDAL(). This was > just a desperate attempt to find the source of the problem. > > This is what I use in my code: > > rDem = raster(sFileNameTiff) > mDem = raster::as.matrix(rDem) > > But maybe this is the same... > > Any further suggestions are much appreciated! > > Thanks again! > > Best, > > Thorsten > > > > > Am 27.04.2020 um 14:50 schrieb Roger Bivand: > > On Mon, 27 Apr 2020, Thorsten Behrens wrote: > > > >> Dear all, > >> > >> my problem is that I want to read a big geotiff raster dataset into R > >> and convert it to a matrix, which does not work. > >> The file is big but there is sufficient memory. I need all the data > >> in the memory at the same time. > >> > >> The error occurs under R 3.6.3 as well as 4.0.0 using Ubuntu 20.04 > >> LTS with the latest version of the packages (see session info below) > >> and 256GB RAM installed. > >> > >> When loading the raster dataset using rgdal (via readGDAL or > >> raster::readAll) I get the follwoing error in R 4.0.0: > >> > >> ``` > >> Error in rgdal::getRasterData(con, offset = offs, region.dim = reg, > >> band = object@data@band) : > >> long vectors not supported yet: memory.c:3782 > >> ``` > > > > On a 16GB Fedora platform: > > > >> library(raster) # 3.1-5 > >> rDemTest = raster(nrow = 48000, ncol = 72000, ext = extent(c(0, 72000, > > 0, > > + 48000))) # all fine > >> rDemTest > > class : RasterLayer > > dimensions : 48000, 72000, 3.456e+09 (nrow, ncol, ncell) > > resolution : 1, 1 (x, y) > > extent : 0, 72000, 0, 48000 (xmin, xmax, ymin, ymax) > > crs: NA > > > >> values(rDemTest) = 1 > > Error: cannot allocate vector of size 25.7 Gb > > > > So you are deceiving yourself into thinking that all is fine at this > > point. Please try to instantiate an example that can be reproduced on > > a machine with 8GB RAM. > > > > Further note that rgdal::readGDAL() is not how you handle very large > > objects in files, and never has been. raster can handle blocks of data > > from bands in file; stars and gdalcubes can use proxy=TRUE for the > > same purpose. Why did you choose rgdal::readGDAL() when this is not > > its purpose? > > > > You did not say how much RAM is on your platform. > > > > Roger > > > >> > >> In R 3.6.3 is is "... memory.c:3717" > >> > >> However, I can load the same file with the tiff package and a file of > >> the same size in the native raster package format (*.grd) with the > >> raster package but again not with the rgdal package. > >> > >> gdalinfo (gdalUtils) does not complain (see below). Hence, Even > >> Rouault assumes the problem is related to rgdal and not gdal > >> (https://github.com/OSGeo/gdal/issues/2442). > >> > >> Below you find reproducible code, which generates a raster file, > >> saves the two formats (.tiff and .grd) and tries to read them with > >> the different packages. > >> > >> Is this a known limitation? Any help is greatly appreciated! > >> > >> Thanks a lot in advance! > >> > >> Best wishes and stay healthy, > >> Thorsten > >> > >> > >> > >> ### Steps to reproduce the problem. > >> > >> R code: > >> > >> ``` > >> library(rgdal) # 1.4-8 > >> library(raster) # 3.1-5 > >> library(tiff) # 0.1-5 > >> > >> ## generate and manipulate a big raster dataset > >> # - generate > >> rDemTest = raster(nrow = 48000, ncol = 72000, ext = extent(c(0, > >> 72000, 0, 48000))) # all fine > >> > >> # - manipulate > >> values(rDemTest) = 1 # all fine > >> > >> # - convert > >> mDemTest = raster::as.matrix(rDemTest) # all fine > >> str(mDemTest) > >> > >> ## save a big dataset > >> > >> # - as raster/gdal > >> sFileNameTiff = "BigData.tif" > >> writeRaster(rDemTest, sFileNameTiff, "GTiff", overwrite = TRUE, > >> NAflag = -) # all fine > >> > >> # - as raster native > >> sFileNameNative = "BigData.grd" > >> writeRaster(rDemTest, sFileNameNative, "raster", overwrite = TRUE, > >> NAflag = -) # all fine > >> > >> > >> ## load the big raster datasets with different packages and options > >> # - load the tiff data with the gdal package via the raster package > >> rDem = raster(sFileNameTiff) # all fine > >> extent(rDem) # all fine > >> mDem = raster::as.matrix(rDem) # error > >> rDem =
Re: [R-sig-Geo] Extract VIIRS data in R
You won't find much on this kind of file in the R community, this is
relatively new NetCDF-4 with "groups", and contains a satellite "granule" -
a kind of pre-mapping raw data array with only technical information about
where the scanning occurred.
In short, raster and stars package (and RNetCDF and ncdf4) will read the
data from this file, but give very little or no help for how to treat it
like a map. The spatial resolution and extent are purely nominal as shown
by raster, a spacing of 1 in two dimensions and extending from 0 to nrows,
0 to ncols (with a half cell shift). There aren't any coordinate arrays in
the file, or any simple way to transform from this matrix-space to
geography as far as I know. (Would love to be corrected on how to do
that).
You'd need to pursue domain-specific expertise to discover the mapping of
this, unless some kind soul turns up to help here. I expect you'll want to
find a product that has been converted into simpler form for these data.
Cheers, Mike.
On Wed, Mar 18, 2020 at 1:39 AM Miluji Sb wrote:
> Dear all,
>
> Hope everyone is keeping safe.
>
> I am trying to extract/read VIIRS nighttime lights data but the output
> seems rather strange.
>
> I have uploaded the file here
> <https://drive.google.com/open?id=1zpqXJ8AlEcnk6ApRb75tfQc4-Y8AfK5h> so as
> not exceed the size limit of the email.
>
> ## Code ##
> library(ncdf4)
> library(rgdal)
>
> netcdf_file <- c("VNP02DNB_NRT.A2020069.1048.001.nc")
> nl <- brick(netcdf_file, lvar=0, values=TRUE,
> varname="observation_data/DNB_observations")
>
> ## Detail ##
> class : RasterLayer
> dimensions : 3232, 4064, 13134848 (nrow, ncol, ncell)
> resolution : 1, 1 (x, y)
> extent : 0.5, 4064.5, 0.5, 3232.5 (xmin, xmax, ymin, ymax)
> crs: NA
> source : C:/Users/shour/Desktop/VNP02DNB_NRT.A2020069.1048.001.nc
> names : DNB.observations.at.pixel.locations
> zvar : observation_data/DNB_observations
>
> The spatial resolution is supposed to be 750m but this shows 1°. What am I
> doing wrong? Any help will be greatly appreciated. Thank you.
>
> Sincerely,
>
> Millu
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] reading an ESRI grid into R
What you've done is perfectly fine. The raster() function invokes the GDAL
library to do the read, via the rgdal package. It doesn't read any data at
first, so dem is just a metadata shell that raster works with - if you plot
or crop or extract that triggers the actual data read as needed.
You might also try the stars package, to keep it lazy and not read all the
data at first start with the proxy argument:
dem_stars <-
stars::read_stars("./Elevation_GRID/NA_Elevation/data/NA_Elevation/na_elevation/w001001.adf",
proxy = TRUE)
This also uses GDAL but does it with code internal to the sf package. These
are pretty much analogous, but stars can do more when grids are more
general.
The GDAL details for this format are here
https://gdal.org/drivers/raster/arcinfo_grid_format.html
Cheers, Mike
On Tue, Feb 11, 2020 at 5:44 AM Julie Lee-Yaw via R-sig-Geo <
r-sig-geo@r-project.org> wrote:
> Hi I'm using the raster package to load a DEM from here:
> http://www.cec.org/tools-and-resources/map-files/elevation-2007
>
> The data appear to be stored as an ESRI grid (adf) and I have no
> experience with this format.
> From what I can tell this format is similar to a shp file with different
> files storing different things pertaining to the grid.
> I'm using the raster command (raster package) to read the w001001.adf file:
>
>
> dem<-raster("./Elevation_GRID/NA_Elevation/data/NA_Elevation/na_elevation/w001001.adf")
>
> The produces what looks like a raster with values. Is that all I need to
> do or should I be reading these data another way?
> Thanks
> Julie
>
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] how to get the value of a pixel and its 8 surrounding pixels from points?
In raster try cellFromXY() and adjacent() which separates out the two tasks, find a cell and find cell neighbours On Wed., 6 Nov. 2019, 17:18 Bede-Fazekas Ákos, wrote: > Dear Cristabel, > function focal() of package raster is what you are searching for. > focal(x, w = matrix(1/9, ncol = 3, nrow = 3), fun = sum) > or > focal(x, w = matrix(1, ncol = 3, nrow = 3), fun = mean) > > HTH, > Ákos Bede-Fazekas > Hungarian Academy of Sciences > > 2019.11.05. 17:10 keltezéssel, "Cristabel Durán Rangel" írta: > > > > I need to get the value of a pixel (I have coordinates for this > > pixels) and its 8 surrounding pixel from the pixel/point. I am working > > with NetCDF files. I am working with R. > > > > This is my code till now: > > > > nc <- nc_open(file, readunlim=FALSE) > > > > mylon <- ncvar_get(nc,"lon") > > mylat <- ncvar_get(nc,"lat") > > > > my coordinates in real-world: lat 52.5935 lon 18.4467 > > > > lat.coor <-mylat[which.min(abs(mylat-52.5935))] > > lon.coor <- mylon[which.min(abs(mylon-18.4467))] > > > > var <- nc[lon.coor, lat.coor, ] > > > > In var are the values for my point. But I also need the values of the > > 8 surrounding pixels to get an average. > > > > Thanks. > > > > > > ___ > > R-sig-Geo mailing list > > R-sig-Geo@r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] dissolve internal borders of polygons using st_union and group_by
Cool, thanks for clarifying On Fri., 18 Oct. 2019, 02:48 Marta Rufino, wrote: > Thank you very much to all :) > > Barry+Roger contributions sorted the problem for the moment, although in > my mind there are still things to be understood. > Mike, yep, thank you for alerting me of rnaturalearth issues- I was not > aware of it. In this case, I am really interested in the process to > apply in other polygons, not in the data *per se*, but will take this > into account in the future. > > Once I have the doc produced (distribution of world > estuaries/bays/transition waters), I will post on the list :) > > Best wishes, > M. > > > [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] dissolve internal borders of polygons using st_union and group_by
rnaturalearth is really dirty data, there's no way forward when things are so bad, only backing up can help. Do you need the whole world? Where do you need, and what scale? Your question suggests that finding the right data is the solution rather than battling with software. Cheers, Mike On Fri., 18 Oct. 2019, 00:27 Roger Bivand, wrote: > On Thu, 17 Oct 2019, Marta Rufino wrote: > > > Dear Roger, > > > > Thank you very much for your quick and dedicated response. > > > > Please state all versions: > >> sessionInfo() > >> sf_extSoftVersion() > > > > See bellow. I though it was a trivial issue, thus sorry for not reporting > > it earlier. > > > > > >> With an updated system, most of your code just does not work for me. > > > > Just updated everything, r included, and it still runs in my machine - > > perhaps because I failed to update to proj6 and I am still using proj4 - > I > > will have to dedicate more time to overcome it. Is there sites that > explain > > win-dummies installation? > > You don't need to update yet, but modern PROJ will upset many workflows. > > > > > > >> You > >> are looking for sf::aggregate(): > >> kk <- aggregate(world_map, list(world_map$continent), head, n=1) > >> plot(st_geometry(kk)) > >> > > > > This will be then a fourth option: > > # Answer from ROGER: > > kk4 <- aggregate(world_map, list(world_map$continent), head, n=1) > > > > object.size(kk4) > > kk4 %>% > > ggplot()+ > > geom_sf() > > > > The sp_transform was simply to check if it was workable the produced file > > (in other cases, the code functioned, but then I could not work with the > > files produced) - so ok to remove it or change it. > > > > You example using rgeos is really nice and I am grateful for it, but I > > really wanted to understand how to do it overall so I can apply in other > > situations I am working with. > > I don't understand the difference between all those options, that > > apparently are giving the same result, but not so much in fact. > > My attempts to use st_precision() to try to find a precision level > that removed the artefacts ended in segmentation faults, so the current > best workaround is as shown to use rgeos, until the correct sf incantation > is forthcoming. > > Roger > > > > > Any further ideas on the subject? > > > > Thank you once again, > > Best wishes, > > M. > > > > > >> sessionInfo() > > R version 3.6.1 (2019-07-05) > > Platform: x86_64-w64-mingw32/x64 (64-bit) > > Running under: Windows >= 8 x64 (build 9200) > > > > Matrix products: default > > > > locale: > > [1] LC_COLLATE=English_United Kingdom.1252 > > [2] LC_CTYPE=English_United Kingdom.1252 > > [3] LC_MONETARY=English_United Kingdom.1252 > > [4] LC_NUMERIC=C > > [5] LC_TIME=English_United Kingdom.1252 > > > > attached base packages: > > [1] stats graphics grDevices utils datasets methods > > [7] base > > > > other attached packages: > > [1] rgeos_0.5-2 ggthemes_4.2.0 ggrepel_0.8.1 > > [4] ggplot2_3.2.1 tidyr_1.0.0 scales_1.0.0 > > [7] dplyr_0.8.3 reshape2_1.4.3 viridis_0.5.1 > > [10] viridisLite_0.3.0 units_0.6-5 rnaturalearth_0.1.0 > > [13] tmap_2.3-1 mapview_2.7.0 plotKML_0.5-9 > > [16] lubridate_1.7.4 zoo_1.8-6 rasterVis_0.46 > > [19] latticeExtra_0.6-28 RColorBrewer_1.1-2 lattice_0.20-38 > > [22] raster_3.0-7maptools_0.9-8 sf_0.8-0 > > [25] gstat_2.0-3 rgdal_1.4-6 sp_1.3-1 > > > > loaded via a namespace (and not attached): > > [1] colorspace_1.4-1 class_7.3-15 colorRamps_2.3 > > [4] leaflet_2.0.2 htmlTable_1.13.2 satellite_1.0.1 > > [7] base64enc_0.1-3dichromat_2.0-0rstudioapi_0.10 > > [10] hexbin_1.27.3 fansi_0.4.0codetools_0.2-16 > > [13] splines_3.6.1 knitr_1.25 zeallot_0.1.0 > > [16] Formula_1.2-3 tmaptools_2.0-2cluster_2.1.0 > > [19] png_0.1-7 shiny_1.4.0compiler_3.6.1 > > [22] backports_1.1.5assertthat_0.2.1 Matrix_1.2-17 > > [25] fastmap_1.0.1 lazyeval_0.2.2 cli_1.1.0 > > [28] later_1.0.0acepack_1.4.1 htmltools_0.4.0 > > [31] tools_3.6.1gtable_0.3.0 glue_1.3.1 > > [34] Rcpp_1.0.2 vctrs_0.2.0leafsync_0.1.0 > > [37] crosstalk_1.0.0lwgeom_0.1-7 xfun_0.10 > > [40] stringr_1.4.0 mime_0.7 lifecycle_0.1.0 > > [43] XML_3.98-1.20 MASS_7.3-51.4 promises_1.1.0 > > [46] parallel_3.6.1 yaml_2.2.0 gridExtra_2.3 > > [49] aqp_1.17 rpart_4.1-15 reshape_0.8.8 > > [52] stringi_1.4.3 plotrix_3.7-6 e1071_1.7-2 > > [55] checkmate_1.9.4intervals_0.15.1 rlang_0.4.0 > > [58] pkgconfig_2.0.3pixmap_0.4-11 RSAGA_1.3.0 > > [61] purrr_0.3.2htmlwidgets_1.5.1 tidyselect_0.2.5 > > [64] plyr_1.8.4 magrittr_1.5 R6_2.4.0 > > [67] Hmisc_4.2-0DBI_1.0.0 pillar_1.4.2 > > [70] foreign_0.8-72 withr_2.1.2shapefiles_0.7 > > [73] xts_0.11-2
Re: [R-sig-Geo] Raster reprojection in R
Hi again, I checked the actual extent and it makes more sense now as it is a tiny region of the Aleutians. The problem in raster's heuristic is that part of your projected raster overlaps the anti-meridan, and so the extent-determiner unhelpfully expands to include the full extent of all longitudes: library(raster) ex <- extent(-9505, -10007555, 5559753, 6671703) prj <- "+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 + +b=6371007.181 +units=m +no_defs" projectExtent(raster(ex, crs = prj, res = 463), crs = "+init=epsg:4326") class : RasterLayer dimensions : 2402, 2402, 5769604 (nrow, ncol, ncell) resolution : 0.1498734, 0.004163854 (x, y) extent : -179.9968, 179.999, 49.99842, 60 (xmin, xmax, ymin, ymax) crs: +init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 Illustrated here: https://gist.github.com/mdsumner/012c4c8e36ffa42e9053621420924eab So, you'd want something like projectRaster(tile, raster(extent(-180, -135, 50, 60), crs = geocrs, res = 0.02) but, that's still outrageously wasteful and unnecessary as mentioned before - but depends on what you are doing. (It's a curious way to store data, but I'm pretty sure the MODIS terra community settled on a global raster in sinusoidal to match the L3 bins in a way the marine community doesn't usually do). Thanks for the example, that gives a good way to approach this question on various forums where it comes up quite a lot and I never was really motivated to pursue before. Cheers, Mike. On Sat, Sep 14, 2019 at 7:49 PM Michael Sumner wrote: > You should set a target raster with the extent and dimensions required. > There are inherent limits in reprojection and heuristics won't always work. > Generally using a target raster is much more efficient anyway. > > But, this is an intensive remodeling of the data, delivered in a > projection for good reason (global equal area probably, and reasonably > close to the L3 bins used for daily statistics. > > You should find an alternative process IMO, are you trying to extract > pixel values or something else? > > Cheers, Mike > > On Sat., 14 Sep. 2019, 11:30 Víctor Rodríguez Galiano, < > vrgali...@gmail.com> wrote: > >> Hello, >> >> I am trying to reproject a raster image from sinusoidal projection using >> “projectRaster”. The size of the image is not very big (922 KB), but when >> applying the reprojection I get this error message: “Error: cannot >> allocate >> vector of size 7.1 Gb”. Please see the code below: >> >> HDFpath <- "C:/images/" # dir with images >> setwd(HDFpath)# set working directory >> library(raster) >> library(rgdal) >> geocrs <- "+proj=longlat +ellps=WGS84 +datum=WGS84" >> tile <- brick(“image.tif”) >> tile_reproj <- projectRaster(tile, crs=geocrs) >> >> Error: cannot allocate vector of size 7.1 Gb >> >> > tile >> class : RasterStack >> dimensions : 2400, 2400, 576, 1 (nrow, ncol, ncell, nlayers) >> resolution : 463.3127, 463.3127 (x, y) >> extent : -9505, -10007555, 5559753, 6671703 (xmin, xmax, ymin, >> ymax) >> coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 >> +b=6371007.181 +units=m +no_defs >> names : INCA.h08v03.Dormancy_median >> min values : 19 >> max values : 540 >> >> ___ >> R-sig-Geo mailing list >> R-sig-Geo@r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo >> > -- Michael Sumner Software and Database Engineer Australian Antarctic Division Hobart, Australia e-mail: mdsum...@gmail.com [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Raster reprojection in R
You should set a target raster with the extent and dimensions required. There are inherent limits in reprojection and heuristics won't always work. Generally using a target raster is much more efficient anyway. But, this is an intensive remodeling of the data, delivered in a projection for good reason (global equal area probably, and reasonably close to the L3 bins used for daily statistics. You should find an alternative process IMO, are you trying to extract pixel values or something else? Cheers, Mike On Sat., 14 Sep. 2019, 11:30 Víctor Rodríguez Galiano, wrote: > Hello, > > I am trying to reproject a raster image from sinusoidal projection using > “projectRaster”. The size of the image is not very big (922 KB), but when > applying the reprojection I get this error message: “Error: cannot allocate > vector of size 7.1 Gb”. Please see the code below: > > HDFpath <- "C:/images/" # dir with images > setwd(HDFpath)# set working directory > library(raster) > library(rgdal) > geocrs <- "+proj=longlat +ellps=WGS84 +datum=WGS84" > tile <- brick(“image.tif”) > tile_reproj <- projectRaster(tile, crs=geocrs) > > Error: cannot allocate vector of size 7.1 Gb > > > tile > class : RasterStack > dimensions : 2400, 2400, 576, 1 (nrow, ncol, ncell, nlayers) > resolution : 463.3127, 463.3127 (x, y) > extent : -9505, -10007555, 5559753, 6671703 (xmin, xmax, ymin, > ymax) > coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 > +b=6371007.181 +units=m +no_defs > names : INCA.h08v03.Dormancy_median > min values : 19 > max values : 540 > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Error in netCDF file: cells are not equally spaced
raster does the wrong thing here in my opinion, the right outcome is to
give a grid in index-extent (0, ncol, 0, nrow) and with no projection
metadata (crs). There will be coordinate arrays in this file, and they need
to be handled as though they are data (with local, x*y dependent values in
every cell).
With brick() you can proceed to process the data with no problem (using
stopIfNotEqualSpaced = FALSE), but you can't rely on the extent being
relevant, or any function that works in geographic space to do the right
thing. To map a layer of these data you might try
grd <- raster(yourfile, stopIfNotEqualSpaced = FALSE)
coords <- brick(raster(yourfile, varname = "lon"),
raster(yourfile, varname = "lat")) ## but only
you will know the names of these variables values "lon", "lat" (might be
"TLON", "TLAT" ? )
In quadmesh there is a way to plot in geographic space that's not
impossibly slow (possibly the coords will need a re-orientation transpose
...):
quadmesh::mesh_plot(grd, coords = coords)
It's likely that will "map" it properly, but CMIP is likely to wrap around
an odd longitude (or something), and so cropping to your area and/or
choosing a good project for the crs argument to mesh_plot is probably
needed. You can investigate with plot(coords[[1]]) and plot(coords[[2]])
to get a sense of their layout.
In the stars package this is all internalized, with
stars::read_stars(yourfile) catching all the information required as much
as possible, and with plotting methods - but variable choice and actual
results vary widely, and depend a lot on very specific details. (angstroms
package has similar helpers for the approach but is unlikely to help here
without access to the file to try)
To help us guess further you can use the "ncdump" output, raster will print
this with
print(raster("yourfile"))
and from that we could provide better guesses at variable names and
subsetting etc.
But, as Edzer asked, nothing is as good as having the file - any chance you
can share it? (Personally I think the world rather needs more R attention
on climate model output.)
Cheers, Mike.
On Thu, Aug 22, 2019 at 1:53 PM Frederico Faleiro
wrote:
>
> Dear list,
>
> I am using some netCDF files from the CMIP5 climate models in raster
> package, but I am having some issues with one model.
> The netCDF file from the GFDL-ESM2G model (e.g.
>
http://aims3.llnl.gov/thredds/fileServer/css03_data/cmip5/output1/NOAA-GFDL/GFDL-ESM2G/rcp45/mon/atmos/Amon/r1i1p1/v20120412/pr/pr_Amon_GFDL-ESM2G_rcp45_r1i1p1_204101-204512.nc
> )
> have the error message as in bellow example.
>
> #Example
> library(raster)
> s1 <- stack("pr_Amon_GFDL-ESM2G_rcp45_r1i1p1_204101-204512.nc")
> Error in .rasterObjectFromCDF(x, type = objecttype, band = band, ...) :
> cells are not equally spaced; you should extract values as points
>
> # I check some solutions that recomend force read the file with the
> argument "stopIfNotEqualSpaced = F" as bellow.
> sf <- stack("pr_Amon_GFDL-ESM2G_rcp45_r1i1p1_204101-204512.nc",
> *stopIfNotEqualSpaced
> = F*)
> bf <- brick("pr_Amon_GFDL-ESM2G_rcp45_r1i1p1_204101-204512.nc",
> *stopIfNotEqualSpaced
> = F*)
>
> I performed some tests and only "works" with brick. However I did not find
> any solution to check where is the problem and fix it in the file.
>
> How can I check if it is really an issue and fix it?
>
> sessionInfo()
> R version 3.5.1 (2018-07-02)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows 10 x64 (build 18362)
>
> Matrix products: default
>
> locale:
> [1] LC_COLLATE=Portuguese_Brazil.1252 LC_CTYPE=Portuguese_Brazil.1252
> [3] LC_MONETARY=Portuguese_Brazil.1252 LC_NUMERIC=C
> [5] LC_TIME=Portuguese_Brazil.1252
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> other attached packages:
> [1] raster_2.8-19 sp_1.3-1
>
> loaded via a namespace (and not attached):
> [1] compiler_3.5.1 rgdal_1.4-3 Rcpp_1.0.1 codetools_0.2-15
> ncdf4_1.16.1
> [6] grid_3.5.1 lattice_0.20-35
>
> Best regards,
>
> Frederico
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] st_simplify outputs the input geometry with no simplification
It has changed it, there's now a mix of POLYGON and MULTIPOLYGON
geometries, but no change in the underlying coordinates. You have to use
the dTolerance argument:
nrow(st_coordinates(nc))
[1] 2529
nrow(st_coordinates(st_cast(nc_simplfy)))
[1] 2529
nrow(st_coordinates(st_cast(st_simplify(nc, dTolerance = 1000
[1] 1941
It still might change the geometry with dTolerance = 0 (the default), in
that case it will remove vertices that are collinear (unnecessarily dense
straight lines, reduce to two-vertex edges). I only learnt that recently.
HTH
On Mon, Jul 22, 2019 at 8:15 PM Hugo Costa wrote:
> Dear list,
>
> function st_simplify outputs exactly the same geometry as the input. This
> is an example:
>
> library(sf)
>
> nc = st_read(system.file("shape/nc.shp", package="sf"))
> nc <- nc %>% st_transform(3857)
> plot(st_geometry(nc))
>
> nc_simplfy<-st_simplify(nc)
> plot(st_geometry(nc_simplfy))
>
> However, I'm quite sure some days ago st_simpliffy was working as expected.
> Could it be something wrong in my machine? What am I missing?
>
> > sessionInfo()
> R version 3.5.3 (2019-03-11)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows >= 8 x64 (build 9200)
>
> Matrix products: default
>
> locale:
> [1] LC_COLLATE=Portuguese_Portugal.1252 LC_CTYPE=Portuguese_Portugal.1252
>LC_MONETARY=Portuguese_Portugal.1252 LC_NUMERIC=C
> [5] LC_TIME=Portuguese_Portugal.1252
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> other attached packages:
> [1] sf_0.7-3
>
> loaded via a namespace (and not attached):
> [1] compiler_3.5.3 magrittr_1.5 class_7.3-15 DBI_1.0.0
> tools_3.5.3units_0.6-2Rcpp_1.0.1 grid_3.5.3 e1071_1.7-0.1
> [10] classInt_0.3-1
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Michael Sumner
Software and Database Engineer
Australian Antarctic Division
Hobart, Australia
e-mail: mdsum...@gmail.com
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Calculate the internal angles of polygons
There's code in trip for the angles, but arranging the underlying coords and expressing those in the right form will be the crux. st_coordinates gives what you need, or pure data frame from spbabel::sptable is what I'd start with. I'll take a look when I can. Cheers, Mike On Sat., 1 Jun. 2019, 15:41 Juan Pablo Carranza, wrote: > Dear members, > I'm having trouble in calculating the internal angles of a set of polygons. > I have a sf "MULTIPOLYGON" object of 5000 land parcels that has been tagged > according to their shape. The objective is to train an algorithm to predict > a large dataset (over 2 million polygons) according to these > classification. But that is a problem that comes forward. > First I need to calculate the internal angles of each one of the polygons > in the sample, in order to eliminate unnecesary nodes (yes, I tryed > st_simplify and also ms_simplify, but I need to control the process by the > angle values). > Here is as far as I get, a few lines of code (with an example polygon). The > next steps will be dealing with a way to generate a dataframe in wich each > row is a line of each polygon (decomposing the geometry but preserving the > polygon entityid in order to go back... easy to do with spdplyr?), and find > a way to calculate the angle between adjscent lines. Need a hand, please! > > x = > > c(4293215,4293205,4293205,4293194,4293174,4293176,4293190,4293201,4293204,4293218,4293215) > y = > > c(6463731,6463733,6463734,6463736,6463740,6463750,6463747,6463744,6463744,6463741,6463731) > cords = cbind(x, y) > library(sp) > p = Polygon(cords) > ps = Polygons(list(p),1) > poly = SpatialPolygons(list(ps)) > plot(poly) > library(sf) > poly = st_as_sf(poly) > lines = st_cast(poly$geometry, "LINESTRING") > plot(lines) > # not run: this turns the object too simple, I need to work on very complex > shapes. > # lines = st_simplify(lines, preserveTopology = FALSE, dTolerance = 1) > > Really sorry for the lousy english... Best regards. > > *MSc. Juan Pablo Carranza* > Economist > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Plotting x and y values using data from two separate netCDF files in R
Use the file name as the first argument, and the variable name you want as
varname =
Raster doesn't work with output of nc_open
See ?brick
Good luck
On Wed, Mar 27, 2019, 08:05 wrote:
> Hi Michael,
>
> Thank you so much for your reply!
>
> I was just trying your suggestion, but when I run the following in R:
>
>
> x<-raster::brick(ncfname, varname="cum_co2_emi-CanESM2")
>
> I receive the following error:
>
> Error in .varName(nc, varname, warn = warn) :
> varname: cum_co2_emi-CanESM2 does not exist in the file. Select one from:
>
>
>
> I tried switching "ncfname" with "Model1", but I then receive this error:
>
> Error in (function (classes, fdef, mtable) :
> unable to find an inherited method for function ‘brick’ for signature
> ‘"ncdf4"’
>
>
> Is there a reason for that?
>
> Thanks, again,
>
> -Original Message-
> From: Michael Sumner
> To: rain1290
> Cc: r-sig-geo
> Sent: Tue, Mar 26, 2019 4:34 pm
> Subject: Re: [R-sig-Geo] Plotting x and y values using data from two
> separate netCDF files in R
>
> I would try for a single point:
>
> x <- raster::brick(ncfname, varname = "cum_co2_emi-CanESM2")
> y <- raster::brick(ncfname1, varname = "onedaymax")
>
> pt <- cbind(30, -5)
> to_plot <- cbind(raster::extract(x, pt), raster::extract(y, pt))
>
> plot(to_plot)
>
> Is that close? You might be better off using raster::as.data.frame(x, xy
> = TRUE, long = TRUE) if you want all locations at their actual centre.
>
> See if the times of the 3rd axis are valid (and the same) in getZ(x) and
> getZ(y).
>
> There's rarely a need to use ncdf4 directly, though that's important
> sometimes, more so for grids that raster's regular-affine referencing model
> doesn't support.
>
> cheers, Mike
>
>
>
> On Wed, 27 Mar 2019 at 05:29 rain1290--- via R-sig-Geo <
> r-sig-geo@r-project.org> wrote:
>
> Hi there,
> I am currently trying to plot precipitation data (y-axis values) with
> cumulative emissions data (x-axis) using R. Both of these data are found on
> two separate netCDF files that I have already read into R. Ultimately, What
> I would like to do is plot precipitation as a function of cumulative
> emissions for a selected location (as shown below in the following code). I
> have, so far, used the following code (with "#" to highlight each step):
> library(raster)
> library(ncdf4)
> library(maps)
> library(maptools)
> library(rasterVis)
> library(ggplot2)
> library(rgdal)
> library(sp)#Geting cumulative emissions data for x-axis
> ncfname<-"cumulative_emissions_1pctCO2.nc"
> Model1<-nc_open(ncfname)
> print(Model1)
> get<-ncvar_get(Model1, "cum_co2_emi-CanESM2") #units of terratones
> of
> carbon (TtC) for x-axis (140 values)
> print(get)
> Year<-ncvar_get(Model1, "time") #140 years
> #Getting Model data for extreme precipitation (units of millimeters/day)
> for y-axis ncfname1<-"MaxPrecCCCMACanESM21pctCO2.nc"
> Model2<-nc_open(ncfname1)
> print(Model2)
> get1<-ncvar_get(Model2, "onedaymax") #units of millimeters/day
> print(get1)
> #Reading in latitude, longitude and time from this file:
> latitude<-ncvar_get(Model2, "lat") #64 degrees latitude
> longitude<-ncvar_get(Model2, "lon") #128 degrees longitude
> Year1<-ncvar_get(Model2, "Year") #140 years
> #Plotting attemptrandompointlon<-30 #selecting a longitude
> randompointlat<--5 #selecting a latitude
> Hope<-extract(r_brick,
> SpatialPoints(cbind(randompointlon,randompointlat)),method='simple')
> df<-data.frame(cumulativeemissions=seq(from=1, to=140, by=1),
> Precipitation=t(Hope))
> ggplot(data=df, aes(x=get, y=Precipitation,
> group=1))+geom_line()+ggtitle("One-day maximum precipitation
> (mm/day)
> for random location for CanESM2 1pctCO2 as a function of cumulative
> emissions")
> print(Model1) yields the following (I read in variable #2 for now):
> File cumulative_emissions_1pctCO2.nc (NC_FORMAT_NETCDF4):
> 14 variables (excluding dimension variables):
> float cum_co2_emi-BNU-ESM[time] (Contiguous storage)
> long_name: Cumulative carbon emissions for BNU-ESM
> units: Tt C
> float cum_co2_emi-CanESM2[time] (Contiguous storage)
> long_name: Cumulative carbon emissions for CanESM2
> units: Tt C
> float cum_co2_emi-CESM1-BGC[time] (Contiguous storage)
>
Re: [R-sig-Geo] Convert map pdf in raster
Try r <- raster::brick(pdffile) #(requires rgdal with PDF driver) raster::plotRGB(r) Alternatively, try stars::read_stars HTH On Wed, Mar 27, 2019, 02:43 Lara Silva wrote: > Hello, > > Is it possible to convert a map im pdf to a raster? > Another question. I need to obtain a raster of land use and land cover of > Europe. Which site to choose? > > Thanks, > > Lara > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Plotting x and y values using data from two separate netCDF files in R
at cum_co2_emi-MPI-ESM-MR[time] (Contiguous storage) > long_name: Cumulative carbon emissions for MPI-ESM-MR > units: Tt C > float cum_co2_emi-NorESM1-ME[time] (Contiguous storage) > long_name: Cumulative carbon emissions for NorESM1-ME > units: Tt C > float cum_co2_emi-GFDL-ESM2G[time] (Contiguous storage) > long_name: Cumulative carbon emissions for GFDL-ESM2G > units: Tt C > float cum_co2_emi-GFDL-ESM2M[time] (Contiguous storage) > long_name: Cumulative carbon emissions for GFDL-ESM2M > units: Tt C 1 dimensions: > time Size:140 > units: years since 0-1-1 0:0:0 > long_name: time > standard_name: time > calender: noleap 4 global attributes: > description: Cumulative carbon emissions for the 1pctCO2 scenario > from the CMIP5 dataset. > history: Created Fri Jul 21 14:50:39 2017 > source: CMIP5 archieve > > print(Model2) yields the following:File MaxPrecCCCMACanESM21pctCO2.nc > (NC_FORMAT_NETCDF4): 3 variables (excluding dimension variables): > double onedaymax[lon,lat,time] (Contiguous storage) > units: mm/day > double fivedaymax[lon,lat,time] (Contiguous storage) > units: mm/day > short Year[time] (Contiguous storage) 3 dimensions: > time Size:140 > lat Size:64 > units: degree North > lon Size:128 > units: degree East 3 global attributes: > description: Annual global maximum precipitation from the CanESM2 > 1pctCO2 scenario > history: Created Mon Jun 4 11:24:02 2018 > contact: rain1...@aim.com > So, in general, this is what I am trying to achieve, but I am not sure if > what I am doing in the ggplot function is the right approach for this. > Any assistance with this would be greatly appreciated! > Thanks, > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Plot coordinates on world map with Robinson CRS - ggplot2
You are very close, just needed the expert-knowledge for the incantation
required to transform the longlat points (as was already done for the
prj.coord, btw).
HTH: https://gist.github.com/mdsumner/afd9228aa1dc4652f8bf193cad4245aa
It does come as a surprise to many that R's plotting is not unit or
projection aware (well, it is sometimes, but not usually ... a modern
example is ggplot2::geom_sf and the sf package but you need quite a lot of
prior-won expertise there too, good luck).
Cheers, Mike.
On Mon, 18 Feb 2019 at 20:01 Miluji Sb wrote:
> Dear all,
>
> I am trying to plot coordinates on a world map with Robinson CRS. While the
> world map is generated without any issues, when I try to plot the points -
> I only get a single point.
>
> The code I am using and the coordinates data is below. What am I doing
> wrong? Any help/suggestions will be highly appreciated.
>
> library(data.table)
> library(foreign)
> library(readstata13)
> library(rgdal)
> library(maptools)
> library(ggplot2)
> library(dplyr)
>
> load(url("
>
> https://github.com/valentinitnelav/RandomScripts/blob/master/NaturalEarth.RData?raw=true
> "))
>
> PROJ <- "+proj=robin +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84
> +units=m +no_defs"
>
> NE_countries_rob <- spTransform(NE_countries, CRSobj = PROJ)
> NE_graticules_rob <- spTransform(NE_graticules, CRSobj = PROJ)
> NE_box_rob<- spTransform(NE_box, CRSobj = PROJ)
>
> # project long-lat coordinates for graticule label data frames
> # (two extra columns with projected XY are created)
> prj.coord <- project(cbind(lbl.Y$lon, lbl.Y$lat), proj=PROJ)
> lbl.Y.prj <- cbind(prj.coord, lbl.Y)
> names(lbl.Y.prj)[1:2] <- c("X.prj","Y.prj")
>
> prj.coord <- project(cbind(lbl.X$lon, lbl.X$lat), proj=PROJ)
> lbl.X.prj <- cbind(prj.coord, lbl.X)
> names(lbl.X.prj)[1:2] <- c("X.prj","Y.prj")
>
> m <- ggplot() +
> # add Natural Earth countries projected to Robinson, give black border
> and fill with gray
> geom_polygon(data=NE_countries_rob, aes(long,lat, group=group),
> colour="black", fill="gray80", size = 0.25) +
> # Note: "Regions defined for each Polygons" warning has to do with
> fortify transformation. Might get deprecated in future!
> # alternatively, use use map_data(NE_countries) to transform to data
> frame and then use project() to change to desired projection.
> # add Natural Earth box projected to Robinson
> geom_polygon(data=NE_box_rob, aes(x=long, y=lat), colour="black",
> fill="transparent", size = 0.25) +
> # add graticules projected to Robinson
> geom_path(data=NE_graticules_rob, aes(long, lat, group=group),
> linetype="dotted", color="grey50", size = 0.25) +
> # add graticule labels - latitude and longitude
> geom_text(data = lbl.Y.prj, aes(x = X.prj, y = Y.prj, label = lbl),
> color="grey50", size=2) +
> geom_text(data = lbl.X.prj, aes(x = X.prj, y = Y.prj, label = lbl),
> color="grey50", size=2) +
> # the default, ratio = 1 in coord_fixed ensures that one unit on the
> x-axis is the same length as one unit on the y-axis
> coord_fixed(ratio = 1) +
> geom_point(data=df,
> aes(x=lon, y=lat), colour="Deep Pink",
> fill="Pink",pch=21, size=2, alpha=I(0.7))
> # remove the background and default gridlines
> theme_void()
>
> ## coordinates dataframe
> dput(df)
> structure(list(lon = c(2.67569724621467, 17.5766416259819,
> 28.4126232192772,
> 23.8147674538232, 29.8917589327105), lat = c(28.1503115976162,
> -12.3388787380201, 9.78891068739477, -22.1873831176644, -3.36546931479253
> )), class = "data.frame", row.names = c(NA, -5L))
>
> Sincerely,
>
> Milu
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Reproject MODIS data using R (results in NAs or no spatial extent)
To answer the "Any ideas on why reprojecting this MODIS data is so
difficult?" - it's that the sinusoidal projection is actually pretty
exotic, it matches the daily aggregation of L2 geophysical variables from
individual swaths (multiple per day) into L3 bins (aggregated daily, then
into longer time periods with bin identity maintained). I personally think
it's a bad choice for a regular gridded product, but it's a way of
"regularizing" the L3 bins, which are quite abstract and hard to use. The
L3 bins are a ragged array of bins at regular latitude intervals, a
diminishing number of bins as absolute latitude increases towards each
pole. Described here:
https://oceancolor.gsfc.nasa.gov/docs/format/l3bins/
My answer to the "how to reproject the sinusoidal grid to regular longlat"
is - you should not do that. The sinusoidal grid is faithful to the
statistical properties of the L3 bins, though not to the bin geometries
themselves. One is meant to query the sinusoidal grid for the data needed,
it's really inappropriate to stretch those data out by resampling/
reprojection. (I don't know if the sinusoidal-regular-grid creators have
written down this rationale).
Generally one can reproject vector points, polygons, lines into the
sinusoidal projection and use extractions in that coordinate system, and
that's what I'd recommend. (Using the L3 bins is also possible in R but
it's a long story about how to do that).
Just a question: do you download the auxialiary .hdf.xml files that
accompany the .hdf files? They perhaps contain extra information that
GDAL can interpret.
Using some inside knowledge, delve into subdatasets with stars/sf:
(sds <- sf::gdal_subdatasets("GLASS02B06.V04.A2013169.2017128.hdf"))
x <- stars::read_stars(sds[[1]])
x
stars object with 2 dimensions and 1 attribute
attribute(s), summary of first 1e+05 cells:
HDF4_EOS:EOS_GRID:"GLASS02B06.V04.A2013169.2017128.hdf":GLASS02B06:ABD_BSA_VIS
Min. : NA
1st Qu.: NA
Median : NA
Mean :NaN
3rd Qu.: NA
Max. : NA
NA's :1e+05
dimension(s):
from tooffsetdelta refsys point values
x1 7200 -20015109 154.438 +proj=sinu +lon_0=0 +x_0=...NA NULL [x]
y1 3600 10007555 -308.875 +proj=sinu +lon_0=0 +x_0=...NA NULL [y]
plot(x)
sf::st_bbox(x)
xmin ymin xmax ymax
-20015109 8895604 -18903159 10007555
And OMG! To my disgust we see these data are not in sinusoidal projection
at all, but in Equidistant Cylindrical (the -20015109 is approximately the
distance from the prime meridian to the anti meridian along the equator
(i.e about pi * 6378137m). So, I don't trust these in-situ metadata at
all. I see this stuff gets messed up fairly frequently, perhaps it's a
GDAL problem interpreting the metadata from the file, or perhaps it's
actually mis-applied in the hdf - finding out requires careful examination
of the metadata, as does choice of datum (as follows).
Using stars to read and raster to "fix it" (raster just because I'm more
sure of my steps that way):
r <- setExtent(stars:::st_as_raster(x), extent(-180, 180, -90, 90))
projection(r) <- "+proj=longlat +datum=WGS84"
plot(r)
maps::map(add = T)
plot(extent(r), add = TRUE)
## convince yourself the extent is correct with things like:
abline(h = c(-90, 90), col = "red")
That use of "datum=WGS84" may well be incorrect, but compared to trying to
project data from a completely mis applied projection it's an improvement.
HTH
Cheers, Mike.
On Fri, 30 Nov 2018 at 06:39 Michael Sumner wrote:
> Ah, never mind - it's the subdataset discovery that's probably not easy
> with rgdal.
> Sorry for the noise.
>
> Mike.
>
> On Fri, 30 Nov 2018 at 06:38 Michael Sumner wrote:
>
>> Fwiw there shouldn't be any need to convert from hdf to tif - could you
>> please try this?
>>
>> x <- readGDAL( ".../GLASS02B05.V04.A1990161.2018062.hdf")
>>
>> If that works it at least removes some steps from your process.
>>
>> Cheers, Mike.
>>
>> On Fri, Nov 30, 2018, 05:03 Elizabeth Webb wrote:
>>
>>> I am using GLASS albedo data stored here<
>>> ftp://ftp.glcf.umd.edu/glcf/GLASS/ABD/AVHRR/> for pre-2000 (AVHRR) data
>>> and here<ftp://ftp.glcf.umd.edu/glcf/GLASS/ABD/MODIS/0.05D/> for
>>> post-2000 data (MODIS). My end goal is to create a raster stack of each
>>> month that contains white sky albedo data from 1982-2015. The problem I
>>> have run into is that the MODIS and AVHRR data are in different spatial
>>> reference systems and I can't seem to reproject them to be in the same
>>> system.
>>>
>>> I convert from hdf to tif using R like this:
>>>
>>> fileavhrr <- ".../GLASS02B05.V04.A1990161.2018062.hd
Re: [R-sig-Geo] Reproject MODIS data using R (results in NAs or no spatial extent)
Ah, never mind - it's the subdataset discovery that's probably not easy
with rgdal.
Sorry for the noise.
Mike.
On Fri, 30 Nov 2018 at 06:38 Michael Sumner wrote:
> Fwiw there shouldn't be any need to convert from hdf to tif - could you
> please try this?
>
> x <- readGDAL( ".../GLASS02B05.V04.A1990161.2018062.hdf")
>
> If that works it at least removes some steps from your process.
>
> Cheers, Mike.
>
> On Fri, Nov 30, 2018, 05:03 Elizabeth Webb wrote:
>
>> I am using GLASS albedo data stored here<
>> ftp://ftp.glcf.umd.edu/glcf/GLASS/ABD/AVHRR/> for pre-2000 (AVHRR) data
>> and here<ftp://ftp.glcf.umd.edu/glcf/GLASS/ABD/MODIS/0.05D/> for
>> post-2000 data (MODIS). My end goal is to create a raster stack of each
>> month that contains white sky albedo data from 1982-2015. The problem I
>> have run into is that the MODIS and AVHRR data are in different spatial
>> reference systems and I can't seem to reproject them to be in the same
>> system.
>>
>> I convert from hdf to tif using R like this:
>>
>> fileavhrr <- ".../GLASS02B05.V04.A1990161.2018062.hdf"
>> filemodis<-".../GLASS02B06.V04.A2013169.2017128.hdf"
>> gdal_translate(get_subdatasets(filemodis)[10], dst_dataset =
>> ".../modis.tif")
>> gdal_translate(get_subdatasets(fileavhrr)[8], projwin =
>> c(-180,90,180,50), dst_dataset = ".../avhrr.tif") #ideally I'd only like
>> data north of 50 degrees
>>
>> avhrr<- raster(".../avhrr.tif")
>>
>> #class : RasterLayer
>> #dimensions : 800, 7200, 576 (nrow, ncol, ncell)
>> #resolution : 0.05, 0.05 (x, y)
>> #extent : -180, 180, 50, 90 (xmin, xmax, ymin, ymax)
>> #coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
>> #values : -32768, 32767 (min, max)
>>
>> modis<- raster(".../modis.tif")
>>
>> #class : RasterLayer
>> #dimensions : 3600, 7200, 2592 (nrow, ncol, ncell)
>> #resolution : 154.4376, 308.8751 (x, y)
>> #extent : -20015109, -18903159, 8895604, 10007555 (xmin, xmax, ymin,
>> ymax)
>> #coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181
>> +b=6371007.181 +units=m +no_defs
>> #values : -32768, 32767 (min, max)
>>
>> Here are things I have tried:
>>
>> 1.) Use the MODIS Reprojection Tool<
>> https://lpdaac.usgs.gov/tools/modis_reprojection_tool>. For whatever
>> reason, this tool seems to think the subdatasets of the MODIS .hdf files
>> are only one tile (the upper left most tile, tile 0,0) and not the global
>> dataset. My understanding is that the MODIS data are global (not in
>> tiles?), so I do not know why the MRT is doing this.
>>
>>
>> 2.) Use the raster package in R.
>>
>> projectedMODIS <- projectRaster(modis,avhrr,method="bilinear")
>>
>> This returns a raster with values that are all NA:
>>
>> class : RasterLayer
>> dimensions : 800, 7200, 576 (nrow,> ncol, ncell)
>> resolution : 0.05, 0.05 (x, y)
>> extent : -180, 180,> 50, 90 (xmin, xmax, ymin, ymax)
>> coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
>> values : NA, NA (min, max)
>>
>> 3.) Use the gdalUtils package in R:
>>
>> gdalwarp(srcfile=get_subdatasets(filemodis)[10], dstfile=
>> ".../gdalMODIS_avhrr.tif", s_srs = crs(modis), t_srs =crs(avhrr) )
>>
>> This returns a raster with essentially no spatial extent.
>>
>> gdalMODISavhrr<-raster(".../gdalMODIS_avhrr.tif")
>> #class : RasterLayer
>> #dimensions : 357, 12850, 4587450 (nrow, ncol, ncell)
>> #resolution : 0.02801551, 0.02801573 (x, y)
>> #extent : -180, 179.9993, 79.99838, 90 (xmin, xmax, ymin, ymax)
>> #coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
>> #values : -32768, 32767 (min, max)
>>
>> Any ideas on why reprojecting this MODIS data is so difficult?
>>
>>
>> [[alternative HTML version deleted]]
>>
>> ___
>> R-sig-Geo mailing list
>> R-sig-Geo@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
> --
> Dr. Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> 203 Channel Highway
> Kingston Tasmania 7050 Australia
>
> --
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Reproject MODIS data using R (results in NAs or no spatial extent)
Fwiw there shouldn't be any need to convert from hdf to tif - could you
please try this?
x <- readGDAL( ".../GLASS02B05.V04.A1990161.2018062.hdf")
If that works it at least removes some steps from your process.
Cheers, Mike.
On Fri, Nov 30, 2018, 05:03 Elizabeth Webb wrote:
> I am using GLASS albedo data stored here<
> ftp://ftp.glcf.umd.edu/glcf/GLASS/ABD/AVHRR/> for pre-2000 (AVHRR) data
> and here<ftp://ftp.glcf.umd.edu/glcf/GLASS/ABD/MODIS/0.05D/> for
> post-2000 data (MODIS). My end goal is to create a raster stack of each
> month that contains white sky albedo data from 1982-2015. The problem I
> have run into is that the MODIS and AVHRR data are in different spatial
> reference systems and I can't seem to reproject them to be in the same
> system.
>
> I convert from hdf to tif using R like this:
>
> fileavhrr <- ".../GLASS02B05.V04.A1990161.2018062.hdf"
> filemodis<-".../GLASS02B06.V04.A2013169.2017128.hdf"
> gdal_translate(get_subdatasets(filemodis)[10], dst_dataset =
> ".../modis.tif")
> gdal_translate(get_subdatasets(fileavhrr)[8], projwin = c(-180,90,180,50),
> dst_dataset = ".../avhrr.tif") #ideally I'd only like data north of 50
> degrees
>
> avhrr<- raster(".../avhrr.tif")
>
> #class : RasterLayer
> #dimensions : 800, 7200, 576 (nrow, ncol, ncell)
> #resolution : 0.05, 0.05 (x, y)
> #extent : -180, 180, 50, 90 (xmin, xmax, ymin, ymax)
> #coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
> #values : -32768, 32767 (min, max)
>
> modis<- raster(".../modis.tif")
>
> #class : RasterLayer
> #dimensions : 3600, 7200, 2592 (nrow, ncol, ncell)
> #resolution : 154.4376, 308.8751 (x, y)
> #extent : -20015109, -18903159, 8895604, 10007555 (xmin, xmax, ymin,
> ymax)
> #coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181
> +b=6371007.181 +units=m +no_defs
> #values : -32768, 32767 (min, max)
>
> Here are things I have tried:
>
> 1.) Use the MODIS Reprojection Tool<
> https://lpdaac.usgs.gov/tools/modis_reprojection_tool>. For whatever
> reason, this tool seems to think the subdatasets of the MODIS .hdf files
> are only one tile (the upper left most tile, tile 0,0) and not the global
> dataset. My understanding is that the MODIS data are global (not in
> tiles?), so I do not know why the MRT is doing this.
>
>
> 2.) Use the raster package in R.
>
> projectedMODIS <- projectRaster(modis,avhrr,method="bilinear")
>
> This returns a raster with values that are all NA:
>
> class : RasterLayer
> dimensions : 800, 7200, 576 (nrow,> ncol, ncell)
> resolution : 0.05, 0.05 (x, y)
> extent : -180, 180,> 50, 90 (xmin, xmax, ymin, ymax)
> coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
> values : NA, NA (min, max)
>
> 3.) Use the gdalUtils package in R:
>
> gdalwarp(srcfile=get_subdatasets(filemodis)[10], dstfile=
> ".../gdalMODIS_avhrr.tif", s_srs = crs(modis), t_srs =crs(avhrr) )
>
> This returns a raster with essentially no spatial extent.
>
> gdalMODISavhrr<-raster(".../gdalMODIS_avhrr.tif")
> #class : RasterLayer
> #dimensions : 357, 12850, 4587450 (nrow, ncol, ncell)
> #resolution : 0.02801551, 0.02801573 (x, y)
> #extent : -180, 179.9993, 79.99838, 90 (xmin, xmax, ymin, ymax)
> #coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
> #values : -32768, 32767 (min, max)
>
> Any ideas on why reprojecting this MODIS data is so difficult?
>
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] stars::RasterIO using extent info?
FWIW I have a raster-delivering-front-end for RasterIO in this dev package:
https://github.com/hypertidy/lazyraster
It uses the more obvious extent() idioms and will even use an open plot if
nothing else is specified.
(It uses an independent binding to GDAL in the vapour package).
That might help, or not. It's on my list to find a sensible way for stars
to leverage this obvious ease-of-use. rgdal::readGDAL always had an
interface to RasterIO, but only raster ever made use of that, and it's
indirect - raster doesn't allow a mix of extent and resolution in its
approach.
Cheers, Mike.
On Thu, 15 Nov 2018 at 02:27 Howard, Tim G (DEC) via R-sig-Geo <
r-sig-geo@r-project.org> wrote:
> Ok, fair enough that there's no magic involved. I've worked through the
> details with the small example as follows. The result is only a couple of
> cells different in each direction.
>
> library(stars)
> tif <- system.file("tif/L7_ETMs.tif", package = "stars")
> x <- read_stars(tif)
> x <- x[,,,1]
> # calculate a circular polygon at the center of the raster
> pol <- x %>% st_bbox() %>% st_as_sfc() %>% st_centroid() %>% st_buffer(500)
>
> bb_pol <- st_bbox(pol)
>
> xoff <- st_dimensions(x)$x$offset
> xdelt <- st_dimensions(x)$x$delta
> yoff <- st_dimensions(x)$y$offset
> ydelt <- st_dimensions(x)$y$delta
> cropXoff <- (bb_pol$xmin - xoff + xdelt)/xdelt
> cropXsize <- (bb_pol$xmax - bb_pol$xmin)/xdelt
> cropYoff <- (bb_pol$ymin - yoff + ydelt)/ydelt
> cropYsize <- (bb_pol$ymax - bb_pol$ymin)/ydelt
>
> # if ydelt is negative, get abs of ysize and move yoffset to the top
> if(cropYsize < 0){
> cropYsize <- abs(cropYsize)
> cropYoff <- cropYoff - cropYsize
> }
>
> rasterio <- list(nXOff = cropXoff, nYOff = cropYoff, nXSize = cropXsize,
> nYSize = cropYsize, bands = c(1))
> (z = read_stars(tif, RasterIO = rasterio))
> plot(z)
>
> # crop it if desired
> plot(z[pol])
>
> ## compare to proxy method
>
> x <- read_stars(tif, proxy = TRUE)
> x <- x[,,,1]
> y <- st_as_stars(x[pol])
> plot(y)
>
> # only a couple of cells off!
> z
> y
>
>
>
>
> --
> Date: Tue, 13 Nov 2018 17:26:16 +0100
> From: Edzer Pebesma
> To: r-sig-geo@r-project.org
> Subject: Re: [R-sig-Geo] stars::RasterIO using extent info?
> Message-ID: <9d037da7-dc9c-9886-d6fc-5864cf8b4...@uni-muenster.de>
> Content-Type: text/plain; charset="utf-8"
>
>
>
> On 11/13/18 4:10 PM, Howard, Tim G (DEC) via R-sig-Geo wrote:
> > Dear list,
> >
> > I am exploring the different options for reading parts of large imagery
> object in stars, as discussed here:
> >
> > https://r-spatial.github.io/stars/articles/proxy.html
> >
> > My ultimate goal is to read into RAM only a clipped portion of a large
> raster (well, actually a raster stack, but taking baby steps here).
> >
> > My immediate question: the `RasterIO` option of read_stars defines cell
> offsets and cell counts (*Size). Is there a straightforward way to
> calculate these values given extent information?
> >
> > Reproducible example (mostly taken from here:
> https://www.r-spatial.org/r/2018/03/22/stars2.html):
> >
> > library(stars)
> > tif <- system.file("tif/L7_ETMs.tif", package = "stars")
> > x <- read_stars(tif) # read entire tif into ram
> > x <- x[,,,1] #get just one layer for now
> > # calculate a circular polygon at the center of the raster
> > pol <- x %>% st_bbox() %>% st_as_sfc() %>% st_centroid() %>%
> st_buffer(500)
> > plot(x)
> > # interestingly, I don't think the circle is in the right place when
> plotted
> > plot(st_geometry(pol), add = TRUE, border = "red")
> > # this is what I'd like to be able to restrict to what is read in memory:
> > plot(x[pol])
> >
> > ## read only portion of tif using proxy object
> > x <- read_stars(tif, proxy = TRUE)
> > x <- x[,,,1]
> > y <- st_as_stars(x[pol])
> > plot(y) # this is cropped to the extent (but not the circle - let's not
> worry about that right now)
> >
> > Question: can I do the equivalent with the RasterIO options in stars?
> Said another way, instead of setting up the proxy, can I map my extent
> object (or bounding box) directly to the cell count values needed for
> RasterIO?
>
> stars can do the math, and so can you; it is explained here:
>
> https://r-spatial.github.io/stars/articles/data_model.html
>
> stars uses some functions directly from GDAL which it doesn't expose to
> the user, but there is no magic going on here.
>
> &
Re: [R-sig-Geo] Reordering geographical coordinates clockwise to make a polygon
alphahull or a hand-crafted "triangulate and cull long/large triangles" might be workable options (geometry and RTriangle have the fastest and most useful triangulation functions). It's not generally a tractable problem afaics. Some refs that came up in recent discussions: https://pdfs.semanticscholar.org/669b/1415cd64d39f1e44fd90c2b9d9453c42581b.pdf http://erikdemaine.org/polygonization/ Cheers, Mike. On Tue, 6 Nov 2018 at 05:26 Edzer Pebesma wrote: > Sorry for my previous answer, I didn't get the question fully. > > I'm afraid I still don't get the question fully, but functions that > might help are sf::st_line_merge (creates a LINESTRING from line pieces) > and sf::st_polygonize (creates a polygon from a LINESTRING that forms a > closed ring) > > On 11/5/18 5:35 PM, Patrick Giraudoux wrote: > > > > Apologize to answer to myself: the way described below would work only > > if there are no concave "peninsula" towards north or south inside the > > polygon. Even thinking about an alternate solution, e.g. the Traveling > > Salesman Problem (TSP), ie. the shortest route linking all points, one > > could get wrong since points of a narrowpeninsula could have points > > closest to the opposite border than from the next point on the border... > > > > Suppose we are stuck, and should redraw polygons by hand > > > > > > Le 05/11/2018 à 14:02, Patrick Giraudoux a écrit : > >> > >> Dear listers, > >> > >> There is an interesting post here: > >> > https://stackoverflow.com/questions/6989100/sort-points-in-clockwise-order > >> dealing on the issue. However, I would like to know if a function has > >> been already developped in a R package. > >> > >> I am coping with a young colleague's shapefile where borders of > >> polygons have been drawn as lines quite inconsistently. To change this > >> SpatialLinesDataFrame into a SpatialPoints object is easy. The idea is > >> to select the points bordering each polygon (delete the others), > >> define the point set as a Polygon, then rebuild step by step a > >> SpatialPolygonsDataFrame with all its (~25) polygons. It would be much > >> quicker than to redraw one by one the 5160 data points (two times on > >> borders shared by two polygons). > >> > >> The problem is that the points must be reordered clockwise (the way > >> lines making the borders is all except clockwise) before making them > >> a polygon. > >> > >> Any function already developped for that ? > >> > >> Cheers, > >> > >> Patrick > >> > >> > >> > > > > > > [[alternative HTML version deleted]] > > > > ___ > > R-sig-Geo mailing list > > R-sig-Geo@r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > -- > Edzer Pebesma > Institute for Geoinformatics > Heisenbergstrasse 2, 48151 Muenster, Germany > Phone: +49 251 8333081 <+49%20251%208333081> > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Sf find number of parts in multigeometry
I see perfectly good answers to this, but can't resist sharing my own
approach.
I use gibble::gibble for a summary of parts. See how the multi-part
object is number 4, and has 3 subobjects (subobject will repeat within
object for holes).
library(sf)
x <- read_sf(system.file("gpkg/nc.gpkg", package = "sf"))
gibble::gibble(x)
#A tibble: 108 x 5
# nrow ncol type subobject object
#
#127 2 MULTIPOLYGON 1 1
#226 2 MULTIPOLYGON 1 2
#328 2 MULTIPOLYGON 1 3
#426 2 MULTIPOLYGON 1 4
#5 7 2 MULTIPOLYGON 2 4
#6 5 2 MULTIPOLYGON 3 4
#734 2 MULTIPOLYGON 1 5
#...
(I use that for mapping out set-based operations on geometry data, it
doesn't make a huge amount of sense on its own. I suppose a rapply scheme
could be constructed to pluck out things, but you'd also want path extents
and sizes and so forth for greater control).
But, if the biggest area of each multipolygon is what you want, give each a
unique ID, use st_cast to POLYGON, group by parent and slice out the
largest.
library(dplyr)
x %>% mutate(ID = row_number()) %>% st_cast("POLYGON") %>%
group_by(ID) %>% arrange(desc(st_area(.))) %>% slice(1) %>% ungroup()
Note that holes within a part might reduce the area logic, but so will
details of the map projection in use and so on. It's helpful to learn the
structure of the underlying geometry lists to craft your own rogue
solutions.
HTH
On Tue, Oct 23, 2018, 13:32 Martin Tomko wrote:
> I am looking for an equivalent to Postgis ST_NumGeometries
> https://postgis.net/docs/ST_NumGeometries.html
> I have multipolygons in an sf df, where most of them are likely to be
> single part. I want to identify those that are not single part, and
> possibly retain only the largest polygon part, and cast all into Polygon.
> Any advice is appreciated, thanks!
>
> Martin
>
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] subsetting a spatial polygons
crop doesn't select it actually cuts on the boundary, for simple shapes I
use coordinates (for centroids in easy matrix) and select with [ using
tests on x and y.
Cheers, Mike
On Wed, Sep 12, 2018, 15:29 Tim Appelhans wrote:
> Antonio,
>
> I am not sure why you think that your solution is not very elegant.
> In case you want to have more visual control over the subsetting, you
> could try mapedit:
>
> library(mapedit)
> myselection = selectFeatures(polys, mode = "draw")
>
> which will let you draw a e.g. rectangle and only return those features
> that intersect it.
>
> Best
> Tim
>
>
> On 09/12/2018 04:18 AM, Antonio Silva wrote:
> > Dear list users
> >
> > I have a SpatialPolygons with several squares. How to subset it to have
> > only the squares between given latitudes and longitudes?
> >
> > In the example
> >
> > library(sp)
> > library(raster)
> > grd <-
> >
> GridTopology(cellcentre.offset=c(-47.75,-25.416667),cellsize=c(10/60,10/60),cells.dim=c(23,12))
> > polys <- as.SpatialPolygons.GridTopology(grd)
> > proj4string(polys) <- CRS("+proj=longlat +datum=WGS84")
> > plot(polys,axes=T)
> >
> > How to select only the squares, let's say, between 24-25°S and 45-46°W?
> >
> > The farthest I went was:
> >
> > e <- extent(-45.9,-45.1,-24.9,-24.1) # which is not very elegant
> > mask <- crop(polys,e)
> > polys2 <- polys[mask,]
> > plot(polys2,add=T,col="green")
> >
> > Thanks a lot. Best regards
> >
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Problems converting rasters from float to integer.
That doesn't help though!
Sorry, I thought that was it, will sleep on it.
Cheers, Mike
On Fri, 24 Aug 2018 at 22:00 Michael Sumner wrote:
> Sorry, I was wrong - the actual issue is that the writeRaster argument is
> "datatype", it gets conflated with the function that is dataType().
>
> f <- "test_int.tif"
> writeRaster(r, f, datatype = "INT2U", overwrite = TRUE)
> s <- raster(f)
> dataType(s)
>
> [1] "INT2U"
>
> Not the first time this has caught me out. I found it by checking
> mode(readGDAL(f)[[1]]) and realizing that the TIFF type was Float32, prior
> to the next steps.
>
> HTH
>
> :)
>
> Cheers, Mike
>
> On Fri, 24 Aug 2018 at 18:26 nevil amos wrote:
>
>> Roger,
>>
>> there was a reproducible example in my post: it seems to have got a bit
>> scrambled in your reply.
>>
>> from the help in raster I thought that if I included datatype="INT2S" in
>> the writeRaster() command the values would be formatted in the output file
>> ( and the subsequent reading using raster() of that file) as integer is
>> this not the case?
>>
>> Here it is the reproducible example
>>
>> v<-c(rep(1.0,25),rep(0.0,50),rep(NA,25))
>> m<-matrix(v,10,10)
>> r<-raster(m)
>> dataType(r)
>> writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T)
>> s<-raster("test_int.tif")
>> dataType(s)
>>
>> On Fri, 24 Aug 2018 at 18:05, Roger Bivand wrote:
>>
>>> Thanks, Mike. I agree that the lower level interface in rgdal is
>>> flexible enough, but as you say non-trivial. Nevil: could you please
>>> provide a small reproducible example to point people in the right direction?
>>>
>>> Roger
>>>
>>> Roger Bivand
>>> Norwegian School of Economics
>>> Bergen, Norway
>>>
>>>
>>>
>>> Fra: Michael Sumner
>>> Sendt: fredag 24. august, 09.14
>>> Emne: Re: [R-sig-Geo] Problems converting rasters from float to integer.
>>> Til: nevil amos
>>> Kopi: r-sig-geo@r-project.org
>>>
>>>
>>> I'm pretty sure readGDAL from rgdal (that raster uses) will keep as
>>> integers, so you can build an empty raster and copy the values over. But
>>> will need to derive from the rgdal version to catch all the metadata
>>> (structure, extent, and crs). I think it's doable and will try later. There
>>> are other options but nothing trivial afaik. Cheers, Mike On Fri, 24 Aug
>>> 2018, 16:45 nevil amos wrote: > I have a large number of rasters ( tiffs)
>>> that contain whole number values > between 0 and 100, and NA values. or
>>> 0,1,and NA > they are currently in Float format, I am trying to rewrite
>>> them as integer > rasters, firstly to save space, and secondly so that I
>>> can later read the > values stack of all the rasters into an integer array.
>>> using getValues(). > > To do this I am setting the dataType to INT2U in
>>> writeRaster, however when > I read the save file back into R the format is
>>> not INT2U but FLT8S > > toy example: > > v m r dataType(r) >
>>> writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T) > s dataType(s)
>>> > > the result I get: > > > v > m > r > dataType(r) > [1] "FLT4S" > >
>>> writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T) > > s >
>>> dataType(s) > [1] "FLT8S" > > > > > Can you suggest how I ensure the values
>>> are stored as integer? > > Many thanks > > [[alternative HTML version
>>> deleted]] > > ___ > R-sig-Geo
>>> mailing list > R-sig-Geo@r-project.org >
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner
>>> Software and Database Engineer Australian Antarctic Division 203
>>> Channel Highway Kingston Tasmania 7050 Australia
>>> <https://maps.google.com/?q=203+Channel%0D%0A+Highway+Kingston+Tasmania+7050+Australia=gmail=g>
>>> [[alternative HTML version deleted]]
>>> ___ R-sig-Geo mailing list
>>> R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>> --
> Dr. Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> 203 Channel Highway
> Kingston Tasmania 7050 Australia
>
> --
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Problems converting rasters from float to integer.
Sorry, I was wrong - the actual issue is that the writeRaster argument is
"datatype", it gets conflated with the function that is dataType().
f <- "test_int.tif"
writeRaster(r, f, datatype = "INT2U", overwrite = TRUE)
s <- raster(f)
dataType(s)
[1] "INT2U"
Not the first time this has caught me out. I found it by checking
mode(readGDAL(f)[[1]]) and realizing that the TIFF type was Float32, prior
to the next steps.
HTH
:)
Cheers, Mike
On Fri, 24 Aug 2018 at 18:26 nevil amos wrote:
> Roger,
>
> there was a reproducible example in my post: it seems to have got a bit
> scrambled in your reply.
>
> from the help in raster I thought that if I included datatype="INT2S" in
> the writeRaster() command the values would be formatted in the output file
> ( and the subsequent reading using raster() of that file) as integer is
> this not the case?
>
> Here it is the reproducible example
>
> v<-c(rep(1.0,25),rep(0.0,50),rep(NA,25))
> m<-matrix(v,10,10)
> r<-raster(m)
> dataType(r)
> writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T)
> s<-raster("test_int.tif")
> dataType(s)
>
> On Fri, 24 Aug 2018 at 18:05, Roger Bivand wrote:
>
>> Thanks, Mike. I agree that the lower level interface in rgdal is flexible
>> enough, but as you say non-trivial. Nevil: could you please provide a small
>> reproducible example to point people in the right direction?
>>
>> Roger
>>
>> Roger Bivand
>> Norwegian School of Economics
>> Bergen, Norway
>>
>>
>>
>> Fra: Michael Sumner
>> Sendt: fredag 24. august, 09.14
>> Emne: Re: [R-sig-Geo] Problems converting rasters from float to integer.
>> Til: nevil amos
>> Kopi: r-sig-geo@r-project.org
>>
>>
>> I'm pretty sure readGDAL from rgdal (that raster uses) will keep as
>> integers, so you can build an empty raster and copy the values over. But
>> will need to derive from the rgdal version to catch all the metadata
>> (structure, extent, and crs). I think it's doable and will try later. There
>> are other options but nothing trivial afaik. Cheers, Mike On Fri, 24 Aug
>> 2018, 16:45 nevil amos wrote: > I have a large number of rasters ( tiffs)
>> that contain whole number values > between 0 and 100, and NA values. or
>> 0,1,and NA > they are currently in Float format, I am trying to rewrite
>> them as integer > rasters, firstly to save space, and secondly so that I
>> can later read the > values stack of all the rasters into an integer array.
>> using getValues(). > > To do this I am setting the dataType to INT2U in
>> writeRaster, however when > I read the save file back into R the format is
>> not INT2U but FLT8S > > toy example: > > v m r dataType(r) >
>> writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T) > s dataType(s)
>> > > the result I get: > > > v > m > r > dataType(r) > [1] "FLT4S" > >
>> writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T) > > s >
>> dataType(s) > [1] "FLT8S" > > > > > Can you suggest how I ensure the values
>> are stored as integer? > > Many thanks > > [[alternative HTML version
>> deleted]] > > _______ > R-sig-Geo
>> mailing list > R-sig-Geo@r-project.org >
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner
>> Software and Database Engineer Australian Antarctic Division 203 Channel
>> Highway Kingston Tasmania 7050 Australia
>> <https://maps.google.com/?q=203+Channel%0D%0A+Highway+Kingston+Tasmania+7050+Australia=gmail=g>
>> [[alternative HTML version deleted]]
>> ___ R-sig-Geo mailing list
>> R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>> --
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Problems converting rasters from float to integer.
I'm pretty sure readGDAL from rgdal (that raster uses) will keep as
integers, so you can build an empty raster and copy the values over. But
will need to derive from the rgdal version to catch all the metadata
(structure, extent, and crs).
I think it's doable and will try later. There are other options but nothing
trivial afaik.
Cheers, Mike
On Fri, 24 Aug 2018, 16:45 nevil amos wrote:
> I have a large number of rasters ( tiffs) that contain whole number values
> between 0 and 100, and NA values. or 0,1,and NA
> they are currently in Float format, I am trying to rewrite them as integer
> rasters, firstly to save space, and secondly so that I can later read the
> values stack of all the rasters into an integer array. using getValues().
>
> To do this I am setting the dataType to INT2U in writeRaster, however when
> I read the save file back into R the format is not INT2U but FLT8S
>
> toy example:
>
> v<-c(rep(1.0,25),rep(0.0,50),rep(NA,25))
> m<-matrix(v,10,10)
> r<-raster(m)
> dataType(r)
> writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T)
> s<-raster("test_int.tif")
> dataType(s)
>
> the result I get:
>
> > v<-c(rep(1.0,25),rep(0.0,50),rep(NA,25))
> > m<-matrix(v,10,10)
> > r<-raster(m)
> > dataType(r)
> [1] "FLT4S"
> > writeRaster(r,"test_int.tif",dataType="INT2U",overwrite=T)
> > s<-raster("test_int.tif")
> > dataType(s)
> [1] "FLT8S"
> >
>
>
> Can you suggest how I ensure the values are stored as integer?
>
> Many thanks
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] how to create several polygons from a list of vertices
a data.frame with the vertices (lon / lat) and codes from
> several
> >>> squares (more than 500 in the real dataset).
> >>> I want to create an object with these polygons (squares) and after this
> >>> export it as a shapefile.
> >>> With the script below I can draw one square.
> >>> library(sp)
> >>> P1 = Polygon(vertices[1:4,1:2])
> >>> Ps1 = SpatialPolygons(list(Polygons(list(P1), ID = "1")),
> >>> proj4string=CRS("+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"))
> >>> plot(Ps1, axes = TRUE)
> >>>
> >>> Now I'm trying to create one object with all squares at once.
> >>> Is it possible?
> >>>
> >>> Thanks a lot,
> >>>
> >>> Antônio Olinto
> >>>
> >>> sample data:vertices
> >>>lon lat cod
> >>> 1 -33 -23 1
> >>> 2 -32 -23 1
> >>> 3 -32 -22 1
> >>> 4 -33 -22 1
> >>> 5 -32 -23 2
> >>> 6 -31 -23 2
> >>> 7 -31 -22 2
> >>> 8 -32 -22 2
> >>> 9 -31 -23 3
> >>> 10 -30 -23 3
> >>> 11 -30 -22 3
> >>> 12 -31 -22 3
> >>> 13 -33 -22 4
> >>> 14 -32 -22 4
> >>> 15 -32 -21 4
> >>> 16 -33 -21 4
> >>> 17 -32 -22 5
> >>> 18 -31 -22 5
> >>> 19 -31 -21 5
> >>> 20 -32 -21 5
> >>> 21 -31 -22 6
> >>> 22 -30 -22 6
> >>> 23 -30 -21 6
> >>> 24 -31 -21 6
> >>>
> >>> [[alternative HTML version deleted]]
> >>>
> >>> ___
> >>> R-sig-Geo mailing list
> >>> R-sig-Geo@r-project.org
> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>
> >>
> >
> > [[alternative HTML version deleted]]
> >
> > ___
> > R-sig-Geo mailing list
> > R-sig-Geo@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Difference in area calculation between QGIS and R
Raster uses a (discretized) cosine-of-latitude approximati (popular amongst
longlat map makers).
QGIS uses a project to local equal area projection method or maybe some
other approach.
There's lots and f options, all that matters is what your work needs.
Cheers, Mike
On Wed, 27 Jun 2018, 22:14 Suncus Etruscus,
wrote:
> Dear List,
> I am trying to use R ("sp" and "raster" packages) to calculate the area of
> several polygons (CRS of the shapefile EPSG: 4326 - WGS84).
>
> I used this line of code:
>
> shapefile_name$area_km2 <- area(shapefile_name)/100
>
> However, when I used QGIS to calculate the area of the same polygons
> (through the "$area" function), I found there was a slight difference (in
> every polygon).
>
> For example, for a polygon of about 30 000 km2, the area calculated in R
> was 50 km2 smaller.
>
> What could be the cause?
>
> Thanks in advance,
> N.
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] What is the Coordinate Reference System?
Agree with Edzer this application of eqc is anomalous and after it confused us initially we now ignore it and apply the unproblematic proj=longlat interpretation to these products (after checking). I might explore the L3 products to find out why this occurred, but the ocean colour forum is the usual place to find discussion. Cheers, Mike On Mon, 18 Jun 2018, 15:59 Edzer Pebesma, wrote: > Plate carree is essentially a linear mapping of long and lat to x and y, > without changing aspect ratio (scale factor 1 at equator). > > projecting long/lat towards plate caree using the proj4 string you > mention will however change the units from degrees to m. So while the > plots of of data in linear long/lat (with aspect ration 1) looks > identical to the one obtained in plate caree (with aspect ratio 1), the > numerical values along their axes are very different. > > Many netcdf files carry a grid (raster) with the x and y coordinates of > grid cells (centers?) in another coordinate system, but the ones you > mention below don't, as far as I can tell. I think what they mean by > mentioning plate carree is just saying that this is a regular long/lat > degree grid; rather unlucky wording IMO. > > On 06/15/2018 08:14 PM, Ben Tupper wrote: > > Hi, > > > > That's an interesting approach. > > > > On the other hand, sometimes I get bitten (well, muddled really) when I > use raster to determine the crs of a ncdf resource. It all sort of depends. > > > > Below is an example for Aqua MODIS imagery from OBPG ( > https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/Mapped/Daily/9km/sst/2017/ > <https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/Mapped/Daily/9km/sst/2017/>). > The OBPG docs (https://oceancolor.gsfc.nasa.gov/products/ < > https://oceancolor.gsfc.nasa.gov/products/>) state that the data is > served up as 'Plate Carrée' which I think has a crs ... > > > > '+proj=eqc +lat_ts=0 +lat_0=0 +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84 > +datum=WGS84 +units=m +no_defs ' > > > > ... as described here http://spatialreference.org/ref/epsg/32662/ < > http://spatialreference.org/ref/epsg/32662/> > > > > You can see below that raster comes up with... > > > > '+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0' > > > > ... which is http://spatialreference.org/ref/epsg/4326/ < > http://spatialreference.org/ref/epsg/4326/> > > > > Explicitly setting the crs while reading doesn't help. I have never > resolved if it is an OBPG documentation bug, my confusion about > projections, or some other mystery. So, my usual practice is to assume > that raster makes the right call. > > > > start > > > > library(raster) > > URI = ' > https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A2017175.L3m_DAY_SST_sst_9km.nc > ' > > > > # ~3.8MB > > ok = download.file(URI, basename(URI)) > > > > R = raster::raster(basename(URI), varname = 'sst') > > R > > # class : RasterLayer > > # dimensions : 2160, 4320, 9331200 (nrow, ncol, ncell) > > # resolution : 0.0834, 0.0834 (x, y) > > # extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax) > > # coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 > > # data source : /Users/Shared/code/R/others/record/spnc/ > A2017175.L3m_DAY_SST_sst_9km.nc > > # names : Sea.Surface.Temperature > > # zvar: sst > > > > S = raster::raster(basename(URI), varname = "sst", crs = '+proj=eqc > +lat_ts=0 +lat_0=0 +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 > +units=m +no_defs') > > # Warning message: > > # In .getProj(prj, crs) : > > # argument "crs" ignored because the file provides a crs > > > > end > > > > > > Cheers, > > Ben > > > > > >> On Jun 14, 2018, at 6:42 PM, Michael Sumner wrote: > >> > >> Try raster::projection(raster::raster(a)) first, then delve if necessary > >> into the ncdump summary with print(raster::raster(a))) > >> > >> But, raster(a) may be enough here? To get that stuff more directly from > the > >> NetCDF API you have to use the inq and att functions of ncdf4 or RNetCDF > >> and that's no fun. > >> > >> I'd also try raster(rgdal::readGDAL(a)) which is a totally different but > >> equivalent read path. > >> > >> Cheers, Mike > >> > >> On Thu, 14 Jun 2018, 11:03 Sergio Ibarra, > wrote: > >> > >>> Greetings, > >>> > >>> I'm want to read NetCDF spatial data from a meteorological model (WRF > >>> model). The NetCDF c
Re: [R-sig-Geo] What is the Coordinate Reference System?
Try raster::projection(raster::raster(a)) first, then delve if necessary
into the ncdump summary with print(raster::raster(a)))
But, raster(a) may be enough here? To get that stuff more directly from the
NetCDF API you have to use the inq and att functions of ncdf4 or RNetCDF
and that's no fun.
I'd also try raster(rgdal::readGDAL(a)) which is a totally different but
equivalent read path.
Cheers, Mike
On Thu, 14 Jun 2018, 11:03 Sergio Ibarra, wrote:
> Greetings,
>
> I'm want to read NetCDF spatial data from a meteorological model (WRF
> model). The NetCDF can have one of these three projections:
>
> Polar stereographic
> Lambert conformal
> Mercator projection
>
> (See figure
>
> https://user-images.githubusercontent.com/27447280/41389756-ed23f91a-6f68-11e8-961e-6ba901913c54.png
> )
>
> What is the Coordinate Reference System for each case?
>
> Thanks!
>
> Here an example with ncdf4 and raster:
>
> library(ncdf4)
> library(raster)
> a <- tempfile()
> download.file(url = "
> http://www.unidata.ucar.edu/software/netcdf/examples/wrfout_v2_Lambert.nc
> ",
> destfile = a) #78 Mb
> wrf <- nc_open(a)
> paste("The file has",wrf$nvars,"variables")
> paste("The file has",names(wrf$var))
> lat <- ncvar_get( wrf, "XLAT" )
> lon <- ncvar_get( wrf, "XLONG" )
> t2 <- ncvar_get( wrf, "T2" ) - 273.15
> dim(t2) #73 60 13
> #image
> image(t2[, , 12])
> # raster
> rt2 <- raster(t(t2[1:dim(t2)[1],
>dim(t2)[2]:1,
>12]),
> xmn = min(lon),
> xmx = max(lon),
> ymn = min(lat),
> ymx = max(lat),
> * crs="+init=epsg:4326") # ???*
> spplot(rt2)
>
>
>
> --
> Sergio Ibarra Espinosa
> Post Doc
> PhD in Atmospheric Sciences
> Instituto de Astronomia, Geofísica e Ciências Atmosféricas
> Universidade de São Paulo
> Rua do Matão, 1226
> São Paulo-SP - Brasil -
> 05508-090
> +55-11-97425-3791
> Skype: sergio_ibarra1
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] raster::clump not working?
Joao, this is what you are after I think. It's important to use sf/fasterize otherwise any holes filled by other patches won't be identified (and it's faster). It won't scale well given rasterToPolygons, but there might be other options using sf related tricks. library(raster) r <- raster(volcano) %/% 20 ## p has six distinct values (multipolygons) p <- rasterToPolygons(r, dissolve = TRUE) ## pp has ten distinct patches pp <- disaggregate(p) pp$patch <- seq_len(nrow(pp)) ## back to raster #rr <- rasterize(pp, r, field = pp$patch) # or faster with sf/fasterize (also this gets the holes filled correctly) rr <- fasterize::fasterize(sf::st_as_sf(pp), r, field = "patch") unique(values(rr)) Cheers, Mike. On Fri, 25 May 2018 at 21:15 João Carreiras <jmbcarrei...@gmail.com> wrote: > Dear Ben, > > Thank you for your prompt reply. > > Now I see what clump does. I just thought clump would give the same > result as ArcGIS "Region Group". I need some command to assign a > different value to each patch. And by patch I mean contiguous pixels > having the same value, so that in this (absurd) example I would get > 648 patches. > > Take care > Joao > > On 25 May 2018 at 11:35, Ben Tupper <btup...@bigelow.org> wrote: > > Hi, > > > > I think it is actually correct - there is only one 'clump' of connected > > cells. In raster clumps are separated from other clumps by backgound (0 > or > > NA). In your example there is no background anywhere so there is just > one > > clump. > > > > You can see the difference if you divide x in to halves with a row of > NAs. > > Note that limited spatial extent so the clumping doesn't wrap around the > > globe which it will for [-180, 180]. > > > > r <- raster(ncols=36, nrows=18, xmn = 0, xmx = 36, ymn = 0, ymx = 18) > > x <- init(r, fun='cell') > > > > x[9,1:36] <- NA > > y <- clump(x) > > y > > # class : RasterLayer > > # dimensions : 18, 36, 648 (nrow, ncol, ncell) > > # resolution : 1, 1 (x, y) > > # extent : 0, 36, 0, 18 (xmin, xmax, ymin, ymax) > > # coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 > > # data source : in memory > > # names : clumps > > # values : 1, 2 (min, max) > > > > x[1:18, 18] <- NA > > y > > # class : RasterLayer > > # dimensions : 18, 36, 648 (nrow, ncol, ncell) > > # resolution : 1, 1 (x, y) > > # extent : 0, 36, 0, 18 (xmin, xmax, ymin, ymax) > > # coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 > > # data source : in memory > > # names : clumps > > # values : 1, 4 (min, max) > > > > > > Cheers, > > Ben > > > > > > On May 25, 2018, at 6:00 AM, João Carreiras <jmbcarrei...@gmail.com> > wrote: > > > > Hi! > > > > I've been trying to run the clump command but the output is consistently > a > > raster with values == 1. Please find below an example. > > > > I'm sure I'm doing something stupid. However, the command is really > > straightforward and I can't seem to identify what the problem might be. > > > > Any help really appreciated. > > Thanks > > Joao > > > > r <- raster(ncols=36, nrows=18) > > x <- init(r, fun='cell') > > x > > class : RasterLayer > > dimensions : 18, 36, 648 (nrow, ncol, ncell) > > resolution : 10, 10 (x, y) > > extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax) > > coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 > > data source : in memory > > names : layer > > values : 1, 648 (min, max) > > a <- clump(x) > > a > > class : RasterLayer > > dimensions : 18, 36, 648 (nrow, ncol, ncell) > > resolution : 10, 10 (x, y) > > extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax) > > coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 > > data source : in memory > > names : clumps > > values : 1, 1 (min, max) > > > > ___ > > R-sig-Geo mailing list > > R-sig-Geo@r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > > > Ben Tupper > > Bigelow Laboratory for Ocean Sciences > > 60 Bigelow Drive, P.O. Box 380 > > East Boothbay, Maine 04544 > > http://www.bigelow.org > > > > Ecological Forecasting: https://eco.bigelow.org/ > > > > > > > > > > > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Measuring length of trajectories
On Thu, 24 May 2018 at 23:32 Jeri Wisman <jwism...@odu.edu> wrote: > Hi all - > > I am trying to measure the length of time of trajectories taken of seabird > movements to be able to relate time of flight to distance of flight. Any > suggestions of a function or package on how to measure the time? Thanks, > There are very many, including adehabitatLT package and the trackDistance function in the trip package (I know how that one works so it's easy for me to help with, but it's not as sophisticated as adehabitat family so pros/cons). Both packages requires you to get your data into the right format. There's more general capability in sp::spDists which has arguments longlat (set to TRUE for great circle distances from longlat) and segments, which gives sequential paired distances. Both adehabitatLT and trip will handle the grouping of locations into separate paths (ordered by time). I'd say it's worth trying some of the purpose-built packages, because then some of the metrics can be extracted very easily (once you know your way around). Explore the "Moving objects, trajectories" section of this Task View https://gist.github.com/mdsumner/0a3cb0e58bf9d37b782943ac269e1eff and this list of recentish additions, pending update to the Task View, which gives a taste of how much activity there is in this space(!). https://gist.github.com/mdsumner/0a3cb0e58bf9d37b782943ac269e1eff Cheers, Mike. > > *Jeri Wisman* | Masters Candidate > Old Dominion University > Department of Biological Sciences > Mills Godwin Building, Room 312 > Norfolk VA 23529 USA > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Intersection of polygons with raster
Hi Kent, this is pretty straightforward with raster::extract. If speed is an issue you can burn the polygon id into the grid and then group values by that. library(raster) r <- raster(volcano) ## very simplistic polygon layer example r2 <- raster(r) res(r2) <- res(r2) * 20 p <- sf::st_as_sf(as(r2, "SpatialPolygonsDataFrame")) ## a dummy threshold value onefoot <- 150 ## grab the first column from extract (it gives multiple columns for multi-layer rasters) p$onefoot <- extract(r, p, fun = function(x, na.rm = TRUE) any(x > onefoot))[,1] plot(sf::st_geometry(p), col = p$onefoot + 1) contour(r, levels = onefoot, col = "white", add = T) ## if speed is an issue, use fasterize for a more abstract workflow library(fasterize) p$rownum <- 1:nrow(p) idgrid <- fasterize(p, r, field = "rownum") p$onefoot <- tapply(values(r), values(idgrid), function(x, na.rm = TRUE) any(x > onefoot)) plot(p) FWIW it's always useful to provide a reprex, since that does take time and might miss the mark for your situation. It's also trickier to provide two objects that are relevant to each other, so any upfront work you can do in an example helps the answerer. HTH On Sat, 19 May 2018 at 12:20 Kent Johnson <kent3...@gmail.com> wrote: > Hi, > > I have a raster object `flood` containing projected flooding levels and a > simple features object `parcels` containing MULTIPOLYGONs representing > property parcels. I would like to find all the parcel polygons for which > there is any flood > 1 foot. What function(s) can do this? Something like > raster::intersect(parcels, flood >= 1) > > I'll put together a reprex if that helps. Hoping someone can point me to > the right function to do this, I haven't found anything promising. > > Thanks, > Kent > > [[alternative HTML version deleted]] > > _______ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Tutorial on reading hyperspectral data
On Sun, 6 May 2018, 03:31 Raja Natarajan, <nr...@tnau.ac.in> wrote: > Hi > > Greetings > > Please share any tutorial on read and open hyperspectral astrology images > using R > Hi Raja, I'd start with running raster::brick on one of your file paths and let us know the result, also your sessionInfo() because support for the likely format in use (what is it?) is platform and version dependent, though things improved greatly recently. There are tutorials on hyperspectral in R with raster and rhdf5 for remote sensing data, I don't know of any astro specific ones.You may not need rhdf5 any more, but geographic assumptions can still bite and make things harder. Just a guess that this may be relevant to you https://www.neonscience.org/create-raster-stack-hsi-hdf5-r Cheers, Mike > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Raster stream network to lines
Not much to add but if you didn't notice there is a degenerate line in that isolated pixel, so if you were looking at sets of connected pixels in turn you could possibly re-process the output of contourLines to join things up: cl <- contourLines(list(x = xFromCol(foo), y = rev(yFromRow(foo)), z = as.matrix(t(flip(foo, "y", levels = 1) ## the third one is a nearly-degenerate line in the centre of the isolated cell cl[[3]] Cheers, Mike. On Thu, 19 Apr 2018 at 08:22 Andy Bunn <andy.b...@wwu.edu> wrote: > Brilliant! And a heck of a workaround. But what if foo is a projected > raster? I can add a projection system but all the coordinate info is lost > in the shuffle. > > On 4/18/18, 1:17 PM, "Joseph Stachelek" <joseph.stache...@gmail.com> > wrote: > > Hi Andy, > > I have posted some code showing one way of doing this > with the `sf` package: > > https://gist.github.com/jsta/d8d8e8d79877c720b6842ae8efa2 > b9b8 > > If you are doing this a lot I recommend picking up GRASS: > > https://grasswiki.osgeo.org/wiki/R.stream.*_modules > https://grass.osgeo.org/grass74/manuals/r.to.vect.html > > --Joe > > -Original Message- > From: Andy Bunn <andy.b...@wwu.edu> > To: R-sig-Geo <r-sig-geo@r-project.org> > Subject: [R-sig-Geo] Raster stream network to lines > Date: Wed, 18 Apr 2018 19:03:21 + > > I have a raster of streams with 0 being non-stream and 1 > being stream. Is there a way to convert this into a > SpatialLinesDataFrame? rasterToContour() almost does what > I want but fails to draw the line on queens rules, etc. > Example here: > > library(raster) > foo <- matrix(0,ncol=9,nrow=9) > foo[1:4,3] <- 1 > foo[5,4] <- 1 > foo[6:9,5] <- 1 > foo <- raster(foo) > plot(foo) > bar <- rasterToContour(foo,nlevels=1) > plot(bar) > > How can I get a continuous line in this example? Many > thanks, Andy > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Rasterize function
On Wed, 18 Apr 2018 at 00:08 Jeri Wisman <jwism...@odu.edu> wrote: > Hi all - > > I am working on a code trying to "rasterize" spatial polygons created by > kernelUD but somewhere in the code the values are getting lost and not > translating when I created a raster dateframe and try to tie in the polygon > lat/lon values into the raster frame. Attached is my code and data. > Hi, this is not really acceptable as a question to this list, because we have to go through your script to figure out what you are referring to. But, my guess is you want the "field" argument to ?rasterize: > If x is a Spatial*DataFrame, this can be the column name of the variable to be transferred. If missing, the attribute index is used (i.e. > numbers from 1 to the number of features). Your line r.polys <- rasterize(shp, r) will put the row number of shp into the appropriate pixel of r.polys, but you might want to set r.polys <- rasterize(shp, r, field = "somename") where "somename" is a member of names(shp). Just a guess. Cheers, Mike > Thanks! > *Jeri Wisman* | Masters Candidate > Old Dominion University > Department of Biological Sciences > Mills Godwin Building, Room 312 > Norfolk VA 23529 USA > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] netcdf data from Antarctica in rotated latlong - how to reproject to normal geographical coordinate system
[[alternative HTML version deleted]] > > > > ___ > > R-sig-Geo mailing list > > R-sig-Geo@r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > -- > Roger Bivand > Department of Economics, Norwegian School of Economics, > Helleveien 30, N-5045 Bergen, Norway. > voice: +47 55 95 93 55 <+47%2055%2095%2093%2055>; e-mail: > roger.biv...@nhh.no > http://orcid.org/-0003-2392-6140 > https://scholar.google.no/citations?user=AWeghB0J=en > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Spatial Downscaling in R
Try ClimDown package, otherwise more generally raster function disaggregate. Cheers On Thu, 15 Mar 2018, 06:53 Miluji Sb, <miluj...@gmail.com> wrote: > Dear all, > > Please forgive my inexperience with spatial downscaling. I am interested in > spatial downscaling of global temperature to grid cell. Is there a package > in R that can perform this function? > > Any help/guidance will be highly appreciated. > > Sincerely, > > Milu > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] [DKIM] Re: Interpolating snowfall values on a Digital Elevation Model [SEC=UNCLASSIFIED]
mailto:R-sig-Geo@r-project.org> > > > https://urlsand.esvalabs.com/?u=https%3A%2F%2Fstat.ethz.ch%< > https://urlsand.esvalabs.com/?u=https%3A%2F%2Fstat.ethz.ch%25> > > 2Fmailman%2Flistinfo%2Fr-sig-geo=52342f8a=f7d4a649=y=y > > > > > > -- > > > > Dr. Daniel Knitter > > CRC1266 -- Scales of Transformation > > Project A2 "Integrative Modeling of Socio-Environmental Dynamics" > > > > Christian-Albrechts-Universität zu Kiel Department of Geography > > Physical Geography -- Landscape Ecology and Geoinformation > > Ludewig-Meyn-Str. 14, room 109 D - 24118 Kiel, Germany > > > > Phone: +49 431 880 2941 <+49%20431%208802941> > > Email: knit...@geographie.uni-kiel.de knit...@geographie.uni-kiel.de> > > > > > > > > AVVISO IMPORTANTE: Questo messaggio di posta elettronica può contenere > > informazioni confidenziali, pertanto è destinato solo a persone > > autorizzate alla ricezione. I messaggi di posta elettronica per i > > client di Regione Marche possono contenere informazioni confidenziali e > con privilegi legali. > > Se non si è il destinatario specificato, non leggere, copiare, > > inoltrare o archiviare questo messaggio. Se si è ricevuto questo > > messaggio per errore, inoltrarlo al mittente ed eliminarlo > > completamente dal sistema del proprio computer. Ai sensi dell’art. 6 > > della DGR n. 1394/2008 si segnala che, in caso di necessità ed > > urgenza, la risposta al presente messaggio di posta elettronica può > essere visionata da persone estranee al destinatario. > > IMPORTANT NOTICE: This e-mail message is intended to be received only > > by persons entitled to receive the confidential information it may > contain. > > E-mail messages to clients of Regione Marche may contain information > > that is confidential and legally privileged. Please do not read, copy, > > forward, or store this message unless you are an intended recipient of > > it. If you have received this message in error, please forward it to > > the sender and delete it completely from your computer system. > > > > -- > > This message was scanned by Libra ESVA and is believed to be clean. > > > > ___ > > R-sig-Geo mailing list > > R-sig-Geo@r-project.org<mailto:R-sig-Geo@r-project.org> > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org<mailto:R-sig-Geo@r-project.org> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > Geoscience Australia Disclaimer: This e-mail (and files transmitted with > it) is intended only for the person or entity to which it is addressed. If > you are not the intended recipient, then you have received this e-mail by > mistake and any use, dissemination, forwarding, printing or copying of this > e-mail and its file attachments is prohibited. The security of emails > transmitted cannot be guaranteed; by forwarding or replying to this email, > you acknowledge and accept these risks. > > - > > > Geoscience Australia Disclaimer: This e-mail (and files transmitted with > it) is intended only for the person or entity to which it is addressed. If > you are not the intended recipient, then you have received this e-mail by > mistake and any use, dissemination, forwarding, printing or copying of this > e-mail and its file attachments is prohibited. The security of emails > transmitted cannot be guaranteed; by forwarding or replying to this email, > you acknowledge and accept these risks. > > - > > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] [R] How to label a polygon window (spatstat package)
Try text(0.5, 0.5, label = "?text") On Sat, 10 Feb 2018, 16:22 Mohammad Tanvir Ahamed via R-help, < r-h...@r-project.org> wrote: > Hi, > I want to label a polygon (circle or polygon) inside. > As for example code > > library(spatstat) > x <- runif(20) > y <- runif(20) > X <- ppp(x, y, window=disc(0.7)) > plot(X) > > Now I want to label that circle inside . Can some one please help me ? > Thanks. > > Regards. > Tanvir Ahamed > Stockholm, Sweden | mashra...@yahoo.com > > __ > r-h...@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] accessing spatial imagery via rgdal's web map tile service driver
I've put a fuller example together, thanks for the prompt, it's nice to
know how to do this:
http://rpubs.com/cyclemumner/358029
Note that raster is using underlying rgdal offset/output.dim functionality
to drive GDAL itself here, something that's not exposed well - raster makes
it seamless, but the underlying tools are a bit obscure and could be used
more widely.
The example uses row/col indexing via the extent(RasterLayer, numeric, ...)
method, you can otherwise use any extent in the native (north Polar
Stereographic) coordinate system. I don't know if raster interprets
aggregate(, fact = ) calls into translations for rgdal's output.dim, but it
would be powerful to be easy to do that.
Cheers, Mike.
On Fri, 9 Feb 2018 at 10:32 Michael Sumner <mdsum...@gmail.com> wrote:
> Oh, I had a typo in the xml_specification due to blithe copy/paste from
> email. It works fine as intended.
>
> I'll update and report back, this is a good example for fleshing out some
> issues.
>
> Cheers!
>
>
> On Fri, 9 Feb 2018 at 08:46 Fischbach, Anthony <afischb...@usgs.gov>
> wrote:
>
>> Mike, thank you for digging in on this. I have submitted my
>> sessionInfo() and comments to your git script regarding an error on the
>> raster function call. - Tony
>>
>> Anthony Fischbach, Wildlife Biologist
>> USGS Alaska Science Center Walrus Research Program
>> 4210 University Drive
>> <https://maps.google.com/?q=4210+University+DriveAnchorage,+AK+99508=gmail=g>
>> Anchorage, AK 99508
>> <https://maps.google.com/?q=4210+University+DriveAnchorage,+AK+99508=gmail=g>
>>
>> voice: (907) 786-7145
>>
>> http://alaska.usgs.gov/science/biology/walrus/
>>
>> On Thu, Feb 8, 2018 at 12:26 PM, Michael Sumner <mdsum...@gmail.com>
>> wrote:
>>
>>> Hi Anthony, what OS are you on? Can you share sessionInfo() or
>>> devtools::session_info()?
>>>
>>> But, it could be simpler, but it doesn't work on my system, I'll find
>>> out more:
>>>
>>> https://gist.github.com/mdsumner/ca332f9c5112e00edcc40fdcf2a4b968
>>>
>>> Cheers, Mike.
>>>
>>> On Thu, 8 Feb 2018 at 06:28 Fischbach, Anthony <afischb...@usgs.gov>
>>> wrote:
>>>
>>>> The GDAL tile web map service appears to require an xml specification
>>>> for
>>>> the TMS and a "window" specification indicating the bounds of the
>>>> region to
>>>> be accessed, specified in the projected coordinate system using the
>>>> -projwin flag. I am uncertain how to pass this -projwin and its bounds
>>>> to
>>>> the rgdal::readGDAL function. The code snippet below stages the
>>>> problem.
>>>> Please advise on how to acquire just the "window" of interest.
>>>>
>>>> #Tile Web Mapping Service to earthdata imagery via rgdal
>>>> library(rgdal) # r implementation of GDAL
>>>> require(raster) # r package for raster image manipulation, required for
>>>> raster capture of the readGDAL function result.
>>>> require(sp) # r package for spatial class objects, including CRS
>>>> #
>>>> prj.geo<-CRS('+proj=longlat +datum=WGS84') # Geographic projection
>>>> prj.Polar_NSIDC <- CRS("+init=epsg:3413")
>>>> # (+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +k=1 +x_0=0 +y_0=0
>>>> +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0)
>>>> prj.StudyArea <- CRS("+proj=laea +lat_0=70 +lon_0=-170 +x_0=0 +y_0=0
>>>> +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0") # study
>>>> area
>>>> projection
>>>> ##
>>>> xLimits <- sort(c(170, -135)) # define the study area polygon using
>>>> geographic coordinates (eastings WGS84)
>>>> yLimits <- sort(c(52, 75)) # (northings WGS84)
>>>> pts<-expand.grid(xLimits, yLimits)
>>>> names(pts)<-c('x','y')
>>>> coordinates(pts) <- ~x+y
>>>> proj4string(pts) <- prj.geo
>>>> pts.Polar_NSIDC<-spTransform(pts, prj.Polar_NSIDC)
>>>> bb <- round(bbox(pts.Polar_NSIDC)) #
>>>> #
>>>> yesterday <- Sys.Date() - 1
>>>> TileLevel <- 3
>>>> xml_specification <-
>>>> paste0('
>>>>
>>>>
>>>>
>>>> https://gibs.earthdata.nasa.gov/wmts/epsg3413/best/MODIS_Aqua_CorrectedReflectance_Bands721/default/
>>>> ',
>>>> format(yesterday, "%Y-%m-%d"),
>>>> '/250m/${z
Re: [R-sig-Geo] accessing spatial imagery via rgdal's web map tile service driver
Oh, I had a typo in the xml_specification due to blithe copy/paste from
email. It works fine as intended.
I'll update and report back, this is a good example for fleshing out some
issues.
Cheers!
On Fri, 9 Feb 2018 at 08:46 Fischbach, Anthony <afischb...@usgs.gov> wrote:
> Mike, thank you for digging in on this. I have submitted my sessionInfo()
> and comments to your git script regarding an error on the raster function
> call. - Tony
>
> Anthony Fischbach, Wildlife Biologist
> USGS Alaska Science Center Walrus Research Program
> 4210 University Drive
> <https://maps.google.com/?q=4210+University+DriveAnchorage,+AK+99508=gmail=g>
> Anchorage, AK 99508
> <https://maps.google.com/?q=4210+University+DriveAnchorage,+AK+99508=gmail=g>
>
> voice: (907) 786-7145
>
> http://alaska.usgs.gov/science/biology/walrus/
>
> On Thu, Feb 8, 2018 at 12:26 PM, Michael Sumner <mdsum...@gmail.com>
> wrote:
>
>> Hi Anthony, what OS are you on? Can you share sessionInfo() or
>> devtools::session_info()?
>>
>> But, it could be simpler, but it doesn't work on my system, I'll find
>> out more:
>>
>> https://gist.github.com/mdsumner/ca332f9c5112e00edcc40fdcf2a4b968
>>
>> Cheers, Mike.
>>
>> On Thu, 8 Feb 2018 at 06:28 Fischbach, Anthony <afischb...@usgs.gov>
>> wrote:
>>
>>> The GDAL tile web map service appears to require an xml specification for
>>> the TMS and a "window" specification indicating the bounds of the region
>>> to
>>> be accessed, specified in the projected coordinate system using the
>>> -projwin flag. I am uncertain how to pass this -projwin and its bounds to
>>> the rgdal::readGDAL function. The code snippet below stages the problem.
>>> Please advise on how to acquire just the "window" of interest.
>>>
>>> #Tile Web Mapping Service to earthdata imagery via rgdal
>>> library(rgdal) # r implementation of GDAL
>>> require(raster) # r package for raster image manipulation, required for
>>> raster capture of the readGDAL function result.
>>> require(sp) # r package for spatial class objects, including CRS
>>> #
>>> prj.geo<-CRS('+proj=longlat +datum=WGS84') # Geographic projection
>>> prj.Polar_NSIDC <- CRS("+init=epsg:3413")
>>> # (+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +k=1 +x_0=0 +y_0=0
>>> +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0)
>>> prj.StudyArea <- CRS("+proj=laea +lat_0=70 +lon_0=-170 +x_0=0 +y_0=0
>>> +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0") # study area
>>> projection
>>> ##
>>> xLimits <- sort(c(170, -135)) # define the study area polygon using
>>> geographic coordinates (eastings WGS84)
>>> yLimits <- sort(c(52, 75)) # (northings WGS84)
>>> pts<-expand.grid(xLimits, yLimits)
>>> names(pts)<-c('x','y')
>>> coordinates(pts) <- ~x+y
>>> proj4string(pts) <- prj.geo
>>> pts.Polar_NSIDC<-spTransform(pts, prj.Polar_NSIDC)
>>> bb <- round(bbox(pts.Polar_NSIDC)) #
>>> #
>>> yesterday <- Sys.Date() - 1
>>> TileLevel <- 3
>>> xml_specification <-
>>> paste0('
>>>
>>>
>>>
>>> https://gibs.earthdata.nasa.gov/wmts/epsg3413/best/MODIS_Aqua_CorrectedReflectance_Bands721/default/
>>> ',
>>> format(yesterday, "%Y-%m-%d"),
>>> '/250m/${z}/${y}/${x}.jpg
>>>
>>>
>>> -4194304
>>> 4194304
>>> 4194304
>>> -4194304
>>> ', TileLevel, '
>>> 2
>>> 2
>>> top
>>>
>>> EPSG:3413
>>> 512
>>> 512
>>> 3
>>>
>>> ')
>>> #
>>> cat(xml_specification)
>>> GDALinfo(fname = xml_specification)
>>> (projWinParameters = paste('-projwin', bb[1, 1], bb[2,2], bb[1,2],
>>> bb[2,1], sep = " "))
>>> aqua <- readGDAL(fname = xml_specification) # , projWinParameters) ?? how
>>> to implement the projwin constraints??
>>> (aqua_stack <- stack(aqua))
>>> #
>>> plotRGB(aqua_stack) # plots the acquired image in RGB !! problem is that
>>> the entire WMS domain is acquired.
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> ___
>>> R-sig-Geo mailing list
>>> R-sig-Geo@r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>> --
>> Dr. Michael Sumner
>> Software and Database Engineer
>> Australian Antarctic Division
>> 203 Channel Highway
>> <https://maps.google.com/?q=203+Channel+Highway+Kingston+Tasmania+7050+Australia=gmail=g>
>> Kingston Tasmania 7050 Australia
>> <https://maps.google.com/?q=203+Channel+Highway+Kingston+Tasmania+7050+Australia=gmail=g>
>>
>>
> --
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] accessing spatial imagery via rgdal's web map tile service driver
Hi Anthony, what OS are you on? Can you share sessionInfo() or
devtools::session_info()?
But, it could be simpler, but it doesn't work on my system, I'll find out
more:
https://gist.github.com/mdsumner/ca332f9c5112e00edcc40fdcf2a4b968
Cheers, Mike.
On Thu, 8 Feb 2018 at 06:28 Fischbach, Anthony <afischb...@usgs.gov> wrote:
> The GDAL tile web map service appears to require an xml specification for
> the TMS and a "window" specification indicating the bounds of the region to
> be accessed, specified in the projected coordinate system using the
> -projwin flag. I am uncertain how to pass this -projwin and its bounds to
> the rgdal::readGDAL function. The code snippet below stages the problem.
> Please advise on how to acquire just the "window" of interest.
>
> #Tile Web Mapping Service to earthdata imagery via rgdal
> library(rgdal) # r implementation of GDAL
> require(raster) # r package for raster image manipulation, required for
> raster capture of the readGDAL function result.
> require(sp) # r package for spatial class objects, including CRS
> #
> prj.geo<-CRS('+proj=longlat +datum=WGS84') # Geographic projection
> prj.Polar_NSIDC <- CRS("+init=epsg:3413")
> # (+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +k=1 +x_0=0 +y_0=0
> +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0)
> prj.StudyArea <- CRS("+proj=laea +lat_0=70 +lon_0=-170 +x_0=0 +y_0=0
> +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0") # study area
> projection
> ##
> xLimits <- sort(c(170, -135)) # define the study area polygon using
> geographic coordinates (eastings WGS84)
> yLimits <- sort(c(52, 75)) # (northings WGS84)
> pts<-expand.grid(xLimits, yLimits)
> names(pts)<-c('x','y')
> coordinates(pts) <- ~x+y
> proj4string(pts) <- prj.geo
> pts.Polar_NSIDC<-spTransform(pts, prj.Polar_NSIDC)
> bb <- round(bbox(pts.Polar_NSIDC)) #
> #
> yesterday <- Sys.Date() - 1
> TileLevel <- 3
> xml_specification <-
> paste0('
>
>
>
> https://gibs.earthdata.nasa.gov/wmts/epsg3413/best/MODIS_Aqua_CorrectedReflectance_Bands721/default/
> ',
> format(yesterday, "%Y-%m-%d"),
> '/250m/${z}/${y}/${x}.jpg
>
>
> -4194304
> 4194304
> 4194304
> -4194304
> ', TileLevel, '
> 2
> 2
> top
>
> EPSG:3413
> 512
> 512
> 3
>
> ')
> #
> cat(xml_specification)
> GDALinfo(fname = xml_specification)
> (projWinParameters = paste('-projwin', bb[1, 1], bb[2,2], bb[1,2],
> bb[2,1], sep = " "))
> aqua <- readGDAL(fname = xml_specification) # , projWinParameters) ?? how
> to implement the projwin constraints??
> (aqua_stack <- stack(aqua))
> #
> plotRGB(aqua_stack) # plots the acquired image in RGB !! problem is that
> the entire WMS domain is acquired.
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Splitting spatial lines at self intersections (line crossings)
) > > > > plot(L) > > > > PSP <- as.psp.SpatialLines(L) > > > > int <- selfcrossing.psp(PSP) > > > > plot(int,add=TRUE,col="red") # identifies more than just the crossings > > > > > > > > > > [[alternative HTML version deleted]] > > > > ___ > > R-sig-Geo mailing list > > R-sig-Geo@r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Splitting spatial lines at self intersections (line crossings)
Ege: I don't believe that this code is enough to get coercion between sp and spatstat: pts <- cbind(c(120:123,121,125),c(100,100,104,102,99,98)) pt2 <- cbind(c(124,124,123,118,124,125),c(100,97,100,104,106,110)) library(sp) library(spatstat) L1 <- SpatialLines(list(Lines(list(Line(pts)), "X"))) L2 <- SpatialLines(list(Lines(list(Line(pt2)), "X"))) l1 <- as.psp(L1) l2 <- as.psp(L2) int <- crossing.psp(l1, l2) Are we relying on some other code, or perhaps a specific version of spatstat or its family? (I know ways to convert from Spatial to psp, but I was taken by this apparent lack of infrastructure - and I'm very keen on seeing more bridges between these rich worlds). Cheers, Mike On Fri, 26 Jan 2018 at 17:48 Ege Rubak <ru...@math.aau.dk> wrote: > The `psp` class in spatstat consists of individual line segments and > `selfcrossing.psp` checks whether each individual line intersects one of > the other lines, which happens at all the points in your plot. > > If instead you treat each of the two line sequences as a `psp` you can > check find the crossings of the two `psp` objects. I.e., continuing your > code (with `spatstat` and `maptools` loaded): > > L1 <- SpatialLines(list(Lines(list(Line(pts)), "X"))) > L2 <- SpatialLines(list(Lines(list(Line(pt2)), "X"))) > l1 <- as.psp(L1) > l2 <- as.psp(L2) > int <- crossing.psp(l1, l2) > > This gives you the four intersections between the lines. I don't know of > a simple way to do the next task in `spatstat`. A useful function if you > are going to try to write some code is `test.crossing.psp` which gives > you a logical matrix indicating which segments cross. > > Cheers, > Ege > > On 01/25/2018 11:26 PM, Glenn Stauffer wrote: > > I have a Spatial Lines object I would like to split at every point where > the > > line self-intersects (crosses or touches itself), or anywhere lines touch > > each other, if there are multiple lines. > > > > I've tried using spatstat to convert the SpatialLines to a psp object and > > then use the selfcrossing.psp function to find the intersection points > (see > > example below). But that seems to identify all the nodes, not just the > > points where the line crosses. Also, even if it identified only the > > crossings (which is what I want), I am not sure how to perform the next > step > > and split the lines at those points. > > > > So, 1) I need to identify the crossings/touches, and 2) split the lines > at > > those points. > > > > > > > > Essentially, what I am looking for is an R equivalent to the planarize > > function in ArgGIS. Is there a relatively easy way to do this in R? > > > > > > > > Thanks, > > > > Glenn > > > > > > > > Here is an example of what I have tried. > > > > > > > > > > > > pts <- cbind(c(120:123,121,125),c(100,100,104,102,99,98)) > > > > pt2 <- cbind(c(124,124,123,118,124,125),c(100,97,100,104,106,110)) > > > > projstr <- "+init=epsg:3071" # make everything in meters > > > > L <- SpatialLines(list(Lines(list(Line(pts),Line(pt2)),"X")),proj4string > = > > CRS(projstr)) > > > > plot(L) > > > > PSP <- as.psp.SpatialLines(L) > > > > int <- selfcrossing.psp(PSP) > > > > plot(int,add=TRUE,col="red") # identifies more than just the crossings > > > > > > > > > > [[alternative HTML version deleted]] > > > > ___ > > R-sig-Geo mailing list > > R-sig-Geo@r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] make a raster from Aquarius files
Ugh, and you must think I'm crazy given that it's clearly HDF5 not HDF4 as
per my opening line (the file naming is more the latter). I'll stop now.
Hope it's useful.
Cheers, Mike.
On Sat, 2 Dec 2017 at 07:44 Michael Sumner <mdsum...@gmail.com> wrote:
> Ah thanks, and sorry I didn't think any of those old HDF4 files were still
> there!
>
> This is HDF5 with subdatasets, from gdalinfo:
>
> Subdatasets:
>
> SUBDATASET_1_NAME=HDF5:"Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg"://l3m_data
> SUBDATASET_1_DESC=[180x360] //l3m_data (32-bit floating-point)
>
> SUBDATASET_2_NAME=HDF5:"Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg"://palette
> SUBDATASET_2_DESC=[3x256] //palette (8-bit unsigned character)
>
> so you need to use the subdataset string directly i.e.
>
> raster("HDF5:"Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg")
>
> or spit them out to seperate files at the command line first e.g.
>
> gdal_translate Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg
> Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.tif -sds
>
> and this assumes a suitable system with HDF5 etc. etc.
>
> rgdal on Windows now has HDF5 drivers so should work there too, and Ben's
> method also.
>
> Cheers, Mike.
>
> On Sat, 2 Dec 2017 at 07:27 Antonio Silva <aolinto@gmail.com> wrote:
>
>> Thanks Ben and Michael for the attention
>>
>> The file I'm trying to rasterize can be downloaded at
>> https://oceancolor.gsfc.nasa.gov/cgi/l3
>>
>> I selected the Aquarius sea surface salinity smoothed file from June 2015
>> (SMI HDF).
>>
>> As Ben pointed it is also available from OBPG Nasa Ocean Color site at
>> https://oceandata.sci.gsfc.nasa.gov/Aquarius/Mapped/Monthly/1deg/V5.0_SSS/
>>
>> or directly from
>> https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.bz2
>>
>> Thanks again
>>
>> Antonio Olinto Avila da Silva
>> Fisheries Institute
>> São Paulo, Brasil
>>
>>
>> 2017-12-01 12:35 GMT-02:00 Ben Tupper <btup...@bigelow.org>:
>>
>>> Hi,
>>>
>>> Those Aquarius files look quite different from others I have used from
>>> OBPG (mostly MODISA and SeaWiFS).
>>>
>>> As a short-cut alternative, you could read all of the values into a
>>> matrix and make a global raster from that. An example is shown below. You
>>> can also subset the extraction as you have shown, but it might be easier to
>>> subset after making the raster using raster::crop()
>>>
>>> CHeers,
>>> Ben
>>>
>>>
>>> ### START
>>>
>>> library(raster)
>>> library(ncdf4)
>>> library(rasterVis)
>>>
>>> filename = "Q20151522015181.L3m_MO_SCIA_V5.0_SSS_1deg"
>>>
>>> nc = ncdf4::nc_open(filename)
>>> m = ncdf4::ncvar_get(nc, 'l3m_data')
>>> ncdf4::nc_close(nc)
>>> r = raster::raster(t(m))
>>> rasterVis::levelplot(r)
>>>
>>> ### END
>>>
>>> > On Dec 1, 2017, at 6:24 AM, Michael Sumner <mdsum...@gmail.com> wrote:
>>> >
>>> > That file name does not correspond to the standard patterns used by the
>>> > oceancolor site. All the L3m products from there are now (NetCDF 4.0)
>>> .nc
>>> > and so will work fine with raster/ncdf4. (Some years ago they were
>>> HDF4 -
>>> > without an extension, as the shortcuts in the image thumbnails hints
>>> > (SMI/HDF and BIN/HDF - SMI/L3m standard mapped image in your case).
>>> >
>>> > I think you've got some other provider's version of a file, but
>>> there's not
>>> > enough information here to know where you got it or what form it's in.
>>> I'm
>>> > happy to look if you can point us to the source of
>>> > Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.
>>> >
>>> > But otherwise, can you share with us the output of
>>> >
>>> > nc.data<-nc_open("Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg")
>>> > print(nc.data)
>>> >
>>> > and if you're on a suitable system with HDF4 support a gdalinfo output
>>> of
>>> > the file would be useful too.
>>> >
>>> > Given that you can read it with ncdf4, and if it actually is NetCDF4
>>> (not
>>> > HDF4 or something else) you might help raster work with it by renaming
>>> it
>>> > to "Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.nc" since (unlike GDAL
>>> and
>>
Re: [R-sig-Geo] make a raster from Aquarius files
Ah thanks, and sorry I didn't think any of those old HDF4 files were still
there!
This is HDF5 with subdatasets, from gdalinfo:
Subdatasets:
SUBDATASET_1_NAME=HDF5:"Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg"://l3m_data
SUBDATASET_1_DESC=[180x360] //l3m_data (32-bit floating-point)
SUBDATASET_2_NAME=HDF5:"Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg"://palette
SUBDATASET_2_DESC=[3x256] //palette (8-bit unsigned character)
so you need to use the subdataset string directly i.e.
raster("HDF5:"Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg")
or spit them out to seperate files at the command line first e.g.
gdal_translate Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg
Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.tif -sds
and this assumes a suitable system with HDF5 etc. etc.
rgdal on Windows now has HDF5 drivers so should work there too, and Ben's
method also.
Cheers, Mike.
On Sat, 2 Dec 2017 at 07:27 Antonio Silva <aolinto@gmail.com> wrote:
> Thanks Ben and Michael for the attention
>
> The file I'm trying to rasterize can be downloaded at
> https://oceancolor.gsfc.nasa.gov/cgi/l3
>
> I selected the Aquarius sea surface salinity smoothed file from June 2015
> (SMI HDF).
>
> As Ben pointed it is also available from OBPG Nasa Ocean Color site at
> https://oceandata.sci.gsfc.nasa.gov/Aquarius/Mapped/Monthly/1deg/V5.0_SSS/
>
> or directly from
> https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.bz2
>
> Thanks again
>
> Antonio Olinto Avila da Silva
> Fisheries Institute
> São Paulo, Brasil
>
>
> 2017-12-01 12:35 GMT-02:00 Ben Tupper <btup...@bigelow.org>:
>
>> Hi,
>>
>> Those Aquarius files look quite different from others I have used from
>> OBPG (mostly MODISA and SeaWiFS).
>>
>> As a short-cut alternative, you could read all of the values into a
>> matrix and make a global raster from that. An example is shown below. You
>> can also subset the extraction as you have shown, but it might be easier to
>> subset after making the raster using raster::crop()
>>
>> CHeers,
>> Ben
>>
>>
>> ### START
>>
>> library(raster)
>> library(ncdf4)
>> library(rasterVis)
>>
>> filename = "Q20151522015181.L3m_MO_SCIA_V5.0_SSS_1deg"
>>
>> nc = ncdf4::nc_open(filename)
>> m = ncdf4::ncvar_get(nc, 'l3m_data')
>> ncdf4::nc_close(nc)
>> r = raster::raster(t(m))
>> rasterVis::levelplot(r)
>>
>> ### END
>>
>> > On Dec 1, 2017, at 6:24 AM, Michael Sumner <mdsum...@gmail.com> wrote:
>> >
>> > That file name does not correspond to the standard patterns used by the
>> > oceancolor site. All the L3m products from there are now (NetCDF 4.0)
>> .nc
>> > and so will work fine with raster/ncdf4. (Some years ago they were
>> HDF4 -
>> > without an extension, as the shortcuts in the image thumbnails hints
>> > (SMI/HDF and BIN/HDF - SMI/L3m standard mapped image in your case).
>> >
>> > I think you've got some other provider's version of a file, but there's
>> not
>> > enough information here to know where you got it or what form it's in.
>> I'm
>> > happy to look if you can point us to the source of
>> > Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.
>> >
>> > But otherwise, can you share with us the output of
>> >
>> > nc.data<-nc_open("Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg")
>> > print(nc.data)
>> >
>> > and if you're on a suitable system with HDF4 support a gdalinfo output
>> of
>> > the file would be useful too.
>> >
>> > Given that you can read it with ncdf4, and if it actually is NetCDF4
>> (not
>> > HDF4 or something else) you might help raster work with it by renaming
>> it
>> > to "Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.nc" since (unlike GDAL
>> and
>> > the NetCDF lib itself) raster uses explicit extension to dispatch to
>> > different format logic code, though it ultimately sends it down to
>> rgdal to
>> > deal with if it can't recognize it - which is why I'm surprised you
>> can't
>> > get it to work and ( I'm guessing wildly now):
>> >
>> > Do you not have rgdal installed?
>> >
>> >
>> > What system are you on? Please use sessionInfo() to share details.
>> >
>> > Cheers, Mike.
>> > On Fri, 1 Dec 2017, 06:14 Antonio Silva, <aolinto@gmail.com> wrote:
>> >
>> >> Hello
>> >>
>> >> Some time ago I prepared scripts t
Re: [R-sig-Geo] make a raster from Aquarius files
That file name does not correspond to the standard patterns used by the
oceancolor site. All the L3m products from there are now (NetCDF 4.0) .nc
and so will work fine with raster/ncdf4. (Some years ago they were HDF4 -
without an extension, as the shortcuts in the image thumbnails hints
(SMI/HDF and BIN/HDF - SMI/L3m standard mapped image in your case).
I think you've got some other provider's version of a file, but there's not
enough information here to know where you got it or what form it's in. I'm
happy to look if you can point us to the source of
Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.
But otherwise, can you share with us the output of
nc.data<-nc_open("Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg")
print(nc.data)
and if you're on a suitable system with HDF4 support a gdalinfo output of
the file would be useful too.
Given that you can read it with ncdf4, and if it actually is NetCDF4 (not
HDF4 or something else) you might help raster work with it by renaming it
to "Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg.nc" since (unlike GDAL and
the NetCDF lib itself) raster uses explicit extension to dispatch to
different format logic code, though it ultimately sends it down to rgdal to
deal with if it can't recognize it - which is why I'm surprised you can't
get it to work and ( I'm guessing wildly now):
Do you not have rgdal installed?
What system are you on? Please use sessionInfo() to share details.
Cheers, Mike.
On Fri, 1 Dec 2017, 06:14 Antonio Silva, <aolinto@gmail.com> wrote:
> Hello
>
> Some time ago I prepared scripts to extract temperature data from Modis
> Aqua files. It can be found at https://gist.github.com/aolinto
>
> HDF files can be downloaded at https://oceancolor.gsfc.nasa.gov/cgi/l3
>
> I got the Aquarius sea surface salinity smoothed file from June 2015.
>
> I could open and read the file:
>
> library(ncdf4)
> library(raster)
>
> nc.data<-nc_open("Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg")
> print(nc.data)
> dim(ncvar_get(nc.data,"l3m_data"))
> ncvar_get(nc.data,"l3m_data")[c(110:160),c(110:117)]
>
> But I could not prepare a raster from it. I tryed many things as:
>
> rst.data <-
> raster("Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg",varname="l3m_data")
> Error in .local(.Object, ...) :
> `AQUARIUS/Q20151522015181.L3m_MO_SCISM_V5.0_SSS_1deg' not recognised as a
> supported file format.
>
> Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer",
> :
> Cannot create a RasterLayer object from this file.
>
> and variations with band and layer.
>
> I would greatly appreciate any suggestions to solve this issue.
>
> Thanks
>
> --
> Antônio Olinto Ávila da Silva
> Fisheries Institute
> São Paulo, Brasil
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Is the raster package still receiving updates?
> weeks. > > > > > > > > > > > > > > > On 10/21/2017 10:41 AM, Roger Bivand wrote: > > > > > > Yes, this sounds fruitful. For now, I suggest that those wishing > to > > > > > enhance raster do so in an additional package enhancing raster and > > > > > depending on it. Immediately, we should help Robert resolve the > notes > > > in > > > > > the tests, they do not affect functionality now, but deserve > > attention > > > to > > > > > bring the package into full CRAN compliance. I can help with this. > > > > > > > > > > > > Roger > > > > > > > > > > > > Roger Bivand > > > > > > Norwegian School of Economics > > > > > > Bergen, Norway > > > > > > > > > > > > > > > > > > > > > > > > Fra: Chris Reudenbach > > > > > > Sendt: lørdag 21. oktober, 09.01 > > > > > > Emne: Re: [R-sig-Geo] Is the raster package still receiving > > updates? > > > > > > Til: r-sig-geo@r-project.org > > > > > > > > > > > > > > > > > > Many of us rely on the raster package as a crucial basis for > their > > > > work. > > > > > Big thank to Robert. Nevertheless to contribute needs to focus > needs > > > and > > > > > ressources. Maybe an information exchange/discussion with Robert > > about > > > > > options such as if/how the interested community can support would > be > > > more > > > > > productive. cheers Chris Am 21.10.2017 um 06:52 schrieb Roger > > Bivand: > > > > > The > > > > > CRAN raster package has not been abandoned, if it had, it would > have > > > been > > > > > reclassified as orphaned. Its test results have notes but no > warnings > > > or > > > > > errors. It would be orphaned if the maintainer did not respond to > > CRAN > > > > > requests to resolve test errors. Consequently, you are suggesting a > > > fork, > > > > > and should rename any package. Only the maintainer may update > > existing > > > > CRAN > > > > > packages. I would not think that forking an existing CRAN package > is > > > the > > > > > most productive way of contributing to the community. The package > has > > > > many > > > > > reverse dependencies, so setting up a test framework to find > > backwards > > > > > incompatibilities would be crucial. > > Of course, all > contributions > > > are > > > > > welcome. > > Roger > > Roger Bivand > Norwegian School of > Economics > > > > > > Bergen, Norway > > > > Fra: Michael Sumner > Sendt: lørdag 21. > > oktober, > > > > > 06.28 > Emne: Re: [R-sig-Geo] Is the raster package still receiving > > > > > updates? > Til: Hodgess, Erin > Kopi: R-sig-geo Mailing List > > > > > > Great! > > > > > I'm keen, but haven't done the work needed to flesh this out > > properly: > > > > > https://github.com/mdsumner/raster-rforge/issues I have not > checked > > if > > > > > there's been any commits to r-forge since I cloned this. Cheers, > Mike > > > On > > > > > Sat, 21 Oct 2017, 12:58 Hodgess, Erin, wrote: > I could take a swat > > at > > > > it, > > > > > if you wish. Please do send me the list of > fixes. I could start > in > > > > about > > > > > a week, if that would work. > Sincerely, > Erin > > > Erin M. > > Hodgess > > > > > > Associate Professor > Department of Mathematics and Statistics > > > > > University > > > > > of Houston - Downtown > mailto: hodge...@uhd.edu > > > > > > > > From: R-sig-Geo on > behalf > > > of > > > > > Michael > Sumner > Sent: Friday, October 20, 2017 8:05 PM > To: > > Thiago > > > V. > > > > > dos Santos > Cc: R-sig-geo Mailing List > Subject: Re: [R-sig-Geo] > Is > > > the > > > > > raster package still receiving updates? > > No, it's effectively > > > > abandoned. > > > > > No concrete plans known. &g
Re: [R-sig-Geo] Plotting GPS Coordinates on a Polar Projection Using rgdal
(thanks Clark! oce is excellent)
On Tue, 24 Oct 2017, 23:29 clark richards, <clark.richa...@gmail.com> wrote:
> I would definitely defer to Michael and the other true "Geo" folks on this
> list**, but if you're ok with base graphics (and a much less technical
> interface) you could accomplish what you want using the `oce` package:
>
> library(oce)
> data(coastlineWorld)
> circ <- list(longitude=seq(-180, 180), latitude=-rep(60, 361)) # fake
> circumnavigation
> ## use mapPlot to make a polar stereographic plot of antarctica
> mapPlot(coastlineWorld, projection="+proj=stere +lat_0=-90",
> latitudelim=c(-90, -45), longitudelim = c(-180, 180), col='grey')
> ## now add the circumnavigation with mapPoints
> mapPoints(circ)
>
> Hope that helps!
> Clark
>
> **as you can see mapPlot is using proj/gdal interface under the hood, but
> being oceanographers and only partially understanding the details of the
> transforms, datums, projections, etc, we tried to write an interface that
> will allow us to make pretty maps without getting too far into the details.
>
>
> Message: 4
>> Date: Mon, 23 Oct 2017 22:53:22 +
>> From: "Mortimer, Linsey Anne" <l.mortimer...@aberdeen.ac.uk>
>> To: Michael Sumner <mdsum...@gmail.com>
>> Cc: "r-sig-geo@r-project.org" <r-sig-geo@r-project.org>
>
>
>> Subject: Re: [R-sig-Geo] Plotting GPS Coordinates on a Polar
>> Projection Using rgdal
>>
> Message-ID:
>> <
>> vi1pr0101mb23173329c1e17653f09ecc8086...@vi1pr0101mb2317.eurprd01.prod.exchangelabs.com
>> >
>>
>> Content-Type: text/plain; charset="Windows-1252"
>>
>> This was a big help, thank you. The coordinates are plotted and look good
>> except some of my GPS coordinates are outside of the plot - I should have
>> check this beforehand. Is there a way to produce a plot using spbabel
>> similar to this:?
>
>
>>
>>
>> library(ggplot2)
>> world <- map_data("world")
>> worldmap <- ggplot(world, aes(x=long, y=lat, group=group)) +
>>
> ? geom_path() +
>> ? scale_y_continuous(breaks=(-2:2) * 30) +
>> ? scale_x_continuous(breaks=(-4:4) * 45)
>
>
>> worldmap + coord_map("ortho", orientation=c(-90, 0, 0))
>>
>>
>> or would it be better to simply use ggplot for this?
>>
>>
>> Many thanks,
>>
>> Linsey
>>
>>
>> From: Michael Sumner <mdsum...@gmail.com>
>> Sent: 23 October 2017 21:35
>> To: Mortimer, Linsey Anne
>> Cc: r-sig-geo@r-project.org
>> Subject: Re: [R-sig-Geo] Plotting GPS Coordinates on a Polar Projection
>> Using rgdal
>>
> ?
>
>
>>
>>
>>
>> On Tue, 24 Oct 2017 at 03:42 Mortimer, Linsey Anne <
>> l.mortimer...@aberdeen.ac.uk> wrote:
>> Hello,
>>
>> I am new to R and attempting to plot a map of Antarctica with the GPS
>> coordinates from a circumpolar navigation of the Southern Ocean. I have
>> searched the archives and rgdal-related manuals/blogs/papers to find
>> variations of the code below. I have two issues: firstly, removing the
>> vertical line at -90 degrees on the plot of Antarctica. Secondly, adding
>> the coordinates to this plot using points(). I have tried transforming the
>> coordinates of the points to fit the projection but every time the output
>> is a single point plotted at the end of the vertical line on the
>> Antarctica map, a separate plot of points in a world map projection
>> (stretched) rather than polar projection, or simply an error about coercing
>> an S4 class to a vector. Any advice on how to correct the code or a point
>> in the direction of a guide to this kind of mapping and spatial analysis
>> would be of great help.
>>
>>
>>
>>
>>
>>
>>
>>
>> You can remove the dummy seam at the south pole by decomposing to raw
>> coordinates and reconstructing with spbabel:?
>
>
>>
>>
>>
>> data("wrld_simpl", package = "maptools")
>> crs <- sp::proj4string(wrld_simpl)
>> antarctica0 <- wrld_simpl[wrld_simpl$NAME == "Antarctica", ]
>> library(spbabel)
>> library(dplyr)
>>
> ant <- spbabel::sptable(antarctica0) %>%?
>> ? dplyr::filter(y_ > -90) %>%?
>> ? spbabel::sp(crs = crs)
>
>
>>
>>
>>
>>
>>
>> pr <- "+proj=laea +lat_0=-90 +ellps=WGS84 +datum=WGS84 +no_defs
>> +towgs84=0,0,0"
>>
>>
>>
>> antarctica <- sp::spTransform(ant, sp:
Re: [R-sig-Geo] Plotting GPS Coordinates on a Polar Projection Using rgdal
lso really painful to use and is in
desperate need of a user-friendly update.) There's no real stand out good
approach for this general problem, so feel free to ask more specifics.
There are a huge number of disparate tracking-packages listed on CRAN so
you might explore them for your needs:
https://cran.r-project.org/web/views/SpatioTemporal.html
(sere under "Moving objects, trajectories")
Cheers, Mike.
> ## Also tried to transform the coordinates but receive an error
>
> >proj4string(GPS) <- CRS("+proj=laea +lat_0=-90 +ellps=WGS84 +datum=WGS84
> +no_defs +towgs84=0,0,0")
>
> > gps.ant <- spTransform(GPS, CRS(antarctica.laea))
>
> Error in CRS(antarctica.laea) :
>
> no method for coercing this S4 class to a vector
>
> In addition: Warning message:
>
> In is.na(projargs) : is.na() applied to non-(list or vector) of type 'S4'
>
> > sessionInfo()
>
> R version 3.4.2 (2017-09-28)
>
> Platform: x86_64-w64-mingw32/x64 (64-bit)
>
> Running under: Windows >= 8 x64 (build 9200)
>
>
>
> Matrix products: default
>
>
>
> locale:
>
> [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United
> Kingdom.1252
>
> [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
>
> [5] LC_TIME=English_United Kingdom.1252
>
>
>
> attached base packages:
>
> [1] stats graphics grDevices utils datasets methods base
>
>
>
> other attached packages:
>
> [1] maptools_0.9-2 rgdal_1.2-13 sp_1.2-5
>
>
>
> loaded via a namespace (and not attached):
>
> [1] compiler_3.4.2 tools_3.4.2 foreign_0.8-69 grid_3.4.2
> lattice_0.20-35
>
>
>
> Many thanks,
>
>
>
> Linsey Mortimer
>
>
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Is the raster package still receiving updates?
No, it's effectively abandoned. No concrete plans known. I'm interested to keep it going and have kept a list of fixes needed, but not sure when or if I'll get to it. I'd support any efforts in this, I'll rely on raster for many years still. Cheers, Mike On Sat, 21 Oct 2017, 02:05 Thiago V. dos Santos via R-sig-Geo, < r-sig-geo@r-project.org> wrote: > Dear list, > > I realized that the latest version of the raster package was released on > CRAN over a year ago. > > I also noticed that Robert's participation in this list has become rather > scarce. > > I was just wondering whether the raster package is still receiving updates? > > Greetings, > -- Thiago V. dos Santos > > Postdoctoral Research Fellow > Department of Climate and Space Science and Engineering > University of Michigan > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Optimized rasterOptions() for a (virtually) infinite RAM machine
See Noam's post here for good advice, avoiding temp files is very important in your case: https://discuss.ropensci.org/t/how-to-avoid-space-hogging-raster-tempfiles/864 For using data frames raster's cell index abstraction is super powerful and sadly underused, see tabularaster for some easy approaches. Don't store coordinates explicitly, for example, at least not until you are ready to plot with ggplot2. Finally, raster us generally great with NetCDF if you let it control the task, but different situations and file setups can really matter so feel free to provide details if things aren't working well. Generally using raster can easily match the best you can achieve with the NetCDF API but lots of specifics can bite. Raster is generally not able to efficiently crop space and time together, for example but functions mapped to slice extraction can he used to hone performance. Cheers, Mike On Sat, 23 Sep 2017, 14:02 Thiago V. dos Santos via R-sig-Geo < r-sig-geo@r-project.org> wrote: > Dear all, > I am using the raster package to process a total of 32 daily climate files > supplied as netcdf files. Each file is a raster brick with 100 rows x 95 > cols x 54750 time slices and weighs about 1.5 GB. > Essentially, all the processing I am performing on each netcdf file is: > a) to subset a specific date rangeb) to extract values using points > After that, I just convert the extracted data to data.tables and keep > working in that format. > Since I extract data for about 450 points, and append all the data in a > huge data.table, I need to use a computer with as much RAM as possible. > I ended up using a spot instance on Amazon EC2. Using an instance with 32 > cores and 244GB of RAM will cost me around $0.30/hour. > Since I will be charged per hour, I need to optimize my code to get my > results as fast as possible. > I don't even copy my data to the instance's hard disk; I send the files > directly to the ram disk (/dev/shm). Even using 48GB of ram disk to store > the files, I'll still have 196GB of RAM. > Under the scenario of having virtually infinite RAM memory, what would be > the best rasterOptions() to make sure I am processing all my rasters in > memory? Any other tips to benefit from such a large amount of RAM? > Thanks, -- Thiago V. dos Santos > Postdoctoral Research FellowDepartment of Climate and Space Science and > EngineeringUniversity of Michigan > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] temp directory not actually reset by rasterOptions()
FYI It's also important to set the max size for raster, default is way too low: https://discuss.ropensci.org/t/how-to-avoid-space-hogging-raster-tempfiles/864 On Thu, 21 Sep 2017, 01:25 Agustin Lobo <alobolis...@gmail.com> wrote: > I have problems with an small /tmp partition > and thought using > rasterOptions(tmpdir="/home/alobo/provi") > > Nevertheless, this has no effect, and temp raster files keep growing > in the default directory within /tmp > > Is this a bug? > > Thanks > Agus > > > sessionInfo() > R version 3.3.3 (2017-03-06) > Platform: i686-pc-linux-gnu (32-bit) > Running under: Debian GNU/Linux buster/sid > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > LC_TIME=en_US.UTF-8 > [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 > LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C > LC_ADDRESS=C > [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 > LC_IDENTIFICATION=C > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] raster_2.5-8 sp_1.2-5 > > loaded via a namespace (and not attached): > [1] rgdal_1.2-11parallel_3.3.3 tools_3.3.3 Rcpp_0.12.12 > grid_3.3.3 > [6] lattice_0.20-35 > > ___________ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] converting the projection
Great thanks, that is good detail -but we still need the extents of both
data to be able to see where the problem is.
You gave us the proj4string for both, but we also need the extent - you can
print each object for a full summary eg.
print(bb)
print(tmp)
and with those anyone can reproduce enough (without any real data) to
recreate your experience. (I am so used to working with this stuff I forget
to remind others that print(theraster) is usually more than enough
information to work through a reproducible example)
But, a guess is - unless the projection metadata is wrong, raster is
probably telling you the truth - but it should be easy to see why there's a
problem with the print output (essentially the proj4string and extent
provide enough information for an experienced problem solver).
Cheers, Mike.
On Wed, 20 Sep 2017 at 11:05 Kyongho Son <kkyon...@hotmail.com> wrote:
> Sorry and thanks, Mike
> I can not share all codes and data since the original data are huge.
> I copied the scripts that I used to reproject one of rasters and then make
> sure that they have same projection
>
> Then clip a large raster with a small raster. However, I got the error
> messages, the two raster maps are not overlap.
>
> They should overlap if the projection was correct.
>
>
>
> # east US map tree phenology map
> tmp<-raster("../../obs/eMODIS_East_SOSTAvg0114_v1/sostavg0114")
>
> proj4string(tmp)
> [1] "+proj=laea +lat_0=-100 +lon_0=6370997 +x_0=45 +y_0=0 +datum=WGS84
> +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0"
>
> # small watershed boundary map in NY state
> bb<-raster("../../obs/gis/bb_soil_def_map.tif")
> proj4string(bb)
> [1] "+proj=utm +zone=18 +datum=WGS84 +units=m +no_defs +ellps=WGS84
> +towgs84=0,0,0"
>
> ex <- projectExtent(bb, projection(tmp))
>
> > ex
> class : RasterLayer
> dimensions : 598, 515, 307970 (nrow, ncol, ncell)
> resolution : 30.12012, 14.94084 (x, y)
> extent : 6691756, 6707267, 9941473, 9950408 (xmin, xmax, ymin, ymax)
> coord. ref. : +proj=laea +lat_0=-100 +lon_0=6370997 +x_0=45 +y_0=0
> +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
>
>
> c<-crop(tmp, ex)
>
> Error in .local(x, y, ...) : extents do not overlap
>
>
>
> *Kyongho Son*
> Postdoctoral Fellow
> Research Foundation of The City University of New York
>
>
>
> --
> *From:* Michael Sumner <mdsum...@gmail.com>
> *Sent:* Tuesday, September 19, 2017 8:32 PM
> *To:* Kyongho Son; RsigGeo
>
> *Subject:* Re: [R-sig-Geo] converting the projection
> Keep on list please.
>
> Share code to demonstrate so we can see the problem. Use projectRaster(a,
> b) for alignment, try crop(b, ex, snap = "out") for inclusive crop.
>
> And apologies I wrote clip instead of crop earlier.
>
> Cheers, Mike
>
> On Wed, 20 Sep 2017, 07:42 Kyongho Son <kkyon...@hotmail.com> wrote:
>
>> I did the exact same way, but the output of the reprojection of 'a'
>> raster did not properly align with 'b' raster.
>>
>>
>>
>>
>>
>> *Kyongho Son*
>> Postdoctoral Fellow
>> Research Foundation of The City University of New York
>>
>>
>>
>> --
>> *From:* Michael Sumner <mdsum...@gmail.com>
>> *Sent:* Tuesday, September 19, 2017 5:03 PM
>> *To:* Kyongho Son; r-sig-geo
>> *Subject:* Re: [R-sig-Geo] converting the projection
>>
>> Try projecting just the extent
>> library(raster)
>> ex <- projectExtent(a, projection(b))
>> clip(b, ex)
>>
>> where a, b are your two raster.
>>
>> Cheers, Mike
>>
>> On Wed, 20 Sep 2017, 06:18 Kyongho Son <kkyon...@hotmail.com> wrote:
>>
>>>
>>> Hi I have two raster data sets but they have two datum and projection.
>>>
>>> I would like to convert one to the other since I would like to clip the
>>> one raster using the other raster.
>>>
>>> they should have same projections.
>>>
>>>
>>> proj4string(bb)
>>> [1] "+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
>>> proj4string(SOSTavg)
>>>
>>> proj4string(SOSTavg)
>>> [1] "+proj=laea +lat_0=-100 +lon_0=6370997 +x_0=45 +y_0=0 +ellps=WGS84
>>> +towgs84=0,0,0,0,0,0,0 +units=m +no_defs"
>>>
>>>
>>> Thanks,
>>>
>>> Kyongho
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> _
Re: [R-sig-Geo] converting the projection
Keep on list please. Share code to demonstrate so we can see the problem. Use projectRaster(a, b) for alignment, try crop(b, ex, snap = "out") for inclusive crop. And apologies I wrote clip instead of crop earlier. Cheers, Mike On Wed, 20 Sep 2017, 07:42 Kyongho Son <kkyon...@hotmail.com> wrote: > I did the exact same way, but the output of the reprojection of 'a' raster > did not properly align with 'b' raster. > > > > > > *Kyongho Son* > Postdoctoral Fellow > Research Foundation of The City University of New York > > > > -- > *From:* Michael Sumner <mdsum...@gmail.com> > *Sent:* Tuesday, September 19, 2017 5:03 PM > *To:* Kyongho Son; r-sig-geo > *Subject:* Re: [R-sig-Geo] converting the projection > > Try projecting just the extent > library(raster) > ex <- projectExtent(a, projection(b)) > clip(b, ex) > > where a, b are your two raster. > > Cheers, Mike > > On Wed, 20 Sep 2017, 06:18 Kyongho Son <kkyon...@hotmail.com> wrote: > >> >> Hi I have two raster data sets but they have two datum and projection. >> >> I would like to convert one to the other since I would like to clip the >> one raster using the other raster. >> >> they should have same projections. >> >> >> proj4string(bb) >> [1] "+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0" >> proj4string(SOSTavg) >> >> proj4string(SOSTavg) >> [1] "+proj=laea +lat_0=-100 +lon_0=6370997 +x_0=45 +y_0=0 +ellps=WGS84 >> +towgs84=0,0,0,0,0,0,0 +units=m +no_defs" >> >> >> Thanks, >> >> Kyongho >> >> >> >> >> >> >> >> >> [[alternative HTML version deleted]] >> >> ___ >> R-sig-Geo mailing list >> R-sig-Geo@r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo >> > -- > > <https://maps.google.com/?q=203+Channel+Highway+%0D+Kingston+Tasmania+7050+Australia=gmail=g> > Dr. Michael Sumner > Software and Database Engineer > Australian Antarctic Division > 203 Channel Highway > <https://maps.google.com/?q=203+Channel+Highway+%0D+Kingston+Tasmania+7050+Australia=gmail=g> > Kingston Tasmania 7050 Australia > <https://maps.google.com/?q=203+Channel+Highway+%0D+Kingston+Tasmania+7050+Australia=gmail=g> > > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] converting the projection
Try projecting just the extent library(raster) ex <- projectExtent(a, projection(b)) clip(b, ex) where a, b are your two raster. Cheers, Mike On Wed, 20 Sep 2017, 06:18 Kyongho Son <kkyon...@hotmail.com> wrote: > > Hi I have two raster data sets but they have two datum and projection. > > I would like to convert one to the other since I would like to clip the > one raster using the other raster. > > they should have same projections. > > > proj4string(bb) > [1] "+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0" > proj4string(SOSTavg) > > proj4string(SOSTavg) > [1] "+proj=laea +lat_0=-100 +lon_0=6370997 +x_0=45 +y_0=0 +ellps=WGS84 > +towgs84=0,0,0,0,0,0,0 +units=m +no_defs" > > > Thanks, > > Kyongho > > > > > > > > > [[alternative HTML version deleted]] > > _______ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Replacing values in a RasterBrick
On Sun, 10 Sep 2017 at 10:45 BEUTEL Terry S <terry.beu...@daf.qld.gov.au>
wrote:
> Thanks Michael.
>
> I tried pulling it all into memory and running on a heftier computer and
> it now works. I also changed the datatype to INT2U which sped things up
> further without losing significant information in my data.
>
>
>
> Consequently the ff option now probably isn’t necessary for me, but if its
> simple to do as you say, and speeds up processing and/or pushes up the size
> of raster/bricks/stacks that users can process it would always help, and
> I’d be happy to trial it on data here if that would help.
>
>
>
Thanks! I'll take you up on that when I look at it again.
Cheers, Mike.
> Cheers
>
> Terry
>
>
>
>
>
> *From:* Michael Sumner [mailto:mdsum...@gmail.com]
> *Sent:* Thursday, 7 September 2017 6:38 PM
> *To:* BEUTEL Terry S; r-sig-geo@r-project.org
> *Subject:* Re: [R-sig-Geo] Replacing values in a RasterBrick
>
>
>
>
>
> On Thu, 7 Sep 2017 at 08:22 BEUTEL Terry S <terry.beu...@daf.qld.gov.au>
> wrote:
>
> Hi.
>
> I'm working with the Raster package. I have a RasterBrick of 374 monthly
> rainfall rasters called Rain2.
>
>
> > Rain2
> class : RasterBrick
> dimensions : 374, 315, 117810, 374 (nrow, ncol, ncell, nlayers)
> resolution : 0.05, 0.05 (x, y)
> extent : 137.925, 153.675, -29.275, -10.575 (xmin, xmax, ymin, ymax)
> coord. ref. : +proj=longlat +ellps=GRS80 +no_defs
> data source : K:\RainStuff\RainRaster.grd
> names :X198601,X198602,X198603,X198604 ...
> min values : 0.00, 0.00, 0.00, 0.00 ...
> max values : 962.01929, 534.42188, 929.24780, 790.83087 ...
>
>
>
> I want to replace the 374 monthly rainfall pixel values at a particular
> location (defined by the intersection of a single row and a single column)
> with a different series of 374 values. But when I try the reassignment, I
> run into memory issues
>
>
>
> As far as I understand it there is no actual partial-write for the this
> format, it's actually a complicated topic given the range of ways raster
> can store data either on file or in memory.
>
>
>
> A Raster in memory is a long-form matrix with the matrix flattened out
> (left right top to bottom, different to R's matrix), and the columns are
> "layers". On disk, the rasterfile (.grd) is a flat binary with an
> orientation ordering (either band sequential, band interleaved by line or
> band interleaved by pixel) and so that can't be easily overridden without
> knowing the details.
>
>
>
> Given the size of the file I would first try pulling it all into memory
> and rewriting in whole:
>
>
>
> Rain2 <- readAll(Rain2)
>
>
>
> ## then run your updates
>
> ## ...
>
> ## now write back to file
>
>
>
> writeRaster(Rain2, "RainRaster_test.grd")
>
>
>
> and check.
>
>
>
> If that fails, because your system can't perform this (wasteful, brute
> force) read and write then you could use ff - it's possible to
> transparently map an ff array to an existing RasterBrick (in rasterfile.grd
> format) and update it using ff's efficient memory mapping.
>
>
>
> Let me know if you that sounds promising and you want to explore more,
> it's a simple thing I should finish up and put on CRAN.
>
>
>
> Cheers, Mike.
>
>
>
>
>
> > Rain2[100,100]<-c(1:374)
>
> Error: cannot allocate vector of size 336.2 Mb
>
>
>
> I've also tried using setValues to reassign the values, as per the
> "Writing functions with the 'raster' package" vignette, but without success
> e.g. ...
>
> > f2<-
>
> + function(x,row,col,a){
>
> + x[row,col]<-setValues(x[row,col],a)
>
> + return(x)}
>
> >
>
> > a<-c(1:374)
>
> > f2(Rain2,100,101,a)
>
>
>
> Error in (function (classes, fdef, mtable) :
>
> unable to find an inherited method for function 'setValues' for
> signature '"matrix"'
>
>
>
>
>
> I don't work much with the Raster package and I'm sure I'm making an
> elementary error here, but I'd appreciate any advice on the best way
> replace these values efficiently.
>
> Many thanks
>
> Terry
>
> --
> The information in this email together with any attachments is intended
> only for the person or entity to which it is addressed and may contain
> confidential and/or privileged material. There is no waiver of any
> confidentiality/privilege by your inadvertent receipt of this material.
> Any form of review, disclosure, modification,
Re: [R-sig-Geo] Replacing values in a RasterBrick
On Thu, 7 Sep 2017 at 08:22 BEUTEL Terry S <terry.beu...@daf.qld.gov.au>
wrote:
> Hi.
>
> I'm working with the Raster package. I have a RasterBrick of 374 monthly
> rainfall rasters called Rain2.
>
>
> > Rain2
> class : RasterBrick
> dimensions : 374, 315, 117810, 374 (nrow, ncol, ncell, nlayers)
> resolution : 0.05, 0.05 (x, y)
> extent : 137.925, 153.675, -29.275, -10.575 (xmin, xmax, ymin, ymax)
> coord. ref. : +proj=longlat +ellps=GRS80 +no_defs
> data source : K:\RainStuff\RainRaster.grd
> names :X198601,X198602,X198603,X198604 ...
> min values : 0.00, 0.00, 0.00, 0.00 ...
> max values : 962.01929, 534.42188, 929.24780, 790.83087 ...
>
>
>
> I want to replace the 374 monthly rainfall pixel values at a particular
> location (defined by the intersection of a single row and a single column)
> with a different series of 374 values. But when I try the reassignment, I
> run into memory issues
>
>
As far as I understand it there is no actual partial-write for the this
format, it's actually a complicated topic given the range of ways raster
can store data either on file or in memory.
A Raster in memory is a long-form matrix with the matrix flattened out
(left right top to bottom, different to R's matrix), and the columns are
"layers". On disk, the rasterfile (.grd) is a flat binary with an
orientation ordering (either band sequential, band interleaved by line or
band interleaved by pixel) and so that can't be easily overridden without
knowing the details.
Given the size of the file I would first try pulling it all into memory and
rewriting in whole:
Rain2 <- readAll(Rain2)
## then run your updates
## ...
## now write back to file
writeRaster(Rain2, "RainRaster_test.grd")
and check.
If that fails, because your system can't perform this (wasteful, brute
force) read and write then you could use ff - it's possible to
transparently map an ff array to an existing RasterBrick (in rasterfile.grd
format) and update it using ff's efficient memory mapping.
Let me know if you that sounds promising and you want to explore more,
it's a simple thing I should finish up and put on CRAN.
Cheers, Mike.
>
> > Rain2[100,100]<-c(1:374)
>
> Error: cannot allocate vector of size 336.2 Mb
>
>
>
> I've also tried using setValues to reassign the values, as per the
> "Writing functions with the 'raster' package" vignette, but without success
> e.g. ...
>
> > f2<-
>
> + function(x,row,col,a){
>
> + x[row,col]<-setValues(x[row,col],a)
>
> + return(x)}
>
> >
>
> > a<-c(1:374)
>
> > f2(Rain2,100,101,a)
>
>
>
> Error in (function (classes, fdef, mtable) :
>
> unable to find an inherited method for function 'setValues' for
> signature '"matrix"'
>
>
>
>
>
> I don't work much with the Raster package and I'm sure I'm making an
> elementary error here, but I'd appreciate any advice on the best way
> replace these values efficiently.
>
> Many thanks
>
> Terry
>
> --
> The information in this email together with any attachments is intended
> only for the person or entity to which it is addressed and may contain
> confidential and/or privileged material. There is no waiver of any
> confidentiality/privilege by your inadvertent receipt of this material.
> Any form of review, disclosure, modification, distribution and/or
> publication of this email message is prohibited, unless as a necessary part
> of Departmental business.
> If you have received this message in error, you are asked to inform the
> sender as quickly as possible and delete this message and any copies of
> this message from your computer and/or your computer system network.
> --
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] [FORGED] Re: using spatialpolygonsdataframe in ppm (or, converting spatialpolygonsdataframe to pixel image or other object useful in ppm)
Thanks Rolf, it's actually not what I meant and I appreciate the clarification :) - owin is clear in its definition of specific types, but spatstat types generally do allow combinations of domain, discretization, and path-based boundaries. So I hope I'm not wrong about *that*. (I aspire to be more expert on this cross-over topic, still working on it :) Cheers, Mike. On Sat, 2 Sep 2017 at 20:27 Rolf Turner <r.tur...@auckland.ac.nz> wrote: > On 02/09/17 19:00, Michael Sumner wrote: > > > > > > > It's not clear to me if you need polygons in spatstat or a raster owin > > version of them - admittedly spatstat does allow a combination of the > two, > > That's not actually true. The spatstat package allows for observation > windows (objects of class "owin" which are either polygonal (of type > "polygonal") *or* "raster-like" (of type "mask"), consisting of a > pixellation of the bounding box with the pixels having the values TRUE > or FALSE. There is also type "rectangle" which is conceptually a > special case of "polygonal" but has (obviously?) practical advantages > when it is applicable. > > However an "owin" object *cannot* be a combination of polygonal and mask > types. > > > > cheers, > > Rolf Turner > > -- > Technical Editor ANZJS > Department of Statistics > University of Auckland > Phone: +64-9-373-7599 ext. 88276 <+64%209-373%207599> > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] using spatialpolygonsdataframe in ppm (or, converting spatialpolygonsdataframe to pixel image or other object useful in ppm)
On Sat, 2 Sep 2017 at 04:04 Christopher W. Ryan <cr...@binghamton.edu>
wrote:
> Hello.
>
> What is the best way to use a spatialpolygonsdataframe, with a numerical
> variable of interest for each polygon (proportion of households in
> poverty for US census tracts in the region of interest) as a predictor
> in ppm() in spatstat? I don't think I can use it directly on the RHS of
> ppm(), because spatialpolygonsdataframe is not listed in the help file
> for ppm() as an acceptable predictor. So is there a way to convert the
> census tract spatialpolygonsdataframe to an acceptable input object for
> ppm(), such as a pixel image with each pixel having the numerical value
> of poverty in its census tract polygon?
>
>
Start with vignette("shapefiles", package = "spatstat") for the spatstat
view of Spatial* - it's relatively straightforward to convert Spatial* to
raster and raster to spatstat, but I don't have an example on hand.
It's not clear to me if you need polygons in spatstat or a raster owin
version of them - admittedly spatstat does allow a combination of the two,
so that's maybe what's needed.
If you have a clear example of using ppm that would do the task if had your
data in spatstat form I'm happy to show the conversion/s.
HTH
Cheers, Mike.
> Or is there a better way to proceed?
>
> Thank you. Below is my sessionInfo
>
> --Chris Ryan
> Broome County Health Department, Binghamton, NY
>
> =
> R version 3.3.3 (2017-03-06)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows 7 x64 (build 7601) Service Pack 1
>
> locale:
> [1] LC_COLLATE=English_United States.1252
> [2] LC_CTYPE=English_United States.1252
> [3] LC_MONETARY=English_United States.1252
> [4] LC_NUMERIC=C
> [5] LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> other attached packages:
> [1] tmap_1.10 rgdal_1.2-6RColorBrewer_1.1-2
> maptools_0.9-2
> [5] sp_1.2-4 spatstat_1.50-0rpart_4.1-10
> nlme_3.1-131
> [9] shapefiles_0.7 foreign_0.8-67 stringr_1.2.0
> dplyr_0.5.0
>
> loaded via a namespace (and not attached):
> [1] viridisLite_0.2.0jsonlite_1.4 splines_3.3.3
> [4] geojsonlint_0.2.0foreach_1.4.3R.utils_2.5.0
> [7] gtools_3.5.0 shiny_1.0.5 assertthat_0.2.0
> [10] expm_0.999-2 stats4_3.3.3 LearnBayes_2.15
> [13] lattice_0.20-35 digest_0.6.12polyclip_1.6-1
> [16] colorspace_1.3-2 plyr_1.8.4 htmltools_0.3.5
> [19] httpuv_1.3.5 Matrix_1.2-8 R.oo_1.21.0
> [22] XML_3.98-1.9 rmapshaper_0.3.0 raster_2.5-8
> [25] gmodels_2.16.2 xtable_1.8-2 webshot_0.4.1
> [28] scales_0.4.1 gdata_2.17.0 tensor_1.5
> [31] satellite_1.0.0 spatstat.utils_1.4-1 tibble_1.3.0
> [34] mgcv_1.8-17 gdalUtils_2.0.1.7mapview_2.1.4
> [37] magrittr_1.5 mime_0.5 deldir_0.1-12
> [40] R.methodsS3_1.7.1MASS_7.3-45 class_7.3-14
> [43] tools_3.3.3 geosphere_1.5-5 V8_1.5
> [46] munsell_0.4.3compiler_3.3.3 e1071_1.6-8
> [49] units_0.4-5 classInt_0.1-24 grid_3.3.3
> [52] tmaptools_1.2-1 RCurl_1.95-4.8 dichromat_2.0-0
> [55] iterators_1.0.8 htmlwidgets_0.8 goftest_1.1-1
> [58] crosstalk_1.0.0 bitops_1.0-6 base64enc_0.1-3
> [61] boot_1.3-18 codetools_0.2-15 abind_1.4-5
> [64] DBI_0.6-1jsonvalidate_1.0.0 curl_2.5
> [67] R6_2.2.0 udunits2_0.13rgeos_0.3-23
> [70] spdep_0.6-12 KernSmooth_2.23-15 stringi_1.1.5
> [73] osmar_1.1-7 Rcpp_0.12.10 sf_0.5-3
> [76] png_0.1-7leaflet_1.1.0coda_0.19-1
> >
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Problem in extracting data from GRIB files
The print-out of rdata is good, but I can't really address the issue you
say about NAs without knowing what is in "pt_dams". For that we need to
know what kind of thing it is:
class(pt_dams)
what projection it's in
projection(pt_dams)
and it's extent, and unfortunately for that you need to know what kind of
thing it is so either
extent(pt_dams) ## assuming it's a classed spatial-kind-of-thing
or this will vaguely give the idea
head(pt_dams) ## assuming it's a matrix or data frame
You've shown a lot of code but I think we only need to concentrated on
"rdata", as a representative of the raster data all your files will have,
and pt_dams and this line:
data_dams_size<-extract(rdata,pts_dams)
(It's also possible to make the entire task much faster by building an
index at that first step, possibly with cellnumbers = TRUE
But, you'll have to make sure that extract line is doing what you need it
to first. It's very hard to do this by email, and you really haven't honed
into the key details yet. If the coordinates in pt_dams don't fall within
the extent of rdata, then you're in trouble (or just in the wrong
projection, probably). raster makes life easy if you know the coordinates
are in the same space as the raster, but none of this metadata stuff is in
any way consistent or complete so as ever "it depends".
Cheers, Mike.
On Mon, 31 Jul 2017 at 20:13 Miluji Sb <miluj...@gmail.com> wrote:
> Dear Mike,
>
> Thank you very much for your suggestion, apologies for the delayed reply -
> I did not have access to internet over the weekend.
>
> Here is the print-out for rdata:
>
> class : RasterStack
> dimensions : 150, 360, 54000, 8 (nrow, ncol, ncell, nlayers)
> resolution : 1, 1 (x, y)
> extent : -180, 180, -60, 90 (xmin, xmax, ymin, ymax)
> coord. ref. : +proj=longlat +a=6367470 +b=6367470 +no_defs
> names : GLDAS_NOA//215507.020, GLDAS_NOA//215507.020,
> GLDAS_NOA//215507.020, GLDAS_NOA//215507.020, GLDAS_NOA//215507.020,
> GLDAS_NOA//215517.020, GLDAS_NOA//215517.020, GLDAS_NOA//215517.020
>
> I also made this change [rdata <- raster::stack(lapply(filelist,
> function(x) raster(x,band=12)))] but still getting NAs. The full (ugly)
> code is below
>
> I would be grateful for any help. Thank you again!
>
> Sincerely,
>
> Milu
>
>
> for (y in year) {
> setwd (y)
>
> day <- dir(path=getwd(),full.names=TRUE, recursive=FALSE,pattern =
> "^[0-9]" )
>
> n=0
>
> for(d in day) {
> setwd (d)
>
> n=n+1
>
>
> filelist<-list.files(pattern=".grb$")
>
> rdata <- raster::stack(lapply(filelist, function(x)
> raster(x,band=12)))
>
> data_dams_size<-extract(rdata,pts_dams)
> DF_data_dams_size<- as.data.frame(data_dams_size)
>
> #Join ISO3, lon, lat, dams data, climate data
> joined_dams <- cbind(foo_dams, DF_data_dams_size)
>
> #Keep only ISO3 LON LAT id
> joined_dams_reduced <- joined_dams
> dams_codes <- joined_dams[,c(1:3)]
>
> names(joined_dams_reduced) <- gsub("GLDAS_NOAH10_3H.A", "",
> names(joined_dams_reduced))
> names(joined_dams_reduced) <- gsub(".020", "",
> names(joined_dams_reduced))
>
> #From mm/s to mm_day
> joined_dams_reduced[joined_dams_reduced==] <- NA
>
> ##for prec, snow and runoff use:
> #joined_dams_reduced$mm_day<-rowSums(joined_dams_reduced[,4:11])*10800
>
> ##for temperature use:
> joined_dams_reduced$mm_day<-rowMeans(joined_dams_reduced[,4:11])-273.15
>
> joined_mm_day<-joined_dams_reduced[,12]
>
> #Names
> yr_name <- substr(filelist,18,21)
> csvname <- paste(paste(yr_name, sep="_"), ".csv", sep="")
> csvname <- csvname[1]
>
> #Stack days to get full year
> if (n==1) {joined_mm_day_year <- joined_mm_day} else
> {joined_mm_day_year <- cbind(joined_mm_day_year, joined_mm_day)}
>
> }
>
> #Join mm_day with dams codes
> joined_mm_day_year_names=cbind(dams_codes,joined_mm_day_year)
>
> newnames <-t(as.data.frame(c(paste ("day",
> seq(1,(ncol(joined_mm_day_year_names)-3),1), sep="_"
> names(joined_mm_day_year_names)[4:ncol(joined_mm_day_year_names)]=
> newnames
>
>
> write.csv(joined_mm_day_year_names,
> file=file.path("C:/Users/Desktop/temp/",csvname))
>
> }
>
>
> On Sat, Jul 29, 2017 at 12:36 AM, Michael Sumner <mdsum...@gmail.com>
> wrote:
>
>>
>>
>> On Sat, 29 Jul 2017 at 02:21 Miluji Sb <miluj...@gmail.com> wrote:
>>
>>> Dear all,
>>>
>&
Re: [R-sig-Geo] Problem in extracting data from GRIB files
On Sat, 29 Jul 2017 at 02:21 Miluji Sb <miluj...@gmail.com> wrote:
> Dear all,
>
> I have a set of coordinates for cities:
>
> structure(list(lon = c(3.7174243, 3.2246995, 2.928656, 33.3822764,
> 10.40237, 24.7535746, 7.2619532, -0.370679, -4.486076, -4.097899
> ), lat = c(51.0543422, 51.209348, 51.21543, 35.1855659, 55.403756,
> 59.4369608, 43.7101728, 49.182863, 48.390394, 47.997542)), .Names =
> c("lon",
> "lat"), na.action = structure(c(5L, 6L, 31L, 55L, 59L, 61L, 67L,
> 68L), .Names = c("5", "6", "31", "55", "59", "61", "67", "68"
> ), class = "omit"), row.names = c(1L, 2L, 3L, 4L, 7L, 8L, 9L,
> 10L, 11L, 12L), class = "data.frame")
>
> I am trying to extract climatic data from GLDAS climatic data (1 degree x 1
> degree) GRIB files with the following code:
>
> # Convert to spatial points
> pts_dams <- SpatialPoints(lonlat,proj4string=CRS("+proj=longlat"))
>
> # Extract
> filelist<-list.files(pattern=".grb$")
>
>
This part is a bit wrong:
> data<-stack(filelist[1])
> data@layers<-sapply(filelist, function(x) raster(x,band=12))
>
Better would be
rdata <- raster::stack(lapply(filelist, function(x) raster(x,band=12)))
For the rest we will need to see details about the data, so please do
print(rdata)
and send the resulting print-out.
A minor thing "data" is already a function, so it's advisable not to use
it as a name - if only to reduce possible confusion.
Also, it's not obvious that the process by which raster(file) goes though
(it passes down to rgdal::readGDAL) will result in a correct interpretation
as the data source was intended, since GRIB is a domain-specific format for
time-series and volume-slice series data and the required translation to
the GDAL and raster data models is not always straight-forward.
It totally can work though, I do it routine with many sources but they all
need some oversight to ensure that upfront translation is complete and
appropriate.
It may just require setting raster::extent and/or raster::projection, but
there's no way to know without exploration.
Cheers, Mike.
> data_dams_size<-extract(data,pts_dams)
> DF_data_dams_size<- as.data.frame(data_dams_size)
>
> Unfortunately, I get mostly NAs for the data, it seems that there's an
> issue with the CRS projections for the city coordinates. Is there a
> specific projection for city level coordinates? Or am I doing something
> completely wrong? Thank you!
>
> Sincerely,
>
> Milu
>
> [[alternative HTML version deleted]]
>
> ___
> R-sig-Geo mailing list
> R-sig-Geo@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia
[[alternative HTML version deleted]]
___
R-sig-Geo mailing list
R-sig-Geo@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] raster - unrotate?
That's great, thanks for the update! I didn't actually realize all that oceandata stuff was on Thredds. You might try ROpenSci's rerddap package too. I will try that source with the in-dev https://Github.com/hypertidy/tidync . Please check it out if you can, and are feeling brave! (tidync aims to provide more general outputs than raster can, more easily than the API wrappers.) Cheers, Mike On Thu, 20 Jul 2017, 10:39 Ben Tupper, <btup...@bigelow.org> wrote: > Ahoy! > > My (current) solution is to user raster::merge(). Works a treat and fits > my workflow nicely. Since I am obtaining the raster data using OPeNDAP it > is easy to get the two regions, rotate the extent of the eastern region, > and then merge. Below is a pseudo-example (that works!) as it crops a > local file twice, once for each region, rather than using an OPeNDaP calls > via `ncdf4::nccvar_get(start = blah, count = something)` to get the two > regions. > > > ### start > > library(raster) > uri <- ' > https://oceandata.sci.gsfc.nasa.gov:443/opendap/MODISA/L3SMI/2016/001/A20160012016032.L3m_R32_SST_sst_9km.nc > ' > R <- raster::raster(uri, varname = 'sst') > > R > #class : RasterLayer > #dimensions : 180, 360, 64800 (nrow, ncol, ncell) > #resolution : 1, 1 (x, y) > #extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax) > #coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 > #data source : in memory > #names : layer > #values : 1.482572e-05, 0.928 (min, max) > > par(mfrow = c(2,1)) > plot(R) > lines(c(180, 100, 100, 180), c(80,80,0,0)) > lines(c(-180,-90,-90,-180), c(80,80,0,0)) > > # eastern section > eR <- raster::crop(R, c(100, 180, 0, 80)) > > # rotate the extent > eExtent <- raster::extent(eR) > raster::extent(eR) <- c(-260, -180, eExtent[3:4]) > > # western section > wR <- raster::crop(R, c(-180, -90, 0, 80)) > > # merge > newR <- raster::merge(eR, wR) > newR > > plot(newR) > > end > > Cheers, > Ben > > > On Jun 22, 2017, at 7:26 PM, Michael Sumner <mdsum...@gmail.com> wrote: > > > > > > > > > > On Thu, 22 Jun 2017 at 21:28 Ben Tupper <btup...@bigelow.org> wrote: > > Hi, > > > > Wow! A treasure from the past! > > > > Hmmm. I see what you mean; it might be best to snip-clip-and-smush > from the native presentation. We are testing out the performance of the > dineof function in the sinkr package ( > https://github.com/marchtaylor/sinkr/blob/master/R/dineof.R) to > interpolate patchy chlorophyl data. > > > > Thanks for the tip! > > > > > > > > For what it's worth, I'm happy to help, there's no one size fits all > here. The dateline is one of those "easy problems" that does not have a > general fix and will bite in many different contexts. :) > > > > The best distillation of my tricks is in this project, but it does > depend on your workflow and the actual data in use. > > > > https://github.com/hypertidy/tabularaster > > > > If you have performance issues raster's cell abstraction will help, but > they are a bit old-school when it comes to multi-part geometries. (It's > identical to standard database indexing concepts as are all "spatial" > optimization tricks) > > > > (I see a desperate need for a general API for this kind of problem so > that we can build tools from a general framework, which I'm working on - > that's some kind of excuse for why this and related projects are quite raw > and unfinished.) > > > > Cheers, Mike. > > > > Cheers, > > Ben > > > >> On Jun 22, 2017, at 4:46 AM, Michael Sumner <mdsum...@gmail.com> wrote: > >> > >> > >> It used to do the inverse, and I preserved a copy of the old behaviour > here: > >> > >> > https://github.com/AustralianAntarcticDivision/raadtools/blob/master/R/utils.R#L9 > >> > >> You're probably best to learn from how it approaches it rather than > just use it, since it's not a general capability, just works for those very > specific cases. > >> > >> I just tested it and it still seems to work fine, I used > >> > >> > oceandata.sci.gsfc.nasa.gov/MODISA/Mapped/Monthly/9km/chlor/A20021822002212.L3m_MO_CHL_chlor_a_9km.nc > >> > >> obtainable with > >> > >> > https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A20021822002212.L3m_MO_CHL_chlor_a_9km.nc > >> > >> If you're extracting points you might as well just convert them into > the "Atlantic" range, but it's painful to get that right for pol
Re: [R-sig-Geo] Any plans to provide access to CGAL in R?
On Mon, 10 Jul 2017 at 22:03 Robin Lovelace <rob...@gmail.com> wrote: > Hi all, > > I've just been made aware of the CGAL <http://www.cgal.org/> C++ library > for computational geometry. > > Hi Robin, we've discussed this here: https://github.com/r-spatial/discuss/issues/6 FWIW a few years ago I did some simple Rcpp tests: https://github.com/r-gris/cgalgris Then I discovered RTriangle (it and TDA use the same underlying library) and that provides the main thing I was after, now in package sfdct and applied for more general use here: https://github.com/r-gris/rangl There is a lot you can do with a constrained triangulation, it's a "hammer" and sometimes overkill but it's extremely flexible and powerful. I'm happy to flesh out examples for problems you think might be solvable with it to help promote why CGAL would be useful in R. I'm very interested in general forms for these data, and the idea is to put all the 3D rangl stuff on this footing eventually: https://github.com/mdsumner/sc tidygraph is a very good example of how we can modernize these indexed data structures, and other newish packages icosa, webglobe, dggridR, also touch on this "mesh" way of working. For that matter spdep and GEOS work that way internally, but we only have fragments of the concept available at an R level. Simon Urbanek has s-u/rcgal on github but it doesn't expose very much. Cheers, Mike. > It seems to be able to do some things that may be useful to some readers of > of this list, as outlined in this overview of a syllabus based on CGAL: > http://delivery.acm.org/10.1145/293/2927362/a8-alliez.pdf > > I've found some non-CRAN packages that link to it but nothing that seems to > provide access to all CGAL functions (listed below). Any ideas if there are > plans afoot for this or even if it's worth doing? > > >- http://kien-kieu.github.io/lite/rlite.html >- https://github.com/rcqls/EBSpatCGAL > > There is also TDA on CRAN: > https://cran.r-project.org/web/packages/TDA/index.html > > And a C++ library that claims to support it for simple features (found by > looking for support in QGIS): https://github.com/Oslandia/SFCGAL > > Plus a question on StackExchange: > > https://stackoverflow.com/questions/14463590/cgal-tools-is-there-an-interface-to-cgal-or-equivalent-toolset-in-r > > Thanks in advance, > > Robin > > [[alternative HTML version deleted]] > > ___________ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] raster - unrotate?
On Thu, 22 Jun 2017 at 21:28 Ben Tupper <btup...@bigelow.org> wrote: > Hi, > > Wow! A treasure from the past! > > Hmmm. I see what you mean; it might be best to snip-clip-and-smush from > the native presentation. We are testing out the performance of the dineof > function in the sinkr package ( > https://github.com/marchtaylor/sinkr/blob/master/R/dineof.R) to > interpolate patchy chlorophyl data. > > Thanks for the tip! > > For what it's worth, I'm happy to help, there's no one size fits all here. The dateline is one of those "easy problems" that does not have a general fix and will bite in many different contexts. :) The best distillation of my tricks is in this project, but it does depend on your workflow and the actual data in use. https://github.com/hypertidy/tabularaster If you have performance issues raster's cell abstraction will help, but they are a bit old-school when it comes to multi-part geometries. (It's identical to standard database indexing concepts as are all "spatial" optimization tricks) (I see a desperate need for a general API for this kind of problem so that we can build tools from a general framework, which I'm working on - that's some kind of excuse for why this and related projects are quite raw and unfinished.) Cheers, Mike. Cheers, > Ben > > On Jun 22, 2017, at 4:46 AM, Michael Sumner <mdsum...@gmail.com> wrote: > > > It used to do the inverse, and I preserved a copy of the old behaviour > here: > > > https://github.com/AustralianAntarcticDivision/raadtools/blob/master/R/utils.R#L9 > > You're probably best to learn from how it approaches it rather than just > use it, since it's not a general capability, just works for those very > specific cases. > > I just tested it and it still seems to work fine, I used > > > oceandata.sci.gsfc.nasa.gov/MODISA/Mapped/Monthly/9km/chlor/A20021822002212.L3m_MO_CHL_chlor_a_9km.nc > > > obtainable with > > > https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A20021822002212.L3m_MO_CHL_chlor_a_9km.nc > > > If you're extracting points you might as well just convert them into the > "Atlantic" range, but it's painful to get that right for polygons or lines > (and very hard to generalize once you get it working anyway). > > For simple regions like the one you show I'd probably split it into two > extents() and use those - but depends on your workflow, all these options > are useful here and there. > > Cheers, Mike. > > > On Thu, 22 Jun 2017 at 18:30 Ben Tupper <btup...@bigelow.org> wrote: > >> Hello, >> >> We have rasters that span [-180, 180] from NASA's Ocean Color ( >> https://oceancolor.gsfc.nasa.gov/) datasets. We are trying to extract >> a region that spans 100E to 90W, that is 100 to -90. The region 'wraps' >> across the edges as shown by the plot at the address below. >> >> https://dl.dropboxusercontent.com/u/8433654/sst.png >> >> >> library(raster) >> uri <- ' >> https://oceandata.sci.gsfc.nasa.gov:443/opendap/MODISA/L3SMI/2016/001/A20160012016032.L3m_R32_SST_sst_9km.nc >> <https://oceandata.sci.gsfc.nasa.gov/opendap/MODISA/L3SMI/2016/001/A20160012016032.L3m_R32_SST_sst_9km.nc> >> ' >> R <- raster::raster(uri, varname = 'sst') >> >> R >> #class : RasterLayer >> #dimensions : 180, 360, 64800 (nrow, ncol, ncell) >> #resolution : 1, 1 (x, y) >> #extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax) >> #coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 >> #data source : in memory >> #names : layer >> #values : 1.482572e-05, 0.928 (min, max) >> >> plot(R) >> lines(c(180, 100, 100, 180), c(80,80,0,0)) >> lines(c(-180,-90,-90,-180), c(80,80,0,0)) >> >> I see that there is the nice raster::rotate() function to rotate raster >> coordinates from [0,360] to [-180,180]. That would make extracting the >> region super easy if the inverse were available. Is there an equivalent >> way to implement the inverse or raster::rotate()? I could dig into the >> source of raster::rotate() to see how to code one up, but I hate like heck >> to reinvent the wheel. >> >> Thanks! >> Ben >> >> >> Ben Tupper >> Bigelow Laboratory for Ocean Sciences >> 60 Bigelow Drive, P.O. Box 380 >> East Boothbay, Maine 04544 >> http://www.bigelow.org >> >> ___ >> R-sig-Geo mailing list >> R-sig-Geo@r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo >> > -- > Dr. Michael Sumner > Software and Database Engineer > Australian Antarctic Division > 203 Channel Highway > Kingston Tasmania 7050 Australia > > > Ben Tupper > Bigelow Laboratory for Ocean Sciences > 60 Bigelow Drive, P.O. Box 380 > East Boothbay, Maine 04544 > http://www.bigelow.org > > > > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Problems with using dismo package on HPC
It might be because the file is absent. You need to trace through from the top, what file path is passed to raster, then try reading that directly with raster() and ensure the file exists. It's possibly a raster object with a link to a file that no longer exists. Raster is a complex polymorphic beast, and can be a thin brittle shell held together by a file name, with no safety net. If you don't have the original code it's harder, but also hard to help without access to your system. Cheers, Mike On Wed, 21 Jun 2017, 07:30 Wall, Wade A ERDC-RDE-CERL-IL CIV, < wade.a.w...@erdc.dren.mil> wrote: > Hi all, > > I am trying to use the dismo package on an HPC and am running into some > problems. The code that I used worked fine until a few days ago, then I > started getting the error message > > Error in rgdal::getRasterData(con, offset = offs, region.dim = c(1, nc), : > Failure during raster IO > > Not sure how to proceed on this. The code works on my desktop computer, > and like I said, worked until a few days ago on the HPC. I have read/write > privileges to the outfolder location, all the environmental layers are the > same extent and resolution, and the spatial points are in the same > coordinate system. > > Any help would be greatly appreciated. > Wade > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] raster - unrotate?
It used to do the inverse, and I preserved a copy of the old behaviour here: https://github.com/AustralianAntarcticDivision/raadtools/blob/master/R/utils.R#L9 You're probably best to learn from how it approaches it rather than just use it, since it's not a general capability, just works for those very specific cases. I just tested it and it still seems to work fine, I used oceandata.sci.gsfc.nasa.gov/MODISA/Mapped/Monthly/9km/chlor/A20021822002212.L3m_MO_CHL_chlor_a_9km.nc obtainable with https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A20021822002212.L3m_MO_CHL_chlor_a_9km.nc If you're extracting points you might as well just convert them into the "Atlantic" range, but it's painful to get that right for polygons or lines (and very hard to generalize once you get it working anyway). For simple regions like the one you show I'd probably split it into two extents() and use those - but depends on your workflow, all these options are useful here and there. Cheers, Mike. On Thu, 22 Jun 2017 at 18:30 Ben Tupper <btup...@bigelow.org> wrote: > Hello, > > We have rasters that span [-180, 180] from NASA's Ocean Color ( > https://oceancolor.gsfc.nasa.gov/) datasets. We are trying to extract a > region that spans 100E to 90W, that is 100 to -90. The region 'wraps' > across the edges as shown by the plot at the address below. > > https://dl.dropboxusercontent.com/u/8433654/sst.png > > > library(raster) > uri <- ' > https://oceandata.sci.gsfc.nasa.gov:443/opendap/MODISA/L3SMI/2016/001/A20160012016032.L3m_R32_SST_sst_9km.nc > ' > R <- raster::raster(uri, varname = 'sst') > > R > #class : RasterLayer > #dimensions : 180, 360, 64800 (nrow, ncol, ncell) > #resolution : 1, 1 (x, y) > #extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax) > #coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0 > #data source : in memory > #names : layer > #values : 1.482572e-05, 0.928 (min, max) > > plot(R) > lines(c(180, 100, 100, 180), c(80,80,0,0)) > lines(c(-180,-90,-90,-180), c(80,80,0,0)) > > I see that there is the nice raster::rotate() function to rotate raster > coordinates from [0,360] to [-180,180]. That would make extracting the > region super easy if the inverse were available. Is there an equivalent > way to implement the inverse or raster::rotate()? I could dig into the > source of raster::rotate() to see how to code one up, but I hate like heck > to reinvent the wheel. > > Thanks! > Ben > > > Ben Tupper > Bigelow Laboratory for Ocean Sciences > 60 Bigelow Drive, P.O. Box 380 > East Boothbay, Maine 04544 > http://www.bigelow.org > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Deal with multiple factorlevel in one grid square
maybe fasterize? Only on GitHub, and requires sf. (Mixing raster and sf is piece-meal, but very doable). https://github.com/ecohealthalliance/fasterize On Tue, 30 May 2017, 21:55 Miriam Püts <miriam.pu...@thuenen.de> wrote: > Hi all, > > @Pat: yes, that is almost what I need. If I would do it in ArcGIS I would > use Polygon to raster (with a second raster as reference) and then choose > majority. there a way to do it in R? If I use raster::extract() I would > extract data from the raster object for the locations of other spatial > data, but I need spatial data for the raster object... > > > Cheers, > Miriam > > <°)))>< >°)))>< >°)))>< >°)))>< > > Miriam Püts > Marine Lebende Ressourcen/ Marine Living Resources > Thünen-Institut für Seefischerei/ Thünen Institute of Sea fisheries > Palmaille 9 > 22767 Hamburg (Germany) > > Tel: +49 40 38905-105 > Mail: miriam.pu...@thuenen.de > > - Ursprüngliche Mail - > Von: "Alexander Herr" <alexander.h...@csiro.au> > An: "patrick schratz" <patrick.schr...@gmail.com>, mdsum...@gmail.com, > "miriam puets" <miriam.pu...@thuenen.de> > CC: r-sig-geo@r-project.org > Gesendet: Donnerstag, 25. Mai 2017 02:28:40 > Betreff: RE: [R-sig-Geo] Deal with multiple factorlevel in one grid square > > Hiya, > > Do you want a raster with attribute information (ie several attributes > from your shape file)? You can achieve this with a workflow that uses > raster::extract() to assigne unique IDs of polygon to raster cells > create a RAT for you raster > assign to the unique IDs of raster RAT the attributes of your polygon > unique IDs (many to one relationship) with something like merge() or > functions in libraries like diplyr or data.table > > Cheers > Herry > > -Original Message- > From: R-sig-Geo [mailto:r-sig-geo-boun...@r-project.org] On Behalf Of > Patrick Schratz > Sent: Wednesday, 24 May 2017 10:48 PM > To: Michael Sumner <mdsum...@gmail.com>; Miriam Püts < > miriam.pu...@thuenen.de> > Cc: r-sig-geo@r-project.org > Subject: Re: [R-sig-Geo] Deal with multiple factorlevel in one grid square > > Do you want to do what is called “zonal statistics” in ArcGIS with > “majority" option? > > You may check out this SO question and try the “mode” function within > raster::extract() - maybe it does what you need. > > (I’m also unsure if I understand the question correctly) > > Cheers, Pat > > PhD Student at Department of Geography - GIScience group > Friedrich-Schiller-University Jena, Germany > Tel.: +49-3641-9-48973 > Web: https://pat-s.github.io > > On 24. May 2017, 12:35 +0200, Miriam Püts <miriam.pu...@thuenen.de>, > wrote: > > Hi Mike, > > > > I will try to explain it a bit more in detail. Maybe it is easier to > understand if I start from the end. In the end I would like to have an > ASCII file to read into Ecospace, which has the same extensions and > coordinates as other files I already created. This ASCII should contain > information about the sediment type within each predefined cell. To create > this ASCII file I have a shape file with the polygons representing the > sediment type and my grid which I applied to other variables to have the > same extend. Now I would like to create a grid containing the information > on sediment. Here, per grid cell the sediment type which covers the most of > the cell should be defined and connected with the coordinates for the grid > cell. > > > > I hope this makes it more clear... > > > > > > <°)))>< >°)))>< >°)))>< >°)))>< > > > > Miriam Püts > > Marine Lebende Ressourcen/ Marine Living Resources Thünen-Institut für > > Seefischerei/ Thünen Institute of Sea fisheries Palmaille 9 > > 22767 Hamburg (Germany) > > > > Tel: +49 40 38905-105 > > Mail: miriam.pu...@thuenen.de > > > > > > Von: "Michael Sumner" <mdsum...@gmail.com > > An: "Miriam Püts" <miriam.pu...@thuenen.de>, r-sig-geo@r-project.org > > Gesendet: Mittwoch, 24. Mai 2017 11:55:14 > > Betreff: Re: [R-sig-Geo] Deal with multiple factorlevel in one grid > > square > > > > > > > > raster::extract(grid, poly, weights = TRUE) is a start. It returns a > list which is painful to deal with at first, but can be collected into one > data frame for standard summarizing. > > > > I'm still a bit confused about whether you want an estimate of a cell > overlap in a polygon or something else. > > > > Cheers, Mike > > On Wed, 24 May 2017, 17:12 Miriam Püts < [ mailto: >
Re: [R-sig-Geo] Recs for R tools that subset/tile raster bricks?
Do also see the raster vignettes, there is one on performance and using line- or block- based chunks for large jobs. Cheers, Mike On Thu, 25 May 2017, 08:32 Michael Sumner <mdsum...@gmail.com> wrote: > Try crop() (or more abstractly extract(raster, cells) once you've > determined which cells x-y are needed). > > Getting the most efficient way to do this depends on the source of the > brick. It's not possible to know without knowing details on your setup. The > file format, method of creation and dimensions of the brick and sizes of > your subset query can all affect the best ways to go. > > Happy to help but it might take a bit of conversation to figure out what's > best. If you can share the output of print on the brick anf the extent of a > subset that is a good start. > > Cheers, Mike > > On Thu, 25 May 2017, 05:50 Megan Maloney <malone...@appstate.edu> wrote: > >> Hello all, >> >> Does anyone know of a good R tool to subset/tile raster bricks? >> >> I'd like to break a raster brick into smaller pieces to process. I'm >> finding functions to subset or tile a single raster layer, but not for >> multi-layer raster bricks. >> >> The brick input is hyperspectral flightlines - very large, 1 m^2 >> resampling >> covering a watershed, 340 layers. My code currently takes in a raster >> brick, extracts values per cell, applies beta coefficients, and outputs a >> nitrogen value to a raster. >> >> I don't need the subsets/tiles to be a particular size or number, just >> smaller. I was warned against dropping a whole flightline into R. I'm new >> to >> R, so I'm not sure what limitations I'm running into except processing >> time. >> All advice is welcome. >> >> Thank you very much for your help. >> Megan >> >> [[alternative HTML version deleted]] >> >> ___ >> R-sig-Geo mailing list >> R-sig-Geo@r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo >> > -- > Dr. Michael Sumner > Software and Database Engineer > Australian Antarctic Division > 203 Channel Highway > Kingston Tasmania 7050 Australia > > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Recs for R tools that subset/tile raster bricks?
Try crop() (or more abstractly extract(raster, cells) once you've determined which cells x-y are needed). Getting the most efficient way to do this depends on the source of the brick. It's not possible to know without knowing details on your setup. The file format, method of creation and dimensions of the brick and sizes of your subset query can all affect the best ways to go. Happy to help but it might take a bit of conversation to figure out what's best. If you can share the output of print on the brick anf the extent of a subset that is a good start. Cheers, Mike On Thu, 25 May 2017, 05:50 Megan Maloney <malone...@appstate.edu> wrote: > Hello all, > > Does anyone know of a good R tool to subset/tile raster bricks? > > I'd like to break a raster brick into smaller pieces to process. I'm > finding functions to subset or tile a single raster layer, but not for > multi-layer raster bricks. > > The brick input is hyperspectral flightlines - very large, 1 m^2 resampling > covering a watershed, 340 layers. My code currently takes in a raster > brick, extracts values per cell, applies beta coefficients, and outputs a > nitrogen value to a raster. > > I don't need the subsets/tiles to be a particular size or number, just > smaller. I was warned against dropping a whole flightline into R. I'm new > to > R, so I'm not sure what limitations I'm running into except processing > time. > All advice is welcome. > > Thank you very much for your help. > Megan > > [[alternative HTML version deleted]] > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Re: [R-sig-Geo] Deal with multiple factorlevel in one grid square
raster::extract(grid, poly, weights = TRUE) is a start. It returns a list which is painful to deal with at first, but can be collected into one data frame for standard summarizing. I'm still a bit confused about whether you want an estimate of a cell overlap in a polygon or something else. Cheers, Mike On Wed, 24 May 2017, 17:12 Miriam Püts <miriam.pu...@thuenen.de> wrote: > Hi everyone, > > I have the following problem: I have a personalized grid and a shape file > with polygons representing sediment types. Now I would like to apply this > grid to the Polygons to identify the sediment type most common within each > grid. I tried it with rasterize, but here I can only chose last or first. > Die you have any suggestions how I might get the sediment type for each > raster cell? > > Thank you for your help! > > <°)))>< >°)))>< >°)))>< >°)))>< > > Miriam Püts > Marine Lebende Ressourcen/ Marine Living Resources > Thünen-Institut für Seefischerei/ Thünen Institute of Sea fisheries > Palmaille 9 > 22767 Hamburg (Germany) > > Tel: +49 40 38905-105 > Mail: miriam.pu...@thuenen.de > > ___ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo -- Dr. Michael Sumner Software and Database Engineer Australian Antarctic Division 203 Channel Highway Kingston Tasmania 7050 Australia [[alternative HTML version deleted]] ___ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
