Yes, you need to set context with session. I could confirm that PO.DAAC OPeNDAP server requires session. Thus, R ncdf4 package needs to do something extra like pydap example does.
-----Original Message----- From: Ben Tupper <btup...@bigelow.org> Sent: Thursday, April 14, 2022 7:14 AM To: Joe Lee <hyok...@hdfgroup.org> Cc: r-sig-geo@r-project.org Subject: Re: [R-sig-Geo] Earthdata CMR access Well, ncdf4 is a pretty good with OPeNDAP (which is one reason why we love it!) Below is an example using your handy testing resource. Hmmm. In your example using pydap, you establish a (credentialed?) session, and then open the OPeNDAP resource within that context? I don't think I have seen that before in R, at least not for OPeNDAP. I'll poke around and see if I can find something comparable. ### Start library(ncdf4) url <- "https://eosdap.hdfgroup.org:8080/opendap/data/hdf5/grid_1_2d.h5"; x <- nc_open(url) library(ncdf4) url <- "https://eosdap.hdfgroup.org:8080/opendap/data/hdf5/grid_1_2d.h5"; x <- nc_open(url) temp <- ncvar_get(x, varid = "temperature", start = c(2,1), count = c(2,3)) temp # [,1] [,2] [,3] # [1,] 10 11 12 # [2,] 10 11 12 ### End On Wed, Apr 13, 2022 at 10:25 PM Joe Lee <hyok...@hdfgroup.org> wrote: > > Wow, what a great start! > > > > Unlike GES DISC OPeNDAP server, > > I think the PODAAC OPeNDAP server that you’d like to access > > doesn’t require Earthdata Login credentials. > > That eliminates a lot of complications. > > > > My suspicion is that your R ncdf4 package is not built with OPeNDAP support. > > Can you check if your R ndf4 package work with our demo OPeNDAP server [1]? > > If not, you may want to install ncdf4 with your own netCDF-4 library that > enabled OPeNDAP support. > > > > Once [1] works with your R ncdf4, > > I think my posting [2] can help you to redirect URLs properly > > for NASA credential-enabled OPeNDAP servers. > > > > Good luck and thanks for sharing your code on GitHub! > > > > > > [1] > https://eosdap.hdfgroup.org:8080/opendap/data/hdf5/grid_1_2d.h5.dmr.ht > ml > > [2] https://hdfeos.org/forums/showthread.php?t=821#4 > > > > From: Ben Tupper <btup...@bigelow.org> > Sent: Wednesday, April 13, 2022 8:59 PM > To: Joe Lee <hyok...@hdfgroup.org> > Cc: r-sig-geo@r-project.org > Subject: Re: [R-sig-Geo] Earthdata CMR access > > > > What I have can search for S3 resources, but I happen to focus on OPeNDAP > instead. > > As far as I can tell, the ncdf4 (and RNetCDF) packages don't provide > utilities for establishing credentialed opendap connections. That is handled > by sort of a third party or some kind of wrapper. Without credentials I can > search, but without credentials I cannot open a connection to the opendap > resources the search yields. > > If I am understanding the error message shown at the bottom of the README > (see below), the server is receiving the request but for lack of credentials > is refusing access. I'm stumped. > > https://github.com/BigelowLab/cmr > > > > On Wed, Apr 13, 2022 at 2:18 PM Joe Lee <hyok...@hdfgroup.org> wrote: > > > > Yes, that's the hard part that I'm referring to. > > > > Once you pass that step, you may also want to look at another > > example in MATLAB [1] to access PODAAC data on S3 using the temporary token. > > > > I'd really appreciate your effort if you can share a working R-code > > project that can both search with CMR and then access NASA data on S3. > > > > Regards, > > > > [1] > > https://www.hdfeos.org/zoo/PODAAC/ATM_SURFACE_TEMP_HUM_WIND_PRES_day > > _mean_ECCO_V4r4_nc_c.m > > > > > > -----Original Message----- > > From: Ben Tupper <btup...@bigelow.org> > > Sent: Wednesday, April 13, 2022 1:09 PM > > To: Joe Lee <hyok...@hdfgroup.org> > > Cc: r-sig-geo@r-project.org > > Subject: Re: [R-sig-Geo] Earthdata CMR access > > > > Hi, > > > > I was planning to use the httr package or the newish httr2 package which > > should be able to handle the transactions. Once a resource is tracked down > > via search, I plan to use the ncdf4 package to access. > > > > I can add headers to the http requests, so presumably that can include > > tokens. > > > > But I am finding even the simple stuff like using the API to > > generate a token, as shown here > > https://wiki.earthdata.nasa.gov/display/EL/How+to+Generate+a+User+To > > ken > > , to be challenging. Thud! > > > > Ben > > > > httr: https://CRAN.R-project.org/package=httr > > httr2: https://CRAN.R-project.org/package=httr2 > > ncdf4: https://CRAN.R-project.org/package=ncdf4 > > > > On Wed, Apr 13, 2022 at 1:03 PM Joe Lee <hyok...@hdfgroup.org> wrote: > > > > > > Hi, > > > > > > I'm also interested in the solution that you're seeking since I've > > > written a Python-equivalent CMR+OPeNDAP workflow a very long time ago [1]. > > > > > > It's both very tricky and difficult nowadays because NASA tries to > > > constrain AWS egress cost via Earthdata Login authentication. > > > > > > Does R provide a cURL-like library + a temporary token-based > > > authentication? > > > > > > [1] > > > https://earthdata.nasa.gov/collaborate/open-data-services-and-soft > > > ware /api/opendap/gesdisc-airs-opendap-cmr > > > > > > > > > -----Original Message----- > > > From: R-sig-Geo <r-sig-geo-boun...@r-project.org> On Behalf Of Ben > > > Tupper > > > Sent: Wednesday, April 13, 2022 11:41 AM > > > To: r-sig-geo <r-sig-geo@r-project.org> > > > Subject: [R-sig-Geo] Earthdata CMR access > > > > > > Hello, > > > > > > I am interested in accessing physical oceanography data from > > > PO.DAAC ( > > > https://earthdata.nasa.gov/eosdis/daacs/podaac) from within R > > > using opendap. NASA is moving its data offerings to the cloud, > > > and it is adopting Common Metadata Repositories ("CMR", > > > https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html) for > > > searching. CMR searching is in lieu of searching THREDDS catalogs. > > > > > > I'm finding the on-ramp to using CMR tricker than I had expected given > > > the sales pitch behind CMR (or maybe the wheels have come off my car!) I > > > am wondering if anyone in this community has waded into using R-based CMR > > > to access cloud-based geospatial data, and, if so, might be willing to > > > share some insights or provide guidance? > > > > > > Regards, > > > Ben > > > > > > -- > > > Ben Tupper (he/him) > > > Bigelow Laboratory for Ocean Science East Boothbay, Maine > > > http://www.bigelow.org/ https://eco.bigelow.org > > > > > > [[alternative HTML version deleted]] > > > > > > _______________________________________________ > > > R-sig-Geo mailing list > > > R-sig-Geo@r-project.org > > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > > > _______________________________________________ > > > R-sig-Geo mailing list > > > R-sig-Geo@r-project.org > > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > > > > > > -- > > Ben Tupper (he/him) > > Bigelow Laboratory for Ocean Science East Boothbay, Maine > > http://www.bigelow.org/ https://eco.bigelow.org > > > > -- > Ben Tupper (he/him) > Bigelow Laboratory for Ocean Science > East Boothbay, Maine > http://www.bigelow.org/ > https://eco.bigelow.org > > -- > > Ben Tupper Bigelow Laboratory for Ocean Sciences > https://www.bigelow.org/ Tick forecast https://eco.bigelow.org/ -- Ben Tupper (he/him) Bigelow Laboratory for Ocean Science East Boothbay, Maine http://www.bigelow.org/ https://eco.bigelow.org _______________________________________________ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo
