Michael,
I like the idea of having the NetCDF files in a external repository.
I was thinking that it might be better to point the people to satellite
data at the different DAACs so that they can download the files
directly. That would work for the "obs4MIPs" program. I would feel
better about it as well, I have been worried to be told by some data
providers (ECMWF) that we are not authorized to distribute their
original data. I initially did not think about this when I checked in
my original code.
I just found out that ECMWF now allows people to download their data in
"NetCDF" instead of "GRIB" using Python [1]. I tried it before, but
could only retrieve GRIB data and did not want to mess with "Grads" ctl
files and CDMS2/CDAT package. So now, I could just create a script to
download the right files and rename them to the appropriate filenames
for obs4MIPs examples.
I would feel much better about this. Let me know what you think.
[1]
https://software.ecmwf.int/wiki/display/WEBAPI/Accessing+ECMWF+data+servers+in+batch
Denis
On 3/10/14 1:06 PM, Michael Joyce wrote:
Hi guys,
An unfortunate side effect of our export from SVN to Git is that we've
ended up with a rather bloated repository. We've had a large number of
binary files in our repo in the past and all of this has been rolled up
into a obnoxious ~500 MB pack file. I've been completely unable to clone
the repo on my home internet because it constantly times out and it's
painfully slow on my faster work connection.
To fix this problem I suggest we do the following:
- Remove all binary files from our repo and host them externally. For
example, NetCDF files can be downloaded when they're needed and cleaned up
afterwards (for tests or examples).
- Remove all the bloat from our pack file. I was digging through stuff
earlier and found a number of very large and outdated files in our pack
file (~300 MB NC file, internal JPL presentations/files from a long time
ago, etc.). We should be able to use [1] to help automate this for us,
although we can also take care of it on our own if need be.
Let me know what you guys think the best course of action is. That being
said, dealing with this sooner rather than later would be nice =D
[1] https://github.com/cmaitchison/git_diet
-- Joyce
--
-----------------------------------------------------
Denis Nadeau, (CSC)
NCCS (NASA Center for Climate Simulation)
NASA Goddard Space Flight Center
Mailcode 606.2
8800 Greenbelt Road
Greenbelt, MD 20771
Email: [email protected]
Phone: (301) 286-7286 Fax: 301.286.1634
-----------------------------------------------------
