On 11/26/2011 9:14 AM, John Caron wrote:
Im intending to incorporate the netcdf-4 C library into the netcdf-java library using JNI. Im hoping to have something working in the next few months, but we'll see. This will be an optional component, and will obviously make portability an issue.
Good idea, none the less.
If you want to use Python, probably the one to use is Jeff Whittaker's at http://code.google.com/p/netcdf4-python/, which is also an interface to the netcdf-4 C library.
yes -- I think that's the best option for Python -- it's nicely done.
I think its time to start using netcdf-4 for large collections of point data which need to be compressed. Instead of first making a standard, we need to try out the possibilities and see how it performs.
That may be true, time to move on eventually! However, you can use netcdf4 for compression, but stille use a netcdf3 compatible data model, so I'd like to see netcdf4-only features used only if they really are necessary to get the data model we need.
I think you want to use Structures, as well as multiple unlimited dimensions. With netcdf, we dont need the ragged array mecahnism - thats only needed to overcome the limitations of the classic model.
Can you tell us more? how do you express a "ragged_array" in netcdf 4? Variable length user-defined types, maybe?
This is all a bit frustrating, as we've had a fair bit of discussion, and I though had settled on ragged arrays, and I don't think anyone said (this would all be so much easier in netcdf 4)
Ute: I'm a bit confused -- was your 11 GB file a result of using the ragged array approach, or of using the rectangular array with LOTS of empty values approach?
I don't think compression is the answer to the problem of how to store what is naturally a ragged array -- partly because it simply doesn't appeal to me aesthetically, but also because it hides, and moves the problem -- the tools really should understand and be able to work with the fact that the number oar particles is not the same at all times, and we don't want to have the client apps to deal with a lot of empty arrays either.
Note that it seems netcdf4 "groups" could be handy for dealing with multiple "spills".
More soon on our current solution. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [email protected] _______________________________________________ CF-metadata mailing list [email protected] http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
