Re: [Numpy-discussion] fast numpy i/o

2011-06-27 Thread Derek Homeier
On 27.06.2011, at 7:11PM, Nils Becker wrote: >>> Finally, the former Scientific.IO NetCDF interface is now part of >>> scipy.io, but I assume it only supports netCDF 3 (the documentation >>> is not specific about that). This might be the easiest option for a >>> portable data format (if Matlab sup

Re: [Numpy-discussion] fast numpy i/o

2011-06-27 Thread Nils Becker
Hi, >> Finally, the former Scientific.IO NetCDF interface is now part of >> scipy.io, but I assume it only supports netCDF 3 (the documentation >> is not specific about that). This might be the easiest option for a >> portable data format (if Matlab supports it). > Yes, it is NetCDF 3. In recent

Re: [Numpy-discussion] fast numpy i/o

2011-06-27 Thread Derek Homeier
On 27.06.2011, at 6:36PM, Robert Kern wrote: >> Some late comments on the note (I was a bit surprised that HDF5 installation >> seems to be a serious hurdle to many - maybe I've just been profiting from >> the fink build system for OS X here - but I also was not aware that the >> current netCDF

Re: [Numpy-discussion] fast numpy i/o

2011-06-27 Thread Robert Kern
On Mon, Jun 27, 2011 at 11:17, Derek Homeier wrote: > On 21.06.2011, at 8:35PM, Christopher Barker wrote: > >> Robert Kern wrote: >>> https://raw.github.com/numpy/numpy/master/doc/neps/npy-format.txt >> >> Just a note. From that doc: >> >> """ >>     HDF5 is a complicated format that more or less

Re: [Numpy-discussion] fast numpy i/o

2011-06-27 Thread Robert Kern
On Tue, Jun 21, 2011 at 13:35, Christopher Barker wrote: > Robert Kern wrote: >> https://raw.github.com/numpy/numpy/master/doc/neps/npy-format.txt > > Just a note. From that doc: > > """ >     HDF5 is a complicated format that more or less implements >     a hierarchical filesystem-in-a-file.  Thi

Re: [Numpy-discussion] fast numpy i/o

2011-06-27 Thread Derek Homeier
On 21.06.2011, at 8:35PM, Christopher Barker wrote: > Robert Kern wrote: >> https://raw.github.com/numpy/numpy/master/doc/neps/npy-format.txt > > Just a note. From that doc: > > """ > HDF5 is a complicated format that more or less implements > a hierarchical filesystem-in-a-file. This f

Re: [Numpy-discussion] fast numpy i/o

2011-06-21 Thread Christopher Barker
Robert Kern wrote: > https://raw.github.com/numpy/numpy/master/doc/neps/npy-format.txt Just a note. From that doc: """ HDF5 is a complicated format that more or less implements a hierarchical filesystem-in-a-file. This fact makes satisfying some of the Requirements difficult. To

Re: [Numpy-discussion] fast numpy i/o

2011-06-21 Thread Simon Lyngby Kokkendorff
Hi, I have been using h5py a lot (both on windows and Mac OSX) and can only recommend it- haven't tried the other options though Cheers, Simon On Tue, Jun 21, 2011 at 8:24 PM, Derek Homeier < de...@astro.physik.uni-goettingen.de> wrote: > On 21.06.2011, at 7:58PM, Neal Becker wrote: >

Re: [Numpy-discussion] fast numpy i/o

2011-06-21 Thread Derek Homeier
On 21.06.2011, at 7:58PM, Neal Becker wrote: > I think, in addition, that hdf5 is the only one that easily interoperates > with > matlab? > > speaking of hdf5, I see: > > pyhdf5io 0.7 - Python module containing high-level hdf5 load and save > functions. > h5py 2.0.0 - Read and write HDF5 fi

Re: [Numpy-discussion] fast numpy i/o

2011-06-21 Thread Christopher Barker
Neal Becker wrote: >> I'm wondering what are good choices for fast numpy array serialization? >> >> mmap: fast, but I guess not self-describing? >> hdf5: ? >> pickle: self-describing, but maybe not fast? >> others? > > I think, in addition, that hdf5 is the only one that easily interoperates > wi

Re: [Numpy-discussion] fast numpy i/o

2011-06-21 Thread Christopher Barker
Neal Becker wrote: > I'm wondering what are good choices for fast numpy array serialization? > > mmap: fast, but I guess not self-describing? > hdf5: ? Should be pretty fast, and self describing -- advantage of being a standard. Disadvantage is that it requires an hdf5 library, which can b a pa

Re: [Numpy-discussion] fast numpy i/o

2011-06-21 Thread Robert Kern
On Tue, Jun 21, 2011 at 12:49, Neal Becker wrote: > I'm wondering what are good choices for fast numpy array serialization? > > mmap: fast, but I guess not self-describing? > hdf5: ? > pickle: self-describing, but maybe not fast? > others? NPY: http://docs.scipy.org/doc/numpy/reference/generated/

Re: [Numpy-discussion] fast numpy i/o

2011-06-21 Thread Neal Becker
Neal Becker wrote: > I'm wondering what are good choices for fast numpy array serialization? > > mmap: fast, but I guess not self-describing? > hdf5: ? > pickle: self-describing, but maybe not fast? > others? I think, in addition, that hdf5 is the only one that easily interoperates with matlab?

[Numpy-discussion] fast numpy i/o

2011-06-21 Thread Neal Becker
I'm wondering what are good choices for fast numpy array serialization? mmap: fast, but I guess not self-describing? hdf5: ? pickle: self-describing, but maybe not fast? others? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.sc