[Pytables-users] Copying huge chunks of CArrays without loading them into memory

Hello all, I have two CArrays, ca1 and ca2. ca1 is meant to hold all the information in ca2, plus a bit more. I want to do ca1[slice] = ca2[:] but slice is far too big to read ca2[:] into memory, then write it into ca1[slice]. Is there an easy way to do this in PyTables or do I need to copy the d

[Pytables-users] How to find total size of a VLArray node?

Hi all, Is there any way to figure out the total size of a VLArray node on disk? Thanks, ANand - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Mobl

[Pytables-users] Wondering if data can be recovered

Hi all, I seem to have lost a long-running simulation... it was writing out to an hdf5 archive, and when I try to open the archive after the simulation is over I get the following and several nodes are missing from Africa.S.db.group0: In [2]: Africa.S.db._group HDF5-DIAG: Error detected in HDF5

Re: [Pytables-users] trouble installing pytables 2.0.2 on Mac OS X 10.5

Hi all, On 21 Jan 2008, at 22:35, Maarten Sneep wrote: > Hi, > > I have trouble installing pytables 2.0.2 on Mac OS X, version 10.5.1 > on Intel. > > Here is the output of python setup.py build_ext --inplace > File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/ > python2.5/dist

Re: [Pytables-users] Best way to store sequences of arbitrary objects containing arrays

OK, that shouldn't be a problem. Thanks for the help! Anand On May 21, 2008, at 9:56 AM, Ivan Vilata i Balaguer wrote: > Anand Patil (el 2008-05-20 a les 22:54:10 +0100) va dir:: > >> On Tue, May 20, 2008 at 9:37 AM, Ivan Vilata i Balaguer <[EMAIL PROTECTED] >> >

Re: [Pytables-users] Best way to store sequences of arbitrary objects containing arrays

On Tue, May 20, 2008 at 9:37 AM, Ivan Vilata i Balaguer <[EMAIL PROTECTED]> wrote: > Then, if you have more than 4096 > nodes, you should be careful not to place them all in the same group to > avoid performance problems with the object tree. > Do you mean 4096 leaf nodes, or 4096 nodes in gener

[Pytables-users] Best way to store sequences of arbitrary objects containing arrays

Hi all, I'd like to store a long sequence of python objects with pytables. The only things I know about the objects are: - Their memory footprint is dominated by a big numpy array, and - The attribute name of the big array for each object is the same; it's x1.big_array, x2.big_array, etc. I

Re: [Pytables-users] Carabos Coop. V. is ceasing operations

Pytables has been a huge help to pymc also. Pymc produces lots of samples from certain probability distributions, some of which can be high-dimensional. David Huard wrote a pytables-based backend that compresses and saves the samples as they're created. The backend makes it much possible to

Re: [Pytables-users] Error message from hdf5

as suggested earlier. Anand Patil (el 2007-11-02 a les 15:14:34 -0700) va dir:: > > > Any hints as to how I can track down the source of this error? It's just > > displayed, no Python exceptions are raised and execution of my program > > continues. > > Could you ple

Re: [Pytables-users] Extracting from a large compressed binary datafile into a PyTables array

> Anand Patil (el 2007-10-31 a les 17:53:17 -0700) va dir:: > > > I have a file full of 32-bit floats, in binary format, compressed with > zip. > > I'd like to get it into a PyTables array, but this: > > > > Z = ZipFile('data_file.zi

[Pytables-users] Error message from hdf5

Hi all, Any hints as to how I can track down the source of this error? It's just displayed, no Python exceptions are raised and execution of my program continues. Thanks, Anand HDF5-DIAG: Error detected in HDF5 (1.8.0-beta3) thread 0: #000: H5Adeprec.c line 161 in H5Acreate(): unable to create

[Pytables-users] Extracting from a large compressed binary datafile into a PyTables array

le.createArray('/', 'data', numpy_array) won't work because I don't have enough memory for the intermediate stages. Is there an easy way to do this piece-by-piece or in a 'streaming' fashion? Thanks, Anand Patil --