Re: [Numpy-discussion] Memory usage of numpy-arrays

2010-07-08 Thread Bruce Southey
On 07/08/2010 08:52 AM, Wes McKinney wrote: On Thu, Jul 8, 2010 at 9:26 AM, Hannes Bretschneider hannes.bretschnei...@wiwi.hu-berlin.de wrote: Dear NumPy developers, I have to process some big data files with high-frequency financial data. I am trying to load a delimited text file

Re: [Numpy-discussion] Memory usage of numpy-arrays

2010-07-08 Thread Hannes Bretschneider
Sebastian Haase seb.haase at gmail.com writes: I would expect a 700MB text file translate into less than 200MB of data - assuming that you are talking about decimal numbers (maybe total of 10 digits each + spaces) and saving as float32 binary. So the problem would only be the loading in -

[Numpy-discussion] memory usage of numpy arrays

2010-03-04 Thread kaby
Hi. I am using numpy arrays and when constructing an array I get a cannot allocate memory for thread-local data: ABORT The array i'm constructing is zeros((numVars, 2, numVars, 2), dtype=float) Where numVars is at about 2000. I was expecting the memory usage to be

Re: [Numpy-discussion] memory usage of numpy arrays

2010-03-04 Thread Robert Kern
On Thu, Mar 4, 2010 at 19:10, kaby kackvo...@gmail.com wrote: Hi. I am using numpy arrays and when constructing an array I get a  cannot allocate memory for thread-local data: ABORT The array i'm constructing is zeros((numVars, 2, numVars, 2), dtype=float) Where numVars is at about 2000. I