Re: [Numpy-discussion] Loading a GB file into array

2007-11-30 Thread Martin Spacek
Kurt Smith wrote: You might try numpy.memmap -- others have had success with it for large files (32 bit should be able to handle a 1.3 GB file, AFAIK). Yeah, I looked into numpy.memmap. Two issues with that. I need to eliminate as much disk access as possible while my app is running. I'm

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

2007-11-30 Thread Robert Kern
Barry Wark wrote: Some remaining issues: - which SDK to build against. Leopard ships with a Python build against the 10.5 SDK. It would be much easier, at least initially, for us to produce binaries against the Leopard Python 2.5. I would prefer that we use the Python binary from

Re: [Numpy-discussion] display numpy array as image (Giorgio F. Gilestro)

2007-11-30 Thread Joe Harrington
I was misinformed about the status of numdisplay's pages. The package is available as both part of stsci_python and independently, and its (up-to-date) home page is here: http://stsdas.stsci.edu/numdisplay/ Googling numdisplay finds that page. My apologies to those inconvenienced by my prior

Re: [Numpy-discussion] Loading a GB file into array

2007-11-30 Thread Martin Spacek
Martin Spacek wrote: Would it be better to load the file one frame at a time, generating nframes arrays of shape (height, width), and sticking them consecutively in a python list? I just tried this, and it works. Looks like it's all in physical RAM (no disk thrashing on the 2GB machine),

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

2007-11-30 Thread Barry Wark
On 11/30/07, Robert Kern [EMAIL PROTECTED] wrote: Barry Wark wrote: Some remaining issues: - which SDK to build against. Leopard ships with a Python build against the 10.5 SDK. It would be much easier, at least initially, for us to produce binaries against the Leopard Python 2.5. I

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

2007-11-30 Thread Giorgio F. Gilestro
Even just a build of the last stable version will do it. Most people (especially those who don't want to go through the hassle of compiling) are going to be perfectly happy with a binary of the latest release. Thanks! Barry Wark wrote: Using the gfortran from http://r.research.att.com/tools/,

Re: [Numpy-discussion] swig numpy2carray converters

2007-11-30 Thread Sebastian Haase
On Nov 30, 2007 5:16 AM, Bill Spotz [EMAIL PROTECTED] wrote: I have just committed the latest version of numpy.i (a swig interface file for bridging between C arrays and numerical python) to the numpy svn repository. There are three relatively new features that are now supported: * It is

Re: [Numpy-discussion] documentation generator based on pyparsing

2007-11-30 Thread Robert Cimrman
Nils Wagner wrote: Thank you for your note. It works fine for me with python2.5. However python2.3 results in ./gendocs.py -m 'scipy.linsolve.umfpack' Traceback (most recent call last): File ./gendocs.py, line 261, in ? main() File ./gendocs.py, line 207, in main default

[Numpy-discussion] Loading a GB file into array

2007-11-30 Thread Martin Spacek
I need to load a 1.3GB binary file entirely into a single numpy.uint8 array. I've been using numpy.fromfile(), but for files 1.2GB on my win32 machine, I get a memory error. Actually, since I have several other python modules imported at the same time, including pygame, I get a pygame parachute

Re: [Numpy-discussion] documentation generator based on pyparsing

2007-11-30 Thread Nils Wagner
On Fri, 30 Nov 2007 09:48:09 +0100 Robert Cimrman [EMAIL PROTECTED] wrote: Nils Wagner wrote: Thank you for your note. It works fine for me with python2.5. However python2.3 results in ./gendocs.py -m 'scipy.linsolve.umfpack' Traceback (most recent call last): File ./gendocs.py, line

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

2007-11-30 Thread Robert Kern
Christopher Barker wrote: Robert Kern wrote: Official binaries intended for distribution from scipy.org or scipy.sf.net should not be linked against FFTW or UMFPACK since they are GPLed. Does that apply to binaries put up on pythonmac.org? It would be nice to have a complete version

Re: [Numpy-discussion] Loading a GB file into array

2007-11-30 Thread Ivan Vilata i Balaguer
Martin Spacek (el 2007-11-30 a les 00:47:41 -0800) va dir:: [...] I find that if I load the file in two pieces into two arrays, say 1GB and 0.3GB respectively, I can avoid the memory error. So it seems that it's not that windows can't allocate the memory, just that it can't allocate enough

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

2007-11-30 Thread Barry Wark
On 11/29/07, David Cournapeau [EMAIL PROTECTED] wrote: Barry Wark wrote: Using the gfortran from http://r.research.att.com/tools/, it's trivial to build a universal build from source. The instructions on scipy.org won't lead you astray. I will ask around at work. Perhaps we can start

Re: [Numpy-discussion] Loading a GB file into array

2007-11-30 Thread Bryan Cole
Well, one thing you could do is dump your data into a PyTables_ ``CArray`` dataset, which you may afterwards access as if its was a NumPy array to get slices which are actually NumPy arrays. PyTables datasets have no problem in working with datasets exceeding memory size. For instance::

Re: [Numpy-discussion] scipy binary for macosx tiger on ppc

2007-11-30 Thread Christopher Barker
Robert Kern wrote: I would prefer that we use the Python binary from www.python.org. That should work on 10.3.9+. +1 -- there have always been enough issue with Apple;'s python that it's best to just use another version -- one binary for 10.3.9 is the way to go. Official binaries intended