[Numpy-discussion] Why does assert_array_almost_equal sometimes raise ValueError instead of AssertionError ?

2009-07-27 Thread David Cournapeau
Hi, In some cases, some of the testing functions assert_array_* raise a ValueError instead of AssertionError: np.testing.assert_array_almost_equal(np.array([1, 2, np.nan]), np.array([1, 2, 3])) # raises ValueError np.testing.assert_array_almost_equal(np.array([1, 2, np.inf]), np.array([1,

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread Kim Hansen
2009/7/24 David Cournapeau da...@ar.media.kyoto-u.ac.jp: Well, the questions has popped up a few times already, so I guess this is not so obvious :) 32 bits architecture fundamentally means that a pointer is 32 bits, so you can only address 2^32 different memory locations. The 2Gb instead of

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread David Cournapeau
Kim Hansen wrote: From my (admittedly ignorant) point of view it seems like an implementation detail for me, that there is a problem with some intermediate memory address space. Yes, it is an implementation detail, but as is 32 vs 64 bits :) My typical use case would be to access and

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread Kim Hansen
I think it would be quite complicated. One fundamental limitation of numpy is that it views a contiguous chunk of memory. You can't have one numpy array which is the union of two memory blocks with a hole in between, so if you slice every 1000 items, the underlying memory of the array still

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread Sebastian Haase
Is PyTables any option for you ? -- Sebastian Haase On Mon, Jul 27, 2009 at 12:37 PM, Kim Hansenslaun...@gmail.com wrote: I think it would be quite complicated. One fundamental limitation of numpy is that it views a contiguous chunk of memory. You can't have one numpy array which is the

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread Robin
On Mon, Jul 27, 2009 at 11:37 AM, Kim Hansenslaun...@gmail.com wrote: The machine is new and shiny with loads of processing power and many TB of HDD storage. I am however bound to 32 bits Win XP OS as there are some other costum made third-party and very expensive applications running on that

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread David Cournapeau
Kim Hansen wrote: The machine is new and shiny with loads of processing power and many TB of HDD storage. I am however bound to 32 bits Win XP OS as there are some other costum made third-party and very expensive applications running on that machine (which generate the large files I analyze),

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread Kim Hansen
You could think about using some kind of virtualisation - this is exactly the sort of situation where I find it really useful. You can run a 64 bit host OS, then have 32 bit XP as a 'guest' in VMware or Virtualbox or some other virtualisation software. With recent CPU's there is very little

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread Kim Hansen
2009/7/27 Sebastian Haase seb.ha...@gmail.com: Is PyTables any option for you ? -- Sebastian Haase That may indeed be something for me! I had heard the name before but I never realized exactly what it was. However, i have just seen their first tutorial video, and it seems like a very, very

Re: [Numpy-discussion] Why does assert_array_almost_equal sometimes raise ValueError instead of AssertionError ?

2009-07-27 Thread Charles R Harris
On Mon, Jul 27, 2009 at 1:00 AM, David Cournapeau da...@ar.media.kyoto-u.ac.jp wrote: Hi, In some cases, some of the testing functions assert_array_* raise a ValueError instead of AssertionError: np.testing.assert_array_almost_equal(np.array([1, 2, np.nan]), np.array([1, 2, 3])) #

Re: [Numpy-discussion] Not enough storage for memmap on 32 bit WinXP for accumulated file size above approx. 1 GB

2009-07-27 Thread Christopher Barker
Kim Hansen wrote: Yes, I see the problem in getting the same kind of reuse of objects using simple indexing. For my specific case, I will just allocate a new array as containing a copy of every 100th element and return this array. It will basically give me the same result as the original

[Numpy-discussion] Installed C libraries and using core math library

2009-07-27 Thread David Cournapeau
Hi, I have recently integrated my work on numpy.distutils to build so called installable C libraries, that is pure C libraries which can be installed and reused by 3rd parties. The general documentation is in the distutils section:

[Numpy-discussion] String manipulation summary

2009-07-27 Thread Christopher Barker
Hi all, When I first saws this problem: reading in a fixed-width text file as numbers, it struck me that you really should be able to do it, and do it well, with numpy by slicing character arrays. I got carried away, and worked out a number of ways to do it. Lastly was a method inspired by

Re: [Numpy-discussion] String manipulation summary

2009-07-27 Thread Chris Colbert
what machine spec are you using? Using your last function line2array5 WITH float conversion, i get the following timing on a mobile quad core extreme: In [24]: a = np.arange(100).astype(str).tostring() In [25]: a Out[25]:

Re: [Numpy-discussion] String manipulation summary

2009-07-27 Thread Christopher Barker
Chris Colbert wrote: what machine spec are you using? Dual 2Ghz PPC OS-X 10.4. Python2.5, numpy 1.3.0rc1 (hmm -- I should upgrade that!) Using your last function line2array5 WITH float conversion, i get the following timing on a mobile quad core extreme: In [24]: a =

Re: [Numpy-discussion] Test failures on FreeBSD buildbot

2009-07-27 Thread David Cournapeau
On Tue, Jul 28, 2009 at 8:44 AM, Charles R Harrischarlesr.har...@gmail.com wrote: ERROR: test_nan_items (test_utils.TestApproxEqual) -- Traceback (most recent call last): File

[Numpy-discussion] datetime Parsing Functions

2009-07-27 Thread Marty Fuhry
I'm a Summer of Code student working on the datetime implementation. The past few weeks I've been writing code to parse between dates and longs with a frequency. I'd like some feedback on my code before it gets integrated into the NumPy datetime branch: Clone URL:

Re: [Numpy-discussion] Test failures on FreeBSD buildbot

2009-07-27 Thread Charles R Harris
On Mon, Jul 27, 2009 at 9:37 PM, David Cournapeau courn...@gmail.comwrote: On Tue, Jul 28, 2009 at 8:44 AM, Charles R Harrischarlesr.har...@gmail.com wrote: ERROR: test_nan_items (test_utils.TestApproxEqual) -- Traceback

Re: [Numpy-discussion] Test failures on FreeBSD buildbot

2009-07-27 Thread David Cournapeau
Charles R Harris wrote: On Mon, Jul 27, 2009 at 9:37 PM, David Cournapeau courn...@gmail.com mailto:courn...@gmail.com wrote: On Tue, Jul 28, 2009 at 8:44 AM, Charles R Harrischarlesr.har...@gmail.com mailto:charlesr.har...@gmail.com wrote: ERROR: test_nan_items

Re: [Numpy-discussion] Test failures on FreeBSD buildbot

2009-07-27 Thread David Cournapeau
Charles R Harris wrote: I'd just look at the difference and see if it exceeded some fraction of the expected value. There is the problem of zero, which could be handled in the usual way as diff abserr + relerr. I think abserr would need to be a new keyword with a default value. Since the

Re: [Numpy-discussion] Test failures on FreeBSD buildbot

2009-07-27 Thread Charles R Harris
On Mon, Jul 27, 2009 at 10:48 PM, David Cournapeau da...@ar.media.kyoto-u.ac.jp wrote: Charles R Harris wrote: I'd just look at the difference and see if it exceeded some fraction of the expected value. There is the problem of zero, which could be handled in the usual way as diff