Re: [Numpy-discussion] Win64 build?

2008-12-17 Thread David Cournapeau
On Mon, Dec 15, 2008 at 1:26 PM, David Cournapeau
da...@ar.media.kyoto-u.ac.jp wrote:
 Christian Heimes wrote:
 David Cournapeau schrieb:

 Do you only need numpy or also scipy ? If you only need numpy, it is
 relatively straightforward because you don't need BLAS/LAPACK nor any
 fortran compiler. You should use the Visual Studio compiler, though: VS
 2005 for python 2.5 or VS 2008 for python 2.6 - mingw does not work well
 yet for 64 bits.


 The offical Windows builds of Python 2.5 are created with Visual C 7.1
 (also known as VS2003). You can compile an extension with VS 2005 but
 that will cause trouble.


 Hm, I may have got confused between the IDE and the compiler version.
 VS2003 cannot build 64 bits binaries, right ? So you need the
 Platform/Windows SDK - which corresponds to the compiler version 14 (VS
 2005) and not 13 (VS 2003), right ?

For the record, if anyone (including me) needs this info: I checked,
and python 2.5.2 on amd64 is indeed build by a compiler reporting MSC
1400 (VS 2005 serie). I don't think VS 2003 compiler is used at all,
actually - maybe the VS 2003 IDE can be set to use the SDK compilers,
though.

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Singular Matrix problem with Matplit lib in Numpy (Windows - AMD64)

2008-12-17 Thread George
David Cournapeau cournape at gmail.com writes:

 
 On Tue, Dec 9, 2008 at 12:50 AM, George Goussard gwg at emss.co.za wrote:
  Hello.
 
 
 
  I have been battling with the following error for the past week. The output
  from the terminal is:
 
 
 What does numpy.test() says ? Did you use an external blas/lapack when
 you built numpy for AMD64
 
 David
 


Hi David.

I accidentally created a new posting previously. I have spent the last month
trying to track down this bug. I am trying to compile Numpy and Matplotlib on
Windows XP 64-bit. I am using the Visual Studio 2005 compiler. Everything
compiles without a problem. However running matplotlib etc. gave me a lot of
problems:

1. The interaction was terrible. It didn't draw anything and the console had a
lot trace output with regard to singular matrices etc. Like you said, I am using
an external library called Intel MKL and I decided to swap this with AMD ACML.
Then the interaction was a lot better and no trace on the console of singular
matrices etc.

2. Using both libraries there are problems with the plotting. In both cases the
graphs are broken. It starts plotting the curve and then it stops with a section
of white space and then some more of the curve etc. The same with the grid lines
etc. In other words there is just something broken.

I have decided to pursue this bug. I would really like to get Numpy working on
AMD64. I ran the test you advised and the tests passed. However I have traced
the problem to the file lines.py of matplotlib. There in a function set_xdata
and set_ydata(also set_data) there is a line like x = np.asarray or y =
np.asarray. My data before that line is fine, but straight after the line is
executed the data is broken and garbage.

I have debugged some more but I am in deep (murky) waters, but I have also ran
out of ideas. If anybody has some more suggestions, please post them.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Singular Matrix problem with Matplitlib in Numpy (Windows - AMD64)

2008-12-17 Thread Michael Abshoff
George wrote:
 David Cournapeau cournape at gmail.com writes:

SNIP

Hi George,

 I have debugged some more but I am in deep (murky) waters, but I have also ran
 out of ideas. If anybody has some more suggestions, please post them.

Could you post a full example with additional version info that you are 
using? Ever since Sage upgraded to Matplotlib 0.98.3 I have been seeing 
issues with uninitilized values being used in certain code paths. This 
could be the source of potential trouble even though it doesn't seem to 
cause any observable trouble with gcc for example.

Cheers,

Michael

 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion
 

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Recent umath changes

2008-12-17 Thread Charles R Harris
On Tue, Dec 16, 2008 at 10:56 PM, David Cournapeau 
da...@ar.media.kyoto-u.ac.jp wrote:

 Charles R Harris wrote:
 
 
  On Tue, Dec 16, 2008 at 8:59 PM, David Cournapeau
  da...@ar.media.kyoto-u.ac.jp mailto:da...@ar.media.kyoto-u.ac.jp
  wrote:


 It does not work at the moment on windows at least :) But more
 essentially, I don't see why you declared those functions: can you
 explain me what was your intention, because I don't understand the
 rationale.


The declarations were for the SPARC. Originally I had them up in an ifdef up
top, but I got curious what different machines would do. They shouldn't
cause a problem unless something is pretty strange. The undefs I put where
they are for similar reasons, but there was a strong temptation to move them
into the if statement where they used to be. Let's say curiousity got the
best of me there. They shouldn't affect anything but macros and I didn't
want the function declarations do be interpreted as macros.



  You are, of course, free to break these builds again. However, I
  designated space at the top of the file for compiler/distro specific
  defines, I think you should use them, there is a reason other folks do.

 The problem is two folds:
   - by declaring functions everywhere in the code, you are effectively
 spreading toolchain specific oddities in the whole source file. This is
 not good, IMHO: those should be detected at configuration stage, and
 dealt in the source code using those informations. That's how every
 autoconf project does it. If a function is actually a macro, this should
 be detected at configuration.
  - declarations are toolchain specific; it is actually worse, it even
 depends on the compiler flags. It is at least the case with MS
 compilers. So there is no way to guarantee that your declaration matches
 the math runtime one (the compiler crash reported is exactly caused by
 this).


Worth knowing ;)  It works on the windows buildbot but that is running
python 2.4. Speaking of which, the BSD buildbot needs nose (I don't know
what happened to it),  the windows box is showing the same old permissions
problem, and one of the SPARC buildbots just times out unless you build
during the right time of day. We are just hobbling along at the moment.


  The macro undef could be moved but I preferred to generate an error if
  there was a conflict with the the standard c function prototypes.
 
  We can't use inline code for these functions as they are passed to the
  generic loops as function pointers.

 Yes, I believe this is another problem when declaring function: if we
 use say cosl, and cosl is an inline function in the runtime, by
 re-declaring it, you are telling the compiler that it is not inline
 anymore, so the compiler does not know anymore you can't take the
 address of cosl, unless it detects the mismatch between the runtime
 declaration and ours, and considers it as an error (I am not sure
 whether this is always an error with MS compilers; it may only be a
 warning on some versions - it is certainly not dealt in a gracious
 manner every time, since the linker crashes in some cases).


Sorry for the late reply, the network was down.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Singular Matrix problem with Matplitlib in Numpy (Windows - AMD64)

2008-12-17 Thread George Goussard
Hi Micheal.

I am going on vacation tomorrow. An example will have to wait until I am back,
but I can give some version information now:

Numpy is version 1.2.1, Matplotlib is version 0.98.5 and I am using
stock-standard(not the Enthought version or other distribution) Python 2.5.
My Python 2.5 I also compiled with MSVC 2005(AMD/em64t setting) because I can't
have dependecies on older crt libraries/dll's. The Intel MKL version is
10.1.0.018 and the AMD ACML version is 4.2.0. I am not using both at the same
time. I have first tried the Intel one and now I am using AMD's ACML as
explained in previous email. Everything is compiled using the AMD64/em64t
settings. I am not compiling on IA32 and IA64 etc. All the other necessary
version information(MSVC), I'll have to check when I am back at the office. 

I also use other libraries like PyQt(3.17.??) and SIP(version 4.7.7). Both of
them are the commercial versions and I also have commercial version of Qt 4.3.3.
Needless to say that my backend agg is Qt4Agg.

I have tested on Linux 64 bit (SuSe 10.?? commercial version not openSuSe) and
it worked. I also tested on Windows XP 32-bit and Linux 32-bit(SuSe 10.??) and
everything worked. 

Constructing and example won't be trivial but I'll try. The reason being thatI
embedded Python in an application and then I display graphs using SIP and PyQt.
The application was done using Qt4. Anyway, but I'll get back to you on that as
soon as I am finished with it.

Another interesting aspect is that in my application where I initially construct
the array using PyArray_SimpleNew, if I change this to, for example
PyArray_SimpleNewFromData then I get a completely different graph which is a
solid line(not the effect I described in the previous email) but it is
completely the wrong graph, with very small numbers(E-16 numbers). One thing
that also bothers me is that on Windows 32-bit. The default was not FORTRAN
arrays, but on Windows XP 64 bit the order and everything is FORTRAN default.
I ain't even using anything remotely to FORTRAN. I think the AMD ACML is
compiled using the Intel FORTRAN compiler, but will that effect it??

Anyway, I'll put an effort into constructing an example, but it will have to be
when I am back at the office from my vacation.

Cheers. Thanks.

George.




___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Singular Matrix problem with Matplit lib in Numpy (Windows - AMD64)

2008-12-17 Thread George Goussard
Hello David.

I am using the Intel MKL BLAS/LAPACK. I have replaced this with AMD's ACML
library. Now there is no exception raised due to a Singular matrix while
trying to move the legend(wiggling the graph). So, the graph is updated and
the interaction is fine(you can wiggle the graph and it updates, minimize,
maximeie etc.). But ... the legend is now only drawn sometimes and the graphs
are drawn with an intermittent line, as if the - - -  pattern was specified.
Something is still not right. I just can't seem to put my finger on it since
there are some many parties involved(numpy,matplotlib,python, ctypes etc.)

I also ran the numpy.test() with NUmpy that I compiled with AMD's ACML. The
results are included:

Running unit tests for numpy
NumPy version 1.2.1
Results of numpy.test()

NumPy is installed in C:\Development\Python\2_5_2\lib\site-packages\numpy
Python version 2.5.2 (r252:60911, Dec 12 2008, 08:38:07) [MSC v.1400 64 bit
(AMD64)]
nose version 0.10.4
Forcing
DISTUTILS_USE_SDK=1
...
.
...
..K..K...
...
.
...K...
..Ignoring MSVCCompiler instance has
 no attribute '_MSVCCompiler__root' (I think it is msvccompiler.py bug)
...S...
.
...
.
...
.
...
.
...
.
...
.
...
.
...

--
Ran 1592 tests in 10.704s

OK (KNOWNFAIL=3, SKIP=1)
nose.result.TextTestResult run=1592 errors=0 failures=0

Thanks.

George.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] genloadtxt : last call

2008-12-17 Thread Ryan May
Pierre GM wrote:
 Ryan,
 OK, I'll look into that. I won't have time to address it before this  
 next week, however. Option #2 looks like the best.

No hurries, I just want to make sure I raise any issues I see while the design 
is 
still up for change.

 In other news, I was considering renaming genloadtxt to genfromtxt,  
 and using ndfromtxt, mafromtxt, recfromtxt, recfromcsv for the  
 function names. That way, loadtxt is untouched.

+1
I know I've changed my tune here, but at this point it seems like there's so 
much 
more functionality here that calling it loadtxt would be a disservice to how 
much 
the new function can do (and how much work you've done).

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Unexpected MaskedArray behavior

2008-12-17 Thread Jim Vickroy

Ryan May wrote:

Pierre GM wrote:
  

On Dec 16, 2008, at 1:57 PM, Ryan May wrote:


I just noticed the following and I was kind of surprised:

  

a = ma.MaskedArray([1,2,3,4,5], mask=[False,True,True,False,False])
b = a*5
b


masked_array(data = [5 -- -- 20 25],
  mask = [False  True  True False False],
  fill_value=99)
  

b.data


array([ 5, 10, 15, 20, 25])

I was expecting that the underlying data wouldn't get modified while  
masked.  Is

this actual behavior expected?
  
Meh. Masked data shouldn't be trusted anyway, so I guess it doesn't  
really matter one way or the other.
But I tend to agree, it'd make more sense leave masked data untouched  
(or at least, reset them to their original value after the operation),  
which would mimic the behavior of gimp/photoshop.
Looks like there's a relatively easy fix. I need time to check whether  
it doesn't break anything elsewhere, nor that it slows things down too  
much. I won't have time to test all that before next week, though. In  
any case, that would be for 1.3.x, not for 1.2.x.

In the meantime, if you need the functionality, use something like
ma.where(a.mask,a,a*5)



I agree that masked values probably shouldn't be trusted, I was just surprised to 
see the behavior.  I just assumed that no operations were taking place on masked 
values.


Just to clarify what I was doing here: I had a masked array of data, where the 
mask was set by a variety of different masked values.  Later on in the code, 
after doing some unit conversions, I went back to look at the raw data to find 
points that had one particular masked value set.  Instead, I was surprised to see 
all of the masked values had changed and I could no longer find any of the 
special values in the data.


Ryan

  
Sorry for being dense about this, but I really do not understand why 
masked values should not be trusted.  If I apply a procedure to an array 
with elements designated as untouchable, I would expect that contract to 
be honored.  What am I missing here?


Thanks for your patience!
-- jv
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Unexpected MaskedArray behavior

2008-12-17 Thread Pierre GM

On Dec 17, 2008, at 12:13 PM, Jim Vickroy wrote:

 Sorry for being dense about this, but I really do not understand why  
 masked values should not be trusted.  If I apply a procedure to an  
 array with elements designated as untouchable, I would expect that  
 contract to be honored.  What am I missing here?

 Thanks for your patience!
 -- jv

Everything depends on your interpretation of masked data.  
Traditionally, masked data indicate invalid data, whatever the cause  
of the invalidity. Operations involving invalid data yield invalid  
data, hence the presence of a mask on the result. However, the value  
underneath the mask is still invalid, hence the statement don't trust  
masked values.
Interpreting a mask as a way to prevent some elements of an array to  
be processed (designating them as untouchable) is a bit of a stretch.  
Nevertheless, I agree that this behavior is not intuitive, so I'll  
check what I can do.


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Unexpected MaskedArray behavior

2008-12-17 Thread Ryan May
Pierre GM wrote:
 On Dec 16, 2008, at 1:57 PM, Ryan May wrote:
 I just noticed the following and I was kind of surprised:

 a = ma.MaskedArray([1,2,3,4,5], mask=[False,True,True,False,False])
 b = a*5
 b
 masked_array(data = [5 -- -- 20 25],
   mask = [False  True  True False False],
   fill_value=99)
 b.data
 array([ 5, 10, 15, 20, 25])

 I was expecting that the underlying data wouldn't get modified while  
 masked.  Is
 this actual behavior expected?
 
 Meh. Masked data shouldn't be trusted anyway, so I guess it doesn't  
 really matter one way or the other.
 But I tend to agree, it'd make more sense leave masked data untouched  
 (or at least, reset them to their original value after the operation),  
 which would mimic the behavior of gimp/photoshop.
 Looks like there's a relatively easy fix. I need time to check whether  
 it doesn't break anything elsewhere, nor that it slows things down too  
 much. I won't have time to test all that before next week, though. In  
 any case, that would be for 1.3.x, not for 1.2.x.
 In the meantime, if you need the functionality, use something like
 ma.where(a.mask,a,a*5)

I agree that masked values probably shouldn't be trusted, I was just surprised 
to 
see the behavior.  I just assumed that no operations were taking place on 
masked 
values.

Just to clarify what I was doing here: I had a masked array of data, where the 
mask was set by a variety of different masked values.  Later on in the code, 
after doing some unit conversions, I went back to look at the raw data to find 
points that had one particular masked value set.  Instead, I was surprised to 
see 
all of the masked values had changed and I could no longer find any of the 
special values in the data.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


[Numpy-discussion] immutable numpy arrays

2008-12-17 Thread Geoffrey Irving
Currently numpy arrays are either writable or unwritable, but
unwritable arrays can still be changed through other copies.  This
means that when a numpy array is passed into an interface that
requires immutability for safety reasons, a copy always has to be
made.

One way around this would be to add a NPY_IMMUTABLE flag signifying
that the contents of the array will never change through other copies.
 This flag would be in addition to the current NPY_WRITEABLE flag, so
it would be fully backwards compatible.  The flag would be propagated
along slices and views.

For example, a numpy array created from C code that guarantees
immutability would have the flag set.  If the array was passed back
into a function that required an immutable array, the code could check
the immutable flag and skip the copy.  This behavior could also be
used to implement safe copy-on-write semantics.

Making this more generally useful would probably require additional
flags to document whether a writeable copy of the array might exists
(something like NPY_LEAKED) in order to avoid copies for newly created
writeable arrays.

Has the issue of immutability been considered before?  It seems like a
basic NPY_IMMUTABLE flag would be fairly easy to add without backwards
compatibility issues, but the secondary features such as NPY_LEAKED
would be more complicated.

Thanks,
Geoffrey
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] immutable numpy arrays

2008-12-17 Thread Robert Kern
On Wed, Dec 17, 2008 at 15:52, Geoffrey Irving irv...@naml.us wrote:
 Currently numpy arrays are either writable or unwritable, but
 unwritable arrays can still be changed through other copies.  This
 means that when a numpy array is passed into an interface that
 requires immutability for safety reasons, a copy always has to be
 made.

 One way around this would be to add a NPY_IMMUTABLE flag signifying
 that the contents of the array will never change through other copies.

This is not possible to guarantee. With the __array_interface__, I can
make a numpy array point at any addressable memory without its
knowledge. We can even mutate immutable str objects, too.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] immutable numpy arrays

2008-12-17 Thread Geoffrey Irving
On Wed, Dec 17, 2008 at 2:24 PM, Robert Kern robert.k...@gmail.com wrote:
 On Wed, Dec 17, 2008 at 15:52, Geoffrey Irving irv...@naml.us wrote:
 Currently numpy arrays are either writable or unwritable, but
 unwritable arrays can still be changed through other copies.  This
 means that when a numpy array is passed into an interface that
 requires immutability for safety reasons, a copy always has to be
 made.

 One way around this would be to add a NPY_IMMUTABLE flag signifying
 that the contents of the array will never change through other copies.

 This is not possible to guarantee. With the __array_interface__, I can
 make a numpy array point at any addressable memory without its
 knowledge. We can even mutate immutable str objects, too.

In python __array_interface__ just returns a big integer representing
a pointer which can't be used for anything.  Well-behaved C code has
to be trusted to care when __array_interface__ marks its data as
unwriteable, so that shouldn't be a problem either.

Is there some other way that arbitrary python code could bypass the
NPY_WRITEABLE flag?

Geoffrey
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] error compiling umathmodule.c (numpy 1.3) on 64-bit windows xp

2008-12-17 Thread Lin Shao
 It is a bug in VS, but the problem is caused by buggy code in numpy,
 so this can be avoided. Incidentally, I was working on it yesterday,
 but went to bed before having fixed everything :)

That's good to know. Thank you for fixing it and let us know when it's
ready for test.

-lin
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] immutable numpy arrays

2008-12-17 Thread Robert Kern
On Wed, Dec 17, 2008 at 16:51, Geoffrey Irving irv...@naml.us wrote:
 On Wed, Dec 17, 2008 at 2:24 PM, Robert Kern robert.k...@gmail.com wrote:
 On Wed, Dec 17, 2008 at 15:52, Geoffrey Irving irv...@naml.us wrote:
 Currently numpy arrays are either writable or unwritable, but
 unwritable arrays can still be changed through other copies.  This
 means that when a numpy array is passed into an interface that
 requires immutability for safety reasons, a copy always has to be
 made.

 One way around this would be to add a NPY_IMMUTABLE flag signifying
 that the contents of the array will never change through other copies.

 This is not possible to guarantee. With the __array_interface__, I can
 make a numpy array point at any addressable memory without its
 knowledge. We can even mutate immutable str objects, too.

 In python __array_interface__ just returns a big integer representing
 a pointer which can't be used for anything.

I can (and do) *make* an array from Python given an
__array_interface__ with that pointer. See numpy/lib/stride_trick.py
in numpy 1.2 for an example.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] immutable numpy arrays

2008-12-17 Thread Geoffrey Irving
On Wed, Dec 17, 2008 at 3:34 PM, Robert Kern robert.k...@gmail.com wrote:
 On Wed, Dec 17, 2008 at 16:51, Geoffrey Irving irv...@naml.us wrote:
 On Wed, Dec 17, 2008 at 2:24 PM, Robert Kern robert.k...@gmail.com wrote:
 On Wed, Dec 17, 2008 at 15:52, Geoffrey Irving irv...@naml.us wrote:
 Currently numpy arrays are either writable or unwritable, but
 unwritable arrays can still be changed through other copies.  This
 means that when a numpy array is passed into an interface that
 requires immutability for safety reasons, a copy always has to be
 made.

 One way around this would be to add a NPY_IMMUTABLE flag signifying
 that the contents of the array will never change through other copies.

 This is not possible to guarantee. With the __array_interface__, I can
 make a numpy array point at any addressable memory without its
 knowledge. We can even mutate immutable str objects, too.

 In python __array_interface__ just returns a big integer representing
 a pointer which can't be used for anything.

 I can (and do) *make* an array from Python given an
 __array_interface__ with that pointer. See numpy/lib/stride_trick.py
 in numpy 1.2 for an example.

Ah.  Yes, that certainly precludes complete safety.

I don't think it precludes the usefulness of an immutable flag though,
just like it doesn't preclude the usefulness of the writeable flag.
The stride_tricks.py code is already well-behaved: it doesn't turn
unwriteable arrays into writeable arrays.  It certainly could, but
this is analogous to ctypes or untrusted C code.

Geoffrey
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] immutable numpy arrays

2008-12-17 Thread Robert Kern
On Wed, Dec 17, 2008 at 17:45, Geoffrey Irving irv...@naml.us wrote:
 On Wed, Dec 17, 2008 at 3:34 PM, Robert Kern robert.k...@gmail.com wrote:
 On Wed, Dec 17, 2008 at 16:51, Geoffrey Irving irv...@naml.us wrote:
 On Wed, Dec 17, 2008 at 2:24 PM, Robert Kern robert.k...@gmail.com wrote:
 On Wed, Dec 17, 2008 at 15:52, Geoffrey Irving irv...@naml.us wrote:
 Currently numpy arrays are either writable or unwritable, but
 unwritable arrays can still be changed through other copies.  This
 means that when a numpy array is passed into an interface that
 requires immutability for safety reasons, a copy always has to be
 made.

 One way around this would be to add a NPY_IMMUTABLE flag signifying
 that the contents of the array will never change through other copies.

 This is not possible to guarantee. With the __array_interface__, I can
 make a numpy array point at any addressable memory without its
 knowledge. We can even mutate immutable str objects, too.

 In python __array_interface__ just returns a big integer representing
 a pointer which can't be used for anything.

 I can (and do) *make* an array from Python given an
 __array_interface__ with that pointer. See numpy/lib/stride_trick.py
 in numpy 1.2 for an example.

 Ah.  Yes, that certainly precludes complete safety.

 I don't think it precludes the usefulness of an immutable flag though,
 just like it doesn't preclude the usefulness of the writeable flag.
 The stride_tricks.py code is already well-behaved: it doesn't turn
 unwriteable arrays into writeable arrays.  It certainly could, but
 this is analogous to ctypes or untrusted C code.

It just seems to me to be another complication that does not provide
any guarantees. You say Currently numpy arrays are either writable or
unwritable, but unwritable arrays can still be changed through other
copies. Adding an immutable flag would just change that to Currently
numpy arrays are either mutable or immutable, but immutable arrays can
still be changed through other copies. Basically, the writable flag
is intended to indicate your use case. It can be circumvented, but the
same methods of circumvention can be applied to any set of flags.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Recent umath changes

2008-12-17 Thread David Cournapeau
Charles R Harris wrote:

 The declarations were for the SPARC. Originally I had them up in an
 ifdef up top, but I got curious what different machines would do.

I still don't understand what exact problem they solve. Since the
declarations are put when HAVE_FOO is defined, the only problems I can
see are problem in the detection code or a platform bug (I seem to
remember for SPARC, this was a platform error, right ?). In either case,
it should be solved elsewhere (at worst, for platform specific, this
should be done within #if PLATFORM/#endif).

 They shouldn't cause a problem unless something is pretty strange.

They do; the default rule should be not to put any external declaration,
because they are heavily toolchain/platform specific. I removed a lot of
them from the old code when I refactored this code, and putting them
back almost totally alleviate my effort :) To quote python code itself
(pyport.h):

/**
Prototypes that are missing from the standard include files on some systems
(and possibly only some versions of such systems.)

Please be conservative with adding new ones, document them and enclose them
in platform-specific #ifdefs.
**/

 The undefs I put where they are for similar reasons, but there was a
 strong temptation to move them into the if statement where they used
 to be.

Could you be more specific ? I want to know the actual error they were
solving.

 Let's say curiousity got the best of me there. They shouldn't affect
 anything but macros and I didn't want the function declarations do be
 interpreted as macros.

Shouldn't affect is not good enough :) The default rule should be to
avoid relying at all on those distinctions, and only care when they
matter. Doing the other way around does not work, there alway be some
strange platform which will break most assumptions, as rationale as they
can be.


 Worth knowing ;)  It works on the windows buildbot but that is running
 python 2.4.

Ah, it is 2.4 ! I was wondering the exact combination. It does not work
with  the platform SDK 6.1 (which includes 64 bits compiler), and this
results in a compiler segfault. The problem is particularly pernicious,
since the segfaults is not seen directly, but put in a temp file which
itself causes problem because two processes try to access it... One of
the nicest build failure I have ever seen :)

 Speaking of which, the BSD buildbot needs nose (I don't know what
 happened to it),  the windows box is showing the same old permissions
 problem, and one of the SPARC buildbots just times out unless you
 build during the right time of day. We are just hobbling along at the
 moment.

Windows problems at least are not specific to the buildbot.


 Sorry for the late reply, the network was down.

No problem,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion