Re: [Numpy-discussion] masked ufuncs in C: on github
Charles R Harris wrote:
On Fri, May 15, 2009 at 7:48 PM, Eric Firing efir...@hawaii.edu
mailto:efir...@hawaii.edu wrote:
http://www.mail-archive.com/numpy-discussion@scipy.org/msg17595.html
Prompted by the thread above, I decided to see what it would take to
implement ufuncs with masking in C. I described the result here:
http://www.mail-archive.com/numpy-discussion@scipy.org/msg17698.html
Now I am starting a new thread. The present state of the work is now in
github: http://github.com/efiring/numpy-work/tree/cfastma
I don't want to do any more until I have gotten some feedback from core
developers. (And I would be delighted if someone wants to help with
this, or take it over.)
Chuck,
Thanks very much for the quick response.
Here the if ... continue needs to follow the declaration:
if (*mp1) continue;
float in1 = *(float *)ip1;
float in2 = *(float *)ip2;
*(float *)op1 = f(in1, in2);
I was surprised to see the declarations inside the loop in the first
place (this certainly is not ANSI-C), and I was also pleasantly
surprised that letting them be after the conditional didn't seem to
bother the compiler at all. Maybe that is a gcc extension.
I think this would be better as
if (!(*mp1)) {
float in1 = *(float *)ip1;
float in2 = *(float *)ip2;
*(float *)op1 = f(in1, in2);
}
I agree, and I thought of that originally--I think I did it with
continue because it was easier to type it in, and it reduced the
difference relative to the non-masked form.
But since this is actually a ternary function, you could define new
functions, something like
double npy_add_m(double a, double b, double mask)
{
if (!mask) {
return a + b;
else {
return a;
}
}
And use the currently existing loops. Well, you would have to add one
for ternary functions.
That would incur the overhead of an extra function call for each
element; I suspect it would slow it down a lot. My motivation is to make
masked array overhead negligible, at least for medium to large arrays.
Also your suggestion above does not handle the case where an output
argument is supplied; it would modify the output under the mask.
Question, what about reduce? I don't think it is defined defined for
ternary functions. Apart from reduce, why not just add, you already have
the mask to tell you which results are invalid.
You mean just do the operation and ignore the results under the mask?
This is the way Pierre originally did it, if I remember correctly, but
fairly recently people started objecting that they didn't want to
disturb values in an output argument under a mask. So now ma jumps
through hoops to satisfy this requirement, and it is consequently slow.
ufunc methods like reduce are supported only for the binary ops with one
output, so they are automatically unavailable for the masked versions.
To get around this would require subclassing the ufunc to make a masked
version. This is probably the best way to go, but I suspect it is much
more complicated than I can handle in the amount of time I can spend.
So maybe my proposed masked ufuncs are a slight abuse of the ufunc
concept, or at least its present implementation. Unary functions with a
mask, which I have not yet tried to implement, would actually be binary,
so they would have reduce etc. methods that would not make any sense.
Is there a way to disable (remove) the methods in this case?
Eric
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] masked ufuncs in C: on github
On Sat, May 16, 2009 at 03:02, Eric Firing efir...@hawaii.edu wrote: Charles R Harris wrote: Here the if ... continue needs to follow the declaration: if (*mp1) continue; float in1 = *(float *)ip1; float in2 = *(float *)ip2; *(float *)op1 = f(in1, in2); I was surprised to see the declarations inside the loop in the first place (this certainly is not ANSI-C), and I was also pleasantly surprised that letting them be after the conditional didn't seem to bother the compiler at all. Maybe that is a gcc extension. I believe they are a part of C99. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] masked ufuncs in C: on github
2009/5/16 Robert Kern robert.k...@gmail.com: On Sat, May 16, 2009 at 03:02, Eric Firing efir...@hawaii.edu wrote: Charles R Harris wrote: Here the if ... continue needs to follow the declaration: if (*mp1) continue; float in1 = *(float *)ip1; float in2 = *(float *)ip2; *(float *)op1 = f(in1, in2); I was surprised to see the declarations inside the loop in the first place (this certainly is not ANSI-C), and I was also pleasantly surprised that letting them be after the conditional didn't seem to bother the compiler at all. Maybe that is a gcc extension. I believe they are a part of C99. Exactly (so not supported by Visual Studio). Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Question about slicing
Hi, I am just starting with numpy, pyhton and related others. I work on image processing basically. Now, my question is: what is the expected behaviour when slicing a view of an array? The following example might give some background on what I tried to do and the results obatined (which I don't understand): I read an image with PIL but, for whatever reason (different conventions I suppose), it comes upside down. This doesn't change when (I don't know the exact term for this) transforming the image to ndarray with 'array(img)'. I don't feel comfortable working with upside down images, so this had to be fixed. I tried to be smart and avoid copying the whole image: aimg = array(img)[::-1] and it worked!, but I am interested actually in sub-regions of this image, so the next I did was: roi = aimg[10:20,45:50,:] And to my surprise the result was like if I was slicing the original, upside down, image instead of aimg. Can someone explain me what's going on here? I searched and looked at the documentation but I couldn't find an answer. Maybe I am not looking properly. Is the only way to turn the image to perform a copy? Thanks, Jorge ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about slicing
Hi Jorge, roi = aimg[10:20,45:50,:] are you working with 3-D images? I didn't know PIL was able to handle 3D images. I wasn't able to reproduce the behavior you observed with a simple example: In [20]: base = np.arange(25).reshape((5,5)) In [21]: base Out[21]: array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24]]) In [22]: flip = base[::-1] In [23]: flip Out[23]: array([[20, 21, 22, 23, 24], [15, 16, 17, 18, 19], [10, 11, 12, 13, 14], [ 5, 6, 7, 8, 9], [ 0, 1, 2, 3, 4]]) In [24]: flip[2:4,2:4] Out[24]: array([[12, 13], [ 7, 8]]) which is what you expect... I also tried the same manipulations as you do starting from a PIL image object, but I also got what I expected (and my image was not flipped vertically by PIL or when transformed into an array). It is quite weird BTW that your images are flipped. How do you visualize PIL image (Image.Image.show?)and arrays (pylab.imshow?) ? Hope someone can help you more than I did :D Cheers, Emmanuelle On Sat, May 16, 2009 at 08:42:50AM +, jorgesmbox...@yahoo.es wrote: Hi, I am just starting with numpy, pyhton and related others. I work on image processing basically. Now, my question is: what is the expected behaviour when slicing a view of an array? The following example might give some background on what I tried to do and the results obatined (which I don't understand): I read an image with PIL but, for whatever reason (different conventions I suppose), it comes upside down. This doesn't change when (I don't know the exact term for this) transforming the image to ndarray with 'array(img)'. I don't feel comfortable working with upside down images, so this had to be fixed. I tried to be smart and avoid copying the whole image: aimg = array(img)[::-1] and it worked!, but I am interested actually in sub-regions of this image, so the next I did was: roi = aimg[10:20,45:50,:] And to my surprise the result was like if I was slicing the original, upside down, image instead of aimg. Can someone explain me what's going on here? I searched and looked at the documentation but I couldn't find an answer. Maybe I am not looking properly. Is the only way to turn the image to perform a copy? Thanks, Jorge ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about slicing
Sat, 16 May 2009 08:42:50 +, jorgesmbox-ml wrote:
[clip]
I don't feel comfortable working with upside down images, so this had to
be fixed. I tried to be smart and avoid copying the whole image:
aimg = array(img)[::-1]
Note that here a copy is made. You can use `asarray` instead of `array`
if you want to avoid making a copy.
and it worked!, but I am interested actually in sub-regions of this
image, so the next I did was:
roi = aimg[10:20,45:50,:]
And to my surprise the result was like if I was slicing the original,
upside down, image instead of aimg. Can someone explain me what's going
on here?
Sounds impossible, and I don't see this:
In [1]: import Image
In [2]: img = Image.open('foo.png')
In [3]: aimg = array(img)
In [4]: imshow(aimg)
Out[4]: matplotlib.image.AxesImage object at 0x9a6ecec
In [5]: imshow(aimg[10:320,5:150])
Out[5]: matplotlib.image.AxesImage object at 0x9f1db2c
The image is here right-side up, both in full and the slice (since imshow
flips it). Also,
In [6]: aimg = array(img)[::-1]
In [7]: imshow(aimg[10:320,5:150])
Out[7]: matplotlib.image.AxesImage object at 0xa007eac
Now, the image is upside down, both in full and in the slice.
I think you should re-check that you are doing what you think you are
doing. Preparing a self-contained code example could help here, at least
this would make pinpointing where the error is more easy.
--
Pauli Virtanen
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Indexing with callables (was: Yorick-like functionality)
Fri, 15 May 2009 16:09:08 -0400, David Huard wrote: Can this indexing syntax do things that are otherwise awkward with the current syntax ? Otherwise, I'm not warm to the idea of making indexing more complex than it is. I think the indexing with callables is more syntax sugar for nested `func(v, axis=n)` than anything else. It may be more useful interactive use than calling the functions, though. Compare: x[:,sum,mean] mean(sum(x, axis=1), axis=1) It might be useful also for broadcasting functions operating on 1D vectors to the whole array, but here the semantics start getting muddier. [clip] getv drop_last append_one zcen These, apart from zcen, were just some demo functions I pulled out from nowhere. The actual list of Yorick functions relevant here appears to be here: http://yorick.sourceforge.net/manual/yorick_46.php#SEC46 http://yorick.sourceforge.net/manual/yorick_47.php#SEC47 I must say that I don't see many functions missing in Numpy... David (Strozzi): are these the functions you meant? Are there more? -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] masked ufuncs in C: on github
Eric Firing wrote: That would incur the overhead of an extra function call for each element; I suspect it would slow it down a lot. My motivation is to make masked array overhead negligible, at least for medium to large arrays. You can use inline in that case - starting with numpy 1.3.0, inline can be used in C code in a portable way (it is a macro which points to compiler specific inline if the C99 inline is not available so it works even on broken compilers). cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] minimal numpy ?
David Cournapeau wrote: Robert wrote: for use in binary distribution where I need only basics and fast startup/low memory footprint, I try to isolate the minimal ndarray type and what I need.. [..] I think you need at least umath to make this work: when doing import numpy.core.multiarray, you pull out the whole numpy (because import foo.bar induces import foo I believe), whereas import multiarray just imports the multiarray C extension. So my suggestion would be to modify numpy such as you can do import numpy after having removed most directories inside numpy. The big ones are distutils and f2py, which should already save 2.5 Mb and are not used at all in numpy itself. IIRC, the only problematic package is numpy.lib (we import numpy.lib in numpy.core IIRC). Did like this - keeping a /numpy/core folder structure. In attachment is a README-minimal-numpy.txt Maybe thats interesting for many users / inclusion somewhere in the docs. Result is: some 300kB compressed. And startup very fast. Strange: most imports in the package are relative - which is good for (flat) repackaging. Just one absolutue from numpy.core.multiarray import ... in a py file. Yet the 2 remaining DLL's obviously contain absolute imports of each other. Just because of that it is not possible to have the minimal numpy in a separate package folder with other name (without recompiling), but one needs to rename/remove the original numpy from the PYTHONPATH :-( Maybe the absolute imports could be removed out of the DLLs in future. By #ifdef or so in the C code the newer Pythons also can be forced to do precisely relative import. Robert HOW TO create a minimal numpy ? (for fast import or freezing) == rk 2009-05-16 * Make a copy of the original numpy folder tree * ( Rename the original tree / remove from sys.path ) * Keep only those files/folders: numpy/ -rw-rw-rw- 1 user group 82 May 16 10:58 __init__.py -rw-rw-rw- 1 user group 593 Apr 28 21:16 version.py numpy/core/ -rw-rw-rw- 1 user group 961 May 16 11:15 __init__.py -rw-rw-rw- 1 user group8938 Apr 28 21:16 _internal.py -rw-rw-rw- 1 user group 16814 May 16 11:31 arrayprint.py -rw-rw-rw- 1 user group 61754 Apr 28 21:16 fromnumeric.py (optional) -rw-rw-rw- 1 user group4635 Apr 28 21:16 info.py -rw-rw-rw- 1 user group 505132 Apr 28 21:16 multiarray.pyd (.so) -rw-rw-rw- 1 user group 56471 May 12 22:12 numeric.py -rw-rw-rw- 1 user group 20774 May 11 23:48 numerictypes.py -rw-rw-rw- 1 user group 318670 Apr 28 21:16 umath.pyd (.so) (about 300kB compressed on win32) * change numpy/__init__.py to simply: == numpy/__init__.py a minimal numpy - only essential stuff of the core #$1 import core from core import * == end of numpy/__init__.py = * run the patch below in numpy/core/ * (the strip-off regarding 'fromnumeric' is optional) * from there re-add just the stuff you need == patch in numpy/core/ === diff -ur -x *.pyc -x *.pyo ..\..\numpyxx\core\__init__.py .\__init__.py --- ..\..\numpyxx\core\__init__.py Tue Apr 28 21:16:32 2009 +++ .\__init__.py Sat May 16 11:15:35 2009 @@ -1,36 +1,34 @@ - +# minimal numpy! # pyXpy transposer to original version: #$1 from info import __doc__ from numpy.version import version as __version__ import multiarray +from multiarray import * #$1 import umath import _internal # for freeze programs import numerictypes as nt multiarray.set_typeDict(nt.sctypeDict) -import _sort +#$1 import _sort +import numeric #$1 from numeric import * -from fromnumeric import * -from defmatrix import * -import defchararray as char -import records as rec -from records import * -from memmap import * -from defchararray import * -import scalarmath -del nt - -from fromnumeric import amax as max, amin as min, \ - round_ as round -from numeric import absolute as abs +#import fromnumeric #$1 +#$1 from fromnumeric import * +#$1 from defmatrix import * +#$1 import defchararray as char +#$1 import records as rec +#$1 from records import * +#$1 from memmap import * +#$1 from defchararray import * +#$1 import scalarmath +#$1 del nt + +#$1 from fromnumeric import amax as max, amin as min, \ +#$1 round_ as round +#$1 from numeric import absolute as abs -__all__ = ['char','rec','memmap'] +__all__ = [] #$1 ['char','rec','memmap'] __all__ += numeric.__all__ -__all__ += fromnumeric.__all__ -__all__ += defmatrix.__all__ -__all__ += rec.__all__ -__all__ += char.__all__ - - -from numpy.testing import Tester -test = Tester().test -bench = Tester().bench +#$1 __all__ += fromnumeric.__all__ +#$1 __all__ += defmatrix.__all__ +#$1 __all__ += rec.__all__ +#$1 __all__ += char.__all__ Only in ..\..\numpyxx\core:
[Numpy-discussion] linear algebra help
Hi- This is what I need to do- I have this equation- Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. I want to find the vector x' such that Ax' is as close as possible to y. Meaning that (Ax' - y) is as close as possible to (0,0,0,...0). I know that I need to use either the lstsq function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq or the svd function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#svd I don't understand the documentation at all. Can someone please show me how to use these functions to solve my problem. Thanks a lot!!! -quilby ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On Sat, 16 May 2009 16:01:00 +0300 Quilby qui...@gmail.com wrote: Hi- This is what I need to do- I have this equation- Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. I want to find the vector x' such that Ax' is as close as possible to y. Meaning that (Ax' - y) is as close as possible to (0,0,0,...0). I know that I need to use either the lstsq function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq or the svd function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#svd I don't understand the documentation at all. Can someone please show me how to use these functions to solve my problem. Thanks a lot!!! -quilby ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion I guess you meant a rectangular matrix http://mathworld.wolfram.com/RectangularMatrix.html from numpy.random import rand, seed from numpy import dot, shape from numpy.linalg import lstsq, norm seed(1) m = 10 n = 20 A = rand(m,n) # random matrix b = rand(m) # rhs x,residues,rank,s = lstsq(A,b) print 'Singular values',s print 'Numerical rank of A',rank print 'Solution',x r=dot(A,x)-b print 'residual',norm(r) Cheers, Nils ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On Sat, May 16, 2009 at 9:01 AM, Quilby qui...@gmail.com wrote: Hi- This is what I need to do- I have this equation- Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. I want to find the vector x' such that Ax' is as close as possible to y. Meaning that (Ax' - y) is as close as possible to (0,0,0,...0). I know that I need to use either the lstsq function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq or the svd function: http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#svd I don't understand the documentation at all. Can someone please show me how to use these functions to solve my problem. Hi, The new docs are more informative and are being improved in the online editor see http://docs.scipy.org/doc/ and http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.lstsq.html#numpy.linalg.lstsq any comments and improvement to the docs are very welcome. Josef ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Obsolete Endo-generated docs
Hi, Sat, 16 May 2009 16:01:00 +0300, Quilby wrote: [clip] http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq [clip] Could we take these old Endo-generated docs down, and make the URL redirect to docs.scipy.org? I believe they are more harmful than helpful... -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Obsolete Endo-generated docs
Sat, 16 May 2009 14:02:34 +, Pauli Virtanen wrote: Hi, Sat, 16 May 2009 16:01:00 +0300, Quilby wrote: [clip] http://www.scipy.org/doc/numpy_api_docs/numpy.linalg.linalg.html#lstsq [clip] Could we take these old Endo-generated docs down, and make the URL redirect to docs.scipy.org? I went and removed the links to them from http://www.scipy.org/Documentation -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] masked ufuncs in C: on github
On Sat, May 16, 2009 at 2:02 AM, Eric Firing efir...@hawaii.edu wrote:
Charles R Harris wrote:
On Fri, May 15, 2009 at 7:48 PM, Eric Firing efir...@hawaii.edu
mailto:efir...@hawaii.edu wrote:
http://www.mail-archive.com/numpy-discussion@scipy.org/msg17595.html
Prompted by the thread above, I decided to see what it would take to
implement ufuncs with masking in C. I described the result here:
http://www.mail-archive.com/numpy-discussion@scipy.org/msg17698.html
Now I am starting a new thread. The present state of the work is now
in
github: http://github.com/efiring/numpy-work/tree/cfastma
I don't want to do any more until I have gotten some feedback from
core
developers. (And I would be delighted if someone wants to help with
this, or take it over.)
Chuck,
Thanks very much for the quick response.
Here the if ... continue needs to follow the declaration:
if (*mp1) continue;
float in1 = *(float *)ip1;
float in2 = *(float *)ip2;
*(float *)op1 = f(in1, in2);
I was surprised to see the declarations inside the loop in the first
place (this certainly is not ANSI-C), and I was also pleasantly
surprised that letting them be after the conditional didn't seem to
bother the compiler at all. Maybe that is a gcc extension.
Declarations at the top of a block have always been valid C.
I think this would be better as
if (!(*mp1)) {
float in1 = *(float *)ip1;
float in2 = *(float *)ip2;
*(float *)op1 = f(in1, in2);
}
I agree, and I thought of that originally--I think I did it with
continue because it was easier to type it in, and it reduced the
difference relative to the non-masked form.
But since this is actually a ternary function, you could define new
functions, something like
double npy_add_m(double a, double b, double mask)
{
if (!mask) {
return a + b;
else {
return a;
}
}
And use the currently existing loops. Well, you would have to add one
for ternary functions.
That would incur the overhead of an extra function call for each
element; I suspect it would slow it down a lot. My motivation is to make
masked array overhead negligible, at least for medium to large arrays.
It overhead would be the same as it is now, the generic loops all use passed
function pointers for functions like sin. Some functions like addition,
which is intrinsic and not part of a library, are done in their own special
loops that you will find further down in that file. The difficulty I see is
that with the current machinery the mask will be converted to the same type
as the added numbers and that could add some overhead.
Also your suggestion above does not handle the case where an output
argument is supplied; it would modify the output under the mask.
Question, what about reduce? I don't think it is defined defined for
ternary functions. Apart from reduce, why not just add, you already have
the mask to tell you which results are invalid.
You mean just do the operation and ignore the results under the mask?
This is the way Pierre originally did it, if I remember correctly, but
fairly recently people started objecting that they didn't want to
disturb values in an output argument under a mask. So now ma jumps
through hoops to satisfy this requirement, and it is consequently slow.
OK. I'm not familiar with the uses of masked arrays.
ufunc methods like reduce are supported only for the binary ops with one
output, so they are automatically unavailable for the masked versions.
To get around this would require subclassing the ufunc to make a masked
version. This is probably the best way to go, but I suspect it is much
more complicated than I can handle in the amount of time I can spend.
I think reduce could be added for ternary functions, but it is a design
decision how it should operate.
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about slicing
Emmanuelle Gouillart emmanuelle.gouillart at normalesup.org writes:
Hi Jorge,
roi = aimg[10:20,45:50,:]
are you working with 3-D images? I didn't know PIL was able to handle 3D
images.
Well, if by 3D you mean color images then yes, PIL is able to handle them
I wasn't able to reproduce the behavior you observed with a simple
example:
In [20]: base = np.arange(25).reshape((5,5))
In [21]: base
Out[21]:
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
In [22]: flip = base[::-1]
In [23]: flip
Out[23]:
array([[20, 21, 22, 23, 24],
[15, 16, 17, 18, 19],
[10, 11, 12, 13, 14],
[ 5, 6, 7, 8, 9],
[ 0, 1, 2, 3, 4]])
In [24]: flip[2:4,2:4]
Out[24]:
array([[12, 13],
[ 7, 8]])
which is what you expect...
You're right. I should have done these tests myself. I apologize for jumping to
the list so quickly.
I also tried the same manipulations as you do starting from a PIL image
object, but I also got what I expected (and my image was not flipped
vertically by PIL or when transformed into an array). It is quite weird
BTW that your images are flipped. How do you visualize PIL image
(Image.Image.show?)and arrays (pylab.imshow?) ?
It is weird indeed. But no so much that they appear upside down (I do use
pylab.imshow() to display images), because at the end of the day it is just a
convention, and different things can use different conventions, but because of
the fact that the numpy array obtained from the PIL image is not.
I downloaded the scipy logo:
http://docs.scipy.org/doc/_static/scipyshiny_small.png and did the following:
img = Image.open('./scipyshiny_small.png')
mpl.pylab.imshow(img) # Comes upside down
aimg = asarray(img)
mpl.pylab.imshow(aimg) # Comes OK
I guess my problem lies with PIL rather than with numpy. I am glad to find that
slicing works as I would have expected it to work!
Hope someone can help you more than I did :D
You did help, thanks!
Jorge
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about slicing
Sat, 16 May 2009 22:05:16 +, Jorge Scandaliaris wrote:
[clip]
I downloaded the scipy logo:
http://docs.scipy.org/doc/_static/scipyshiny_small.png and did the
following:
img = Image.open('./scipyshiny_small.png')
mpl.pylab.imshow(img) # Comes upside down
aimg = asarray(img)
mpl.pylab.imshow(aimg) # Comes OK
Ah, that was it. Apparently, matplotlib imshow uses a different
conversion mechanism if the input is a Image.Image from PIL, than if it
is an array. There's a dedicated function matplotlib.image.pil_to_array
which seems to work differently from asarray.
This may actually be a bug in matplotlib; perhaps you should ask the
people on the matplotlib lists if this is really the intended behavior.
--
Pauli Virtanen
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about slicing
Pauli Virtanen pav at iki.fi writes:
snip
img = array(img)[::-1]
Note that here a copy is made. You can use `asarray` instead of `array`
if you want to avoid making a copy.
Thanks, that's good info!
and it worked!, but I am interested actually in sub-regions of this
image, so the next I did was:
roi = aimg[10:20,45:50,:]
And to my surprise the result was like if I was slicing the original,
upside down, image instead of aimg. Can someone explain me what's going
on here?
Sounds impossible, and I don't see this:
In [1]: import Image
In [2]: img = Image.open('foo.png')
In [3]: aimg = array(img)
In [4]: imshow(aimg)
Out[4]: matplotlib.image.AxesImage object at 0x9a6ecec
In [5]: imshow(aimg[10:320,5:150])
Out[5]: matplotlib.image.AxesImage object at 0x9f1db2c
The image is here right-side up, both in full and the slice (since imshow
flips it). Also,
In [6]: aimg = array(img)[::-1]
In [7]: imshow(aimg[10:320,5:150])
Out[7]: matplotlib.image.AxesImage object at 0xa007eac
Now, the image is upside down, both in full and in the slice.
I think you should re-check that you are doing what you think you are
doing. Preparing a self-contained code example could help here, at least
this would make pinpointing where the error is more easy.
You're right. I was using imshow to see img (the IPL iamge, not the numpy
array), and that comes upside down, at least here. That made me think the numpy
array was upside down too when in fact it wasn't, so my 'fix' actually was
flipping it.
I'll further investigate as why the IPL image appears upside down, but my
questions about slicing are answered now. Sorry for mixing things up, and thanks
for helping out.
Jorge
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] linear algebra help
On 5/16/2009 9:01 AM Quilby apparently wrote: Ax = y Where A is a rational m*n matrix (m=n), and x and y are vectors of the right size. I know A and y, I don't know what x is equal to. I also know that there is no x where Ax equals exactly y. If m=n, that can only be true if there are not m linearly independent columns of A. Are you sure you have the dimensions right? Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Leopard install
I'm continuing to have this problem. I have installed Python 2.6.2 from the source that is found on www.python.org. I'm running Leopard 10.5.7. Entering python from the shell shows: Python 2.6.2 (r262:71600, May 16 2009, 19:04:59) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type help, copyright, credits or license for more information. Stuart Edwards wrote: Hi I am trying to install numpy 1.3.0 on Leopard 10.5.6 and at the point in the install process where I select a destination, my boot disc is excluded with the message: You cannot install numpy 1.3.0 on this volume. numpy requires System Python 2.5 to install. I'm not sure what 'System Python 2.5' is as compared to 'Python 2.5' but in the terminal when I type 'python' I get: Python 2.5.1 (r251:54863, Jan 13 2009, 10:26:13) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin so the Python 2.5 requirement seems to be met. Any ideas on what is happening here and why the installer can't see my python installation? (I notice that this issue was raised 3/28 also, but no resolution yet) Thanks for any assistance Stu ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- View this message in context: http://www.nabble.com/Leopard-install-tp23012456p23579011.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Leopard install
On Sat, May 16, 2009 at 19:26, ljhardy ljha...@gmail.com wrote: I'm continuing to have this problem. I have installed Python 2.6.2 from the source that is found on www.python.org. I'm running Leopard 10.5.7. You cannot use a binary of numpy built for Python 2.5 with your Python 2.6. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about slicing
the reason for all this is that the bitmap image format specifies the image
origin as the lower left corner. This is the convention used by PIL. The
origin of a numpy array is the upper right corner. Matplot lib does not
handle this discrepancy in the function pil_to_array, which is called
internally when you invoke imshow(img) on a PIL image. Recently, PIL has
implemented the array interface for PIL images. So if you call asarray(img)
on a PIL image, you will get a (height, width, 3) array (for RGB) with the
origin in the upper left corner. This is why the image appears right side up
in matplotlib doing things this way.
The matplot lib code should probably be updated to make use of the array
interface. It just reshapes the raw string data currently.
Chris
On Sat, May 16, 2009 at 6:42 PM, Jorge Scandaliaris
jorgesmbox...@yahoo.eswrote:
Pauli Virtanen pav at iki.fi writes:
snip
img = array(img)[::-1]
Note that here a copy is made. You can use `asarray` instead of `array`
if you want to avoid making a copy.
Thanks, that's good info!
and it worked!, but I am interested actually in sub-regions of this
image, so the next I did was:
roi = aimg[10:20,45:50,:]
And to my surprise the result was like if I was slicing the original,
upside down, image instead of aimg. Can someone explain me what's going
on here?
Sounds impossible, and I don't see this:
In [1]: import Image
In [2]: img = Image.open('foo.png')
In [3]: aimg = array(img)
In [4]: imshow(aimg)
Out[4]: matplotlib.image.AxesImage object at 0x9a6ecec
In [5]: imshow(aimg[10:320,5:150])
Out[5]: matplotlib.image.AxesImage object at 0x9f1db2c
The image is here right-side up, both in full and the slice (since imshow
flips it). Also,
In [6]: aimg = array(img)[::-1]
In [7]: imshow(aimg[10:320,5:150])
Out[7]: matplotlib.image.AxesImage object at 0xa007eac
Now, the image is upside down, both in full and in the slice.
I think you should re-check that you are doing what you think you are
doing. Preparing a self-contained code example could help here, at least
this would make pinpointing where the error is more easy.
You're right. I was using imshow to see img (the IPL iamge, not the numpy
array), and that comes upside down, at least here. That made me think the
numpy
array was upside down too when in fact it wasn't, so my 'fix' actually was
flipping it.
I'll further investigate as why the IPL image appears upside down, but my
questions about slicing are answered now. Sorry for mixing things up, and
thanks
for helping out.
Jorge
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy slices limited to 32 bit values?
Today at Sage Days we tried slices on a few large arrays (no mmap) and found that slicing breaks on arrays somewhere between 2.0e9 and 2.5e9 elements. The failure mode is the same, no error thrown, basically nothing happens This was on one of the big sage machines. I don't know the specific OS / CPU but it was definitely 64 bit and lots of available memory etc. -glenn On Thu, May 14, 2009 at 7:54 AM, Gael Varoquaux gael.varoqu...@normalesup.org wrote: On Thu, May 14, 2009 at 07:40:58AM -0700, Glenn Tarbox, PhD wrote: Hum, I am wondering: could it be that Sage has not been compiled in 64bits? That number '32' seems to me to point toward a 32bit pointer issue (I may be wrong). The other tests I posted indicate everything else is working... For example, np.sum(fp) runs over the full set of 1e10 doubes and seems to work fine. Correct. I had missed that. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Glenn H. Tarbox, PhD || 206-274-6919 http://www.tarbox.org ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
