Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

[Sturla Molden]

Antoine Pitrou  wrote:

> When writing C code to interact with buffer-providing objects, you
> usually don't bother with memoryviews at all.  You just use a Py_buffer
> structure.

I was taking about "typed memoryviews" which is a Cython abstraction for a
Py_buffer struct. I was not taking about Python memoryviews, which is
something else. When writing Cython code that interacts with a buffer we
usually use typed memoryviews, not Py_buffer structs directly.

Sturla

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[R Schumacher]

At 01:15 PM 5/24/2016, you wrote:

On 5/24/2016 3:57 PM, Eric Moore wrote:
Changing np.arange(10)**3 to have a non-integer dtype seems like a 
big change.



What about np.arange(100)**5?


Interesting, one warning per instantiation (Py2.7):

>>> import numpy
>>> a=numpy.arange(100)**5
:1: RuntimeWarning: invalid value encountered in power
>>> a=numpy.arange(100)**5.
>>> b=numpy.arange(100.)**5
>>> a==b
array([ True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True,  True,
True,  True,  True,  True,  True,  True,  True,  True, 
True,  True], dtype=bool)

>>> numpy.arange(100)**5
array([  0,   1,  32, 243,1024,
  3125,7776,   16807,   32768,   59049,
10,  161051,  248832,  371293,  537824,
759375, 1048576, 1419857, 1889568, 2476099,
   320, 4084101, 5153632, 6436343, 7962624,
   9765625,11881376,14348907,17210368,20511149,
  2430,28629151,33554432,39135393,45435424,
  52521875,60466176,69343957,79235168,90224199,
 10240,   115856201,   130691232,   147008443,   164916224,
 184528125,   205962976,   229345007,   254803968,   282475249,
 31250,   345025251,   380204032,   418195493,   459165024,
 503284375,   550731776,   601692057,   656356768,   714924299,
 77760,   844596301,   916132832,   992436543,  1073741824,
1160290625,  1252332576,  1350125107,  1453933568,  1564031349,
168070,  1804229351,  1934917632,  2073071593, -2147483648,
   -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,
   -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,
   -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,
   -2147483648, -2147483648, -2147483648, -2147483648, -2147483648,
   -2147483648, -2147483648, -2147483648, -2147483648, -2147483648])
>>>
>>> numpy.arange(100, dtype=numpy.int64)**5
array([ 0,  1, 32,243,   1024,
 3125,   7776,  16807,  32768,  59049,
   10, 161051, 248832, 371293, 537824,
   759375,1048576,1419857,1889568,2476099,
  320,4084101,5153632,6436343,7962624,
  9765625,   11881376,   14348907,   17210368,   20511149,
 2430,   28629151,   33554432,   39135393,   45435424,
 52521875,   60466176,   69343957,   79235168,   90224199,
10240,  115856201,  130691232,  147008443,  164916224,
184528125,  205962976,  229345007,  254803968,  282475249,
31250,  345025251,  380204032,  418195493,  459165024,
503284375,  550731776,  601692057,  656356768,  714924299,
77760,  844596301,  916132832,  992436543, 1073741824,
   1160290625, 1252332576, 1350125107, 1453933568, 1564031349,
   168070, 1804229351, 1934917632, 2073071593, 2219006624,
   2373046875, 2535525376, 2706784157, 2887174368, 3077056399,
   327680, 3486784401, 3707398432, 3939040643, 4182119424,
   4437053125, 4704270176, 4984209207, 5277319168, 5584059449,
   590490, 6240321451, 6590815232, 6956883693, 7339040224,
   7737809375, 8153726976, 8587340257, 9039207968, 9509900499], 
dtype=int64) ___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Alan Isaac]

On 5/24/2016 3:57 PM, Eric Moore wrote:

Changing np.arange(10)**3 to have a non-integer dtype seems like a big change.



What about np.arange(100)**5?

Alan Isaac
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Eric Moore]

Yes, I'm fully aware of that.  I'm speaking toward changing the default
result dtype.  Raising an error for negative exponents is a fine idea.
Changing np.arange(10)**3 to have a non-integer dtype seems like a big
change.

Speaking of this, that some of the integer array operation errors can be
controlled via the np.seterr and some cannot should also be addressed
longer term.

Eric

On Tue, May 24, 2016 at 3:05 PM, Nathaniel Smith  wrote:

> On Tue, May 24, 2016 at 10:36 AM, Eric Moore 
> wrote:
> > I'd say the most compelling case for it is that is how it has always
> worked.
> > How much code will break if we make that change? (Or if not break, at
> least
> > have a change in dtype?)  Is that worth it?
>
> The current behavior for arrays is:
>
> # Returns int
> In [2]: np.arange(10) ** 2
> Out[2]: array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])
>
> # Returns nonsensical/useless results
> In [3]: np.arange(10) ** -1
> /home/njs/.user-python3.5-64bit/bin/ipython:1: RuntimeWarning: divide by
> zero encountered in power
>   #!/home/njs/.user-python3.5-64bit/bin/python3.5
> /home/njs/.user-python3.5-64bit/bin/ipython:1: RuntimeWarning: invalid
> value encountered in power
>   #!/home/njs/.user-python3.5-64bit/bin/python3.5
> Out[3]:
> array([-9223372036854775808,1,0,
>   0,0,0,
>   0,0,0,
>   0])
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Nathaniel Smith]

On Tue, May 24, 2016 at 10:36 AM, Eric Moore  wrote:
> I'd say the most compelling case for it is that is how it has always
worked.
> How much code will break if we make that change? (Or if not break, at
least
> have a change in dtype?)  Is that worth it?

The current behavior for arrays is:

# Returns int
In [2]: np.arange(10) ** 2
Out[2]: array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])

# Returns nonsensical/useless results
In [3]: np.arange(10) ** -1
/home/njs/.user-python3.5-64bit/bin/ipython:1: RuntimeWarning: divide by
zero encountered in power
  #!/home/njs/.user-python3.5-64bit/bin/python3.5
/home/njs/.user-python3.5-64bit/bin/ipython:1: RuntimeWarning: invalid
value encountered in power
  #!/home/njs/.user-python3.5-64bit/bin/python3.5
Out[3]:
array([-9223372036854775808,1,0,
  0,0,0,
  0,0,0,
  0])

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Alan Isaac]

On 5/24/2016 1:41 PM, Matthew Brett wrote:

It's a well-understood promise though - you always have to be careful
of integer overflow.



Of course.  But **almost all** cases overflow.

And "well understood" assume a certain sophistication
of the user, while `arange` will certainly be used
by beginners.

Alan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Alan Isaac]

On 5/24/2016 1:36 PM, Eric Moore wrote:

How much code will break if we make that change?



Since even arange(10)**10 is already broken,
there will probably not be much new breakage.

But under any of the new proposals, *something* will break.
So the question is, which shows the most foresight?
Having (**) actually work seems worth quite a lot.

Alan Isaac

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Matthew Brett]

On Tue, May 24, 2016 at 1:31 PM, Alan Isaac  wrote:
> On 5/24/2016 1:19 PM, Stephan Hoyer wrote:
>>
>> the int ** 2 example feels quite compelling to me
>
>
>
> Yes, but that one case is trivial: a*a

Right, but you'd have to know to change your code when numpy makes
this change.   Your code will suddenly have a new datatype, that you
may be passing into other code, which may generate different results,
that will be hard to track down.

> And at least as compelling is not have a**-2 fail

I imagine that's about an order of magnitude less common in real code,
but an error seems an acceptable result to me, so I know I have to do
a ** -2.0

> and not being tricked by say np.arange(10)**10.
> The latter is a promise of hidden errors.

It's a well-understood promise though - you always have to be careful
of integer overflow.

Best,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Stephan Hoyer]

On Tue, May 24, 2016 at 10:31 AM, Alan Isaac  wrote:

> Yes, but that one case is trivial: a*a


an_explicit_name ** 2 is much better than an_explicit_name *
an_explicit_name, though.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Eric Moore]

I'd say the most compelling case for it is that is how it has always
worked.  How much code will break if we make that change? (Or if not break,
at least have a change in dtype?)  Is that worth it?

Eric

On Tue, May 24, 2016 at 1:31 PM, Alan Isaac  wrote:

> On 5/24/2016 1:19 PM, Stephan Hoyer wrote:
>
>> the int ** 2 example feels quite compelling to me
>>
>
>
> Yes, but that one case is trivial: a*a
>
> And at least as compelling is not have a**-2 fail
> and not being tricked by say np.arange(10)**10.
> The latter is a promise of hidden errors.
>
> Alan
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Alan Isaac]

On 5/24/2016 1:19 PM, Stephan Hoyer wrote:

the int ** 2 example feels quite compelling to me



Yes, but that one case is trivial: a*a

And at least as compelling is not have a**-2 fail
and not being tricked by say np.arange(10)**10.
The latter is a promise of hidden errors.

Alan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Stephan Hoyer]

On Tue, May 24, 2016 at 9:41 AM, Alan Isaac  wrote:

> What exactly is the argument against *always* returning float
> (even for positive integer exponents)?
>

If we were starting over from scratch, I would agree with you, but the int
** 2 example feels quite compelling to me. I would guess there lots of code
out there that expects the result to have integer dtype.

As a contrived example, I might write np.arange(n) ** 2 to produce an
indexer for the diagonal elements of an array.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Alan Isaac]

On 5/24/2016 10:03 AM, Marten van Kerkwijk wrote:

The error on int ** (-int) is OK with me too (even though I prefer just 
returning floats).



What exactly is the argument against *always* returning float
(even for positive integer exponents)?  Relevant context is:

- int ** int will often ("usually") overflow
- a numpy function cd meet specialized exponentiation needs

Thanks,
Alan Isaac

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Marten van Kerkwijk]

The error on int ** (-int) is OK with me too (even though I prefer just
returning floats).
Having a `floor_pow` function may still be good with this solution too.
-- Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Integers to integer powers

[Antoine Pitrou]

On Thu, 19 May 2016 20:30:40 -0700
Nathaniel Smith  wrote:
> 
> Given these immutable and contradictory constraints, the last bad
> option IMHO would be that we make int ** (negative int) an error in
> all cases, and the error message can suggest that instead of writing
> 
> np.array(2) ** -2
> 
> they should instead write
> 
> np.array(2) ** -2.0
> 
> (And similarly for np.int64(2) ** -2 versus np.int64(2) ** -2.0.)
> 
> Definitely annoying, but all the other options seem even more
> inconsistent and confusing, and likely to encourage the writing of
> subtly buggy code...

That sounds like a good compromise, indeed.

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy arrays shareable among related processes (PR #7533)

[Antoine Pitrou]

On Thu, 12 May 2016 23:14:36 + (UTC)
Sturla Molden  wrote:
> Antoine Pitrou  wrote:
> 
> > Can you define "expensive"?
> 
> Slow enough to cause complaints on the Cython mailing list.

What kind of complaints? Please be specific.

> > Buffer acquisition itself only calls a single C callback and uses a
> > stack-allocated C structure. It shouldn't be "expensive".
> 
> I don't know the reason, only that buffer acquisition from NumPy arrays
> with typed memoryviews

Again, what have memoryviews to do with it?  "Acquiring a buffer" is
just asking the buffer provider (the Numpy array) to fill a Py_buffer
structure's contents.  That has nothing to do with memoryviews.

When writing C code to interact with buffer-providing objects, you
usually don't bother with memoryviews at all.  You just use a Py_buffer
structure.

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion