Re: [Numpy-discussion] how is y += x computed when y.strides = (0, 8) and x.strides=(16, 8) ?
I'd like to have a look at the implementation of iadd in numpy, but I'm having a real hard time to find the corresponding code. I'm basically stuck at https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/number.c#L487 Could someone give me a pointer where to find it? Respectively, could someone point me to some documentation where the (folder/file) structure of the numpy sources is explained? Sebastian On Thu, Sep 6, 2012 at 2:58 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Sep 6, 2012 at 1:41 AM, Sebastian Berg sebast...@sipsolutions.net wrote: Hey, No idea if this is simply not support or just a bug, though I am guessing that such usage simply is not planned. I think that's right... currently numpy simply makes no guarantees about what order ufunc loops will be performed in, or even if they will be performed in any strictly sequential order. In ordinary cases this lets it make various optimizations, but it means that you can't count on any specific behaviour for the unusual case where different locations in the output array are stored in overlapping memory. Fixing this would require two things: (a) Some code to detect when an array may have internal overlaps (sort of like np.may_share_memory for axes). Not entirely trivial. (b) A fallback mode for ufuncs where if the code in (a) detects that we are (probably) dealing with one of these arrays, it processes the operations in some predictable order without buffering. I suppose if someone wanted to come up with these two pieces, and it didn't look like it would cause slowdowns in common cases, the code in (b) avoided creating duplicate code paths that increased maintenance burden, etc., then probably no-one would object to making these arrays act in a better defined way? I don't think most people are that worried about this though. Your original code would be much clearer if it just used np.sum... -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] A change with minor compatibility questions
Hey all, https://github.com/numpy/numpy/pull/482 is a pull request that changes the hash function for numpy void scalars. These are the objects returned from fully indexing a structured array: array[i] if array is a 1-d structured array. Currently their hash function just hashes the pointer to the underlying data. This means that void scalars can be used as keys in a dictionary but the behavior is non-intuitive because another void scalar with the same data but pointing to a different region of memory will hash differently. The pull request makes it so that two void scalars with the same data will hash to the same value (using the same algorithm as a tuple hash).This pull request also only allows read-only scalars to be hashed. There is a small chance this will break someone's code if they relied on this behavior. I don't believe anyone is currently relying on this behavior -- but I've been proven wrong before. What do people on this list think? Should we raise a warning in the next release when a hash function on a void scalar is called or just make the change, put it in the release notes and make a few people change their code if needed. The problem was identified by a couple of users of NumPy currently which is why I think that people who have tried using numpy void scalars as keys aren't doing it right now but are instead converting them to tuples first. -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A change with minor compatibility questions
On Wed, Oct 17, 2012 at 10:22:28AM -0500, Travis Oliphant wrote: There is a small chance this will break someone's code if they relied on this behavior. I don't believe anyone is currently relying on this behavior -- but I've been proven wrong before. What do people on this list think? I think that the change is acceptable. Thanks for asking! Should we raise a warning in the next release when a hash function on a void scalar is called or just make the change, put it in the release notes and make a few people change their code if needed. I think that there should be a section in the release notes that list such changes in a very visible way. I don't think that a warning is necessary here, but I guess that others might have a different point of view. G ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A change with minor compatibility questions
On 10/17/2012 05:22 PM, Travis Oliphant wrote:
Hey all,
https://github.com/numpy/numpy/pull/482
is a pull request that changes the hash function for numpy void
scalars. These are the objects returned from fully indexing a
structured array: array[i] if array is a 1-d structured array.
Currently their hash function just hashes the pointer to the underlying
data.This means that void scalars can be used as keys in a
dictionary but the behavior is non-intuitive because another void scalar
with the same data but pointing to a different region of memory will
hash differently.
The pull request makes it so that two void scalars with the same data
will hash to the same value (using the same algorithm as a tuple hash).
This pull request also only allows read-only scalars to be hashed.
There is a small chance this will break someone's code if they relied on
this behavior. I don't believe anyone is currently relying on this
behavior -- but I've been proven wrong before. What do people on this
list think?
I support working on fixing this, but if I understand your fix correctly
this change just breaks things in a different way.
Specifically, in this example:
arr = np.ones(4, dtype=[('a', np.int64)])
x = arr[0]
d = { x : 'value' }
arr[0]['a'] = 4
print d[x]
Does the last line raise a KeyError? If I understand correctly it does.
(Of course, the current situation just breaks lookups in another situation.)
I propose to BOTH make x unhashable (thus being a good Python citizen
and following Python rules) AND provide x.askey() or x.immutable()
which returns something immutable you can use as a key.
The places where that breaks things is probably buggy code that must be
fixed (either one way or the other) anyway. Perhaps a warning period is
in order then (one would raise a warning in __hash__, telling people to
use the askey() method).
(I would really prefer to always have x be immutable, but that
probably breaks working code.)
Dag Sverre
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A change with minor compatibility questions
On 10/17/2012 06:56 PM, Dag Sverre Seljebotn wrote:
On 10/17/2012 05:22 PM, Travis Oliphant wrote:
Hey all,
https://github.com/numpy/numpy/pull/482
is a pull request that changes the hash function for numpy void
scalars. These are the objects returned from fully indexing a
structured array: array[i] if array is a 1-d structured array.
Currently their hash function just hashes the pointer to the underlying
data.This means that void scalars can be used as keys in a
dictionary but the behavior is non-intuitive because another void scalar
with the same data but pointing to a different region of memory will
hash differently.
The pull request makes it so that two void scalars with the same data
will hash to the same value (using the same algorithm as a tuple hash).
This pull request also only allows read-only scalars to be hashed.
There is a small chance this will break someone's code if they relied on
this behavior. I don't believe anyone is currently relying on this
behavior -- but I've been proven wrong before. What do people on this
list think?
I support working on fixing this, but if I understand your fix correctly
this change just breaks things in a different way.
Specifically, in this example:
arr = np.ones(4, dtype=[('a', np.int64)])
x = arr[0]
d = { x : 'value' }
arr[0]['a'] = 4
print d[x]
Does the last line raise a KeyError? If I understand correctly it does.
Argh. I overlooked both Travis' second commit, and the explicit mention
of read-only above.
Isn't it possible to produce a read-only array from a writeable one
though, and so get a read-only scalar whose underlying value can still
change?
Anyway, sorry about being so quick to post.
Dag Sverre
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A change with minor compatibility questions
On Oct 17, 2012, at 12:48 PM, Dag Sverre Seljebotn wrote:
On 10/17/2012 06:56 PM, Dag Sverre Seljebotn wrote:
On 10/17/2012 05:22 PM, Travis Oliphant wrote:
Hey all,
https://github.com/numpy/numpy/pull/482
is a pull request that changes the hash function for numpy void
scalars. These are the objects returned from fully indexing a
structured array: array[i] if array is a 1-d structured array.
Currently their hash function just hashes the pointer to the underlying
data.This means that void scalars can be used as keys in a
dictionary but the behavior is non-intuitive because another void scalar
with the same data but pointing to a different region of memory will
hash differently.
The pull request makes it so that two void scalars with the same data
will hash to the same value (using the same algorithm as a tuple hash).
This pull request also only allows read-only scalars to be hashed.
There is a small chance this will break someone's code if they relied on
this behavior. I don't believe anyone is currently relying on this
behavior -- but I've been proven wrong before. What do people on this
list think?
I support working on fixing this, but if I understand your fix correctly
this change just breaks things in a different way.
Specifically, in this example:
arr = np.ones(4, dtype=[('a', np.int64)])
x = arr[0]
d = { x : 'value' }
arr[0]['a'] = 4
print d[x]
Does the last line raise a KeyError? If I understand correctly it does.
Argh. I overlooked both Travis' second commit, and the explicit mention
of read-only above.
Isn't it possible to produce a read-only array from a writeable one
though, and so get a read-only scalar whose underlying value can still
change?
Yes, it is possible to do that (just like it is currently possible to change a
tuple with a C-extension or even Cython or a string with NumPy).
We won't be able to prevent people from writing code that will have odd
behavior, but we can communicate correctly about what one should do.
-Travis
Anyway, sorry about being so quick to post.
Dag Sverre
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Issue tracking
On Tue, Oct 16, 2012 at 11:54 PM, Nathaniel Smith n...@pobox.com wrote: On Tue, Oct 16, 2012 at 9:06 PM, Thouis (Ray) Jones tho...@gmail.com wrote: On Sun, Oct 7, 2012 at 10:15 AM, Thouis (Ray) Jones tho...@gmail.com wrote: I plan to import all the Trac issues to github by the end of this week. I want to get an up-to-date snapshot of the Trac DB, and run another test import with it (just to make sure there's nothing in recent bugs that isn't handled). Previous test imports here: https://github.com/thouis/numpy-trac-migration/issues I successfully imported all the issues from a more recent snapshot to the test repository as @numpy-gitbot, rather than @thouis (to save myself from getting pulled into every bug's discussion). If no one sees any problems with the latest imports (basically, the last 2000 or so by github issue #) , I think it's ready for the real transfer to the numpy github repository. This is really fabulous; thanks for all the effort you've put in! Looks great to me too, don't see anything missing. Thanks a lot Ray! Ralf But... I am quite concerned that we're still linking to attachments in the trac, e.g.: https://github.com/thouis/numpy-trac-migration/issues/6002#issuecomment-9492223 This means that we can't ever take down the trac without breaking all our github issues; we're committing to keeping the trac running indefinitely. Doesn't that kind of defeat a lot of the point...? -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] how is y += x computed when y.strides = (0, 8) and x.strides=(16, 8) ?
On Wed, Oct 17, 2012 at 11:38 AM, Sebastian Walter sebastian.wal...@gmail.com wrote: I'd like to have a look at the implementation of iadd in numpy, but I'm having a real hard time to find the corresponding code. I'm basically stuck at https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/number.c#L487 n_ops is essentially a map of function pointers set in the umath module (see PyArray_SetNumericOps, used in umath module). IOW, n_ops.add is a pointer to umath.add, itself set up through a generated file __umath_generated.c: f = PyUFunc_FromFuncAndData(add_functions, ...); PyDict_SetItemString(dictionary, add, f); At that point, you need to look into the ufunc machinery: for double all along, the add type resolver should be pretty simple and in the end call DOUBLE_add. Could someone give me a pointer where to find it? Respectively, could someone point me to some documentation where the (folder/file) structure of the numpy sources is explained? There is sadly not much explanation on the code structure for the C part. src/multiarray contains the code for the multiarray extension (array, dtype, broadcasting, iteration) and src/umath contains the code for the umath extension (ufunc machinery + core loops implementation). David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] numpy.arrange not returning expected results (bug?)
Hi, I've discovered calling numpy.arange(1.1, 17.1) and numpy(1.1, 16.1) both return the same results. Could this be a numpy bug, or is there some behaviour I'm possibly not aware of here? I've pasted in the results of an interactive Python session comparing and contrasting these with some other similar calls. I'm using NumPy 1.6.2. Thanks in advance, Simon 14:21:46.77 @ C:\Users\simon python Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy as np np.arange(1.1, 17.1) array([ 1.1, 2.1, 3.1, 4.1, 5.1, 6.1, 7.1, 8.1, 9.1, 10.1, 11.1, 12.1, 13.1, 14.1, 15.1, 16.1]) np.arange(1.1, 16.1) array([ 1.1, 2.1, 3.1, 4.1, 5.1, 6.1, 7.1, 8.1, 9.1, 10.1, 11.1, 12.1, 13.1, 14.1, 15.1, 16.1]) np.arange(1.1, 15.1) array([ 1.1, 2.1, 3.1, 4.1, 5.1, 6.1, 7.1, 8.1, 9.1, 10.1, 11.1, 12.1, 13.1, 14.1]) np.arange(1, 17) array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]) np.arange(1, 16) array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]) np.arange(1, 15) array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
