Re: [Numpy-discussion] OpenOpt Suite release 0.45
--- Исходное сообщение --- От кого: Alan G Isaac alan.is...@gmail.com Дата: 15 марта 2013, 22:54:21 On 3/15/2013 3:34 PM, Dmitrey wrote: the suspected bugs are not documented yet I'm going to guess that the state of the F_i changes when you use them as keys (i.e., when you call __le__. no, their state doesn't change for operations like __le__ . AFAIK searching Python dict doesn't calls __le__ on the object keys at all, it operates with method .__hash__(), and latter returns fixed integer numbers assigned to the objects earlier (at least in my case). It is very hard to imagine that this is a Python or NumPy bug. Cheers, Alan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
Hi, Different objects can have the same hash, so it compares to find the actual correct object. Usually when you store something in a dict and later you can't find it anymore, it is that the internal state changed and that the hash is not the same anymore. Matthieu 2013/3/16 Dmitrey tm...@ukr.net --- Исходное сообщение --- От кого: Alan G Isaac alan.is...@gmail.com Дата: 15 марта 2013, 22:54:21 On 3/15/2013 3:34 PM, Dmitrey wrote: the suspected bugs are not documented yet I'm going to guess that the state of the F_i changes when you use them as keys (i.e., when you call __le__. no, their state doesn't change for operations like __le__ . AFAIK searching Python dict doesn't calls __le__ on the object keys at all, it operates with method .__hash__(), and latter returns fixed integer numbers assigned to the objects earlier (at least in my case). It is very hard to imagine that this is a Python or NumPy bug. Cheers, Alan ___ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
--- Исходное сообщение --- От кого: Matthieu Brucher matthieu.bruc...@gmail.com Дата: 16 марта 2013, 11:33:39 Hi, Different objects can have the same hash, so it compares to find the actual correct object. Usually when you store something in a dict and later you can't find it anymore, it is that the internal state changed and that the hash is not the same anymore. my objects (oofuns) definitely have different __hash__() results - it's just integers 1,2,3 etc assigned to the oofuns (stored in oofun._id field) when they are created. D. Matthieu 2013/3/16 Dmitrey tm...@ukr.net --- Исходное сообщение --- От кого: Alan G Isaac alan.is...@gmail.com Дата: 15 марта 2013, 22:54:21 On 3/15/2013 3:34 PM, Dmitrey wrote: the suspected bugs are not documented yet I'm going to guess that the state of the F_i changes when you use them as keys (i.e., when you call __le__. no, their state doesn't change for operations like __le__ . AFAIK searching Python dict doesn't calls __le__ on the object keys at all, it operates with method .__hash__(), and latter returns fixed integer numbers assigned to the objects earlier (at least in my case). It is very hard to imagine that this is a Python or NumPy bug. Cheers, Alan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
Even if they have different hashes, they can be stored in the same underlying list before they are retrieved. Then, an actual comparison is done to check if the given key (i.e. object instance, not hash) is the same as one of the stored keys. 2013/3/16 Dmitrey tm...@ukr.net --- Исходное сообщение --- От кого: Matthieu Brucher matthieu.bruc...@gmail.com Дата: 16 марта 2013, 11:33:39 Hi, Different objects can have the same hash, so it compares to find the actual correct object. Usually when you store something in a dict and later you can't find it anymore, it is that the internal state changed and that the hash is not the same anymore. my objects (oofuns) definitely have different __hash__() results - it's just integers 1,2,3 etc assigned to the oofuns (stored in oofun._id field) when they are created. D. Matthieu 2013/3/16 Dmitrey tm...@ukr.net --- Исходное сообщение --- От кого: Alan G Isaac alan.is...@gmail.com Дата: 15 марта 2013, 22:54:21 On 3/15/2013 3:34 PM, Dmitrey wrote: the suspected bugs are not documented yet I'm going to guess that the state of the F_i changes when you use them as keys (i.e., when you call __le__. no, their state doesn't change for operations like __le__ . AFAIK searching Python dict doesn't calls __le__ on the object keys at all, it operates with method .__hash__(), and latter returns fixed integer numbers assigned to the objects earlier (at least in my case). It is very hard to imagine that this is a Python or NumPy bug. Cheers, Alan ___ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ ___ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
--- Исходное сообщение --- От кого: Matthieu Brucher matthieu.bruc...@gmail.com Дата: 16 марта 2013, 12:39:07 Even if they have different hashes, they can be stored in the same underlying list before they are retrieved. Then, an actual comparison is done to check if the given key (i.e. object instance, not hash) is the same as one of the stored keys. but, as I have already mentioned, comparison of oofun(s) via __le__, __eq__ etc doesn't change their inner state (but the methods can create additional oofun(s), although). I have checked via debugger - my methods __le__, __eq__, __lt__, __gt__, __ge__ are not called from the buggy place of code, only __hash__ is called from there. Python could check key objects equivalence via id(), although, but I don't see any possible bug source from using id(). D. 2013/3/16 Dmitrey tm...@ukr.net --- Исходное сообщение --- От кого: Matthieu Brucher matthieu.bruc...@gmail.com Дата: 16 марта 2013, 11:33:39 Hi, Different objects can have the same hash, so it compares to find the actual correct object. Usually when you store something in a dict and later you can't find it anymore, it is that the internal state changed and that the hash is not the same anymore. my objects (oofuns) definitely have different __hash__() results - it's just integers 1,2,3 etc assigned to the oofuns (stored in oofun._id field) when they are created. D. Matthieu 2013/3/16 Dmitrey tm...@ukr.net --- Исходное сообщение --- От кого: Alan G Isaac alan.is...@gmail.com Дата: 15 марта 2013, 22:54:21 On 3/15/2013 3:34 PM, Dmitrey wrote: the suspected bugs are not documented yet I'm going to guess that the state of the F_i changes when you use them as keys (i.e., when you call __le__. no, their state doesn't change for operations like __le__ . AFAIK searching Python dict doesn't calls __le__ on the object keys at all, it operates with method .__hash__(), and latter returns fixed integer numbers assigned to the objects earlier (at least in my case). It is very hard to imagine that this is a Python or NumPy bug. Cheers, Alan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
On 16 Mar 2013 11:49, Dmitrey tm...@ukr.net wrote: --- Исходное сообщение --- От кого: Matthieu Brucher matthieu.bruc...@gmail.com Дата: 16 марта 2013, 12:39:07 Even if they have different hashes, they can be stored in the same underlying list before they are retrieved. Then, an actual comparison is done to check if the given key (i.e. object instance, not hash) is the same as one of the stored keys. but, as I have already mentioned, comparison of oofun(s) via __le__, __eq__ etc doesn't change their inner state (but the methods can create additional oofun(s), although). I have checked via debugger - my methods __le__, __eq__, __lt__, __gt__, __ge__ are not called from the buggy place of code, only __hash__ is called from there. Python could check key objects equivalence via id(), although, but I don't see any possible bug source from using id(). Dict lookup always calls both __hash__ and __eq__. I guess it might use id() to shortcut the __eq__ call in some cases - there are some places in python that do. Anyway there's no point trying to debug this code by ESP... It's not even clear from what's been said whether dict lookups have anything to do with the problem. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] indexing of arbitrary axis and arbitrary slice?
Dear all, Is there some way to index the numpy array by specifying arbitrary axis and arbitrary slice, while not knowing the actual shape of the data? For example, I have a 3-dim data, data.shape = (3,4,5) Is there a way to retrieve data[:,0,:] by using something like np.retrieve_data(data,axis=2,slice=0), by this way you don't have to know the actual shape of the array. for for 4-dim data, np.retrieve_data(data,axis=2,slice=0) will actually be data[:,0,:,:] thanks in advance, Chao -- *** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] indexing of arbitrary axis and arbitrary slice?
On 16 Mar 2013 16:41, Chao YUE chaoyue...@gmail.com wrote: Dear all, Is there some way to index the numpy array by specifying arbitrary axis and arbitrary slice, while not knowing the actual shape of the data? For example, I have a 3-dim data, data.shape = (3,4,5) Is there a way to retrieve data[:,0,:] by using something like np.retrieve_data(data,axis=2,slice=0), by this way you don't have to know the actual shape of the array. for for 4-dim data, np.retrieve_data(data,axis=2,slice=0) will actually be data[:,0,:,:] I don't know of anything quite like that, but it's easy to fake it: def retrieve_data(a, ax, idx): full_idx = [slice(None)] * a.ndim full_idx[ax] = idx return a[tuple(full_idx)] Or for the specific case where you do know the axis in advance, you just don't know how many trailing axes there are, use a[:, :, 0, ...] and the ... will expand to represent the appropriate number of :'s. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
On Sat, Mar 16, 2013 at 10:39 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Even if they have different hashes, they can be stored in the same underlying list before they are retrieved. Then, an actual comparison is done to check if the given key (i.e. object instance, not hash) is the same as one of the stored keys. Right. And the rule is that if two objects compare equal, then they must also hash equal. Unfortunately, it looks like `oofun` objects do not obey this property. oofun.__eq__() seems to return a Constraint rather than a bool, so oofun objects should simply not be used as dictionary keys. That's quite possibly the source of the bug. Or at least, that's a bug that needs to get fixed first before attempting to debug anything else or attribute bugs to Python or numpy. Also, the lack of a bool-returning __eq__() will prevent proper sorting, which also seems to be used in the code snippet that Dmitrey showed. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
--- Исходное сообщение --- От кого: Robert Kern robert.k...@gmail.com Дата: 16 марта 2013, 19:54:51 On Sat, Mar 16, 2013 at 10:39 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Even if they have different hashes, they can be stored in the same underlying list before they are retrieved. Then, an actual comparison is done to check if the given key (i.e. object instance, not hash) is the same as one of the stored keys. Right. And the rule is that if two objects compare equal, then they must also hash equal. Unfortunately, it looks like `oofun` objects do not obey this property. oofun.__eq__() seems to return a Constraint rather than a bool, so oofun objects should simply not be used as dictionary keys. It is one of several base features FuncDesigner is build on and is used extremely often and wide; then whole FuncDesigner would work incorrectly while it is used intensively and solves many problems better than its competitors. That's quite possibly the source of the bug. Or at least, that's a bug that needs to get fixed first before attempting to debug anything else or attribute bugs to Python or numpy. Also, the lack of a bool-returning __eq__() will prevent proper sorting, which also seems to be used in the code snippet that Dmitrey showed. as I have already mentioned, I ensured via debugger that my __eq__, __le__ etc are not involved from the buggy place of the code, only __hash__ is involved from there. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] OpenOpt Suite release 0.45
On Sat, Mar 16, 2013 at 6:19 PM, Dmitrey tm...@ukr.net wrote: --- Исходное сообщение --- От кого: Robert Kern robert.k...@gmail.com Дата: 16 марта 2013, 19:54:51 On Sat, Mar 16, 2013 at 10:39 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Even if they have different hashes, they can be stored in the same underlying list before they are retrieved. Then, an actual comparison is done to check if the given key (i.e. object instance, not hash) is the same as one of the stored keys. Right. And the rule is that if two objects compare equal, then they must also hash equal. Unfortunately, it looks like `oofun` objects do not obey this property. oofun.__eq__() seems to return a Constraint rather than a bool, so oofun objects should simply not be used as dictionary keys. It is one of several base features FuncDesigner is build on and is used extremely often and wide; then whole FuncDesigner would work incorrectly while it is used intensively and solves many problems better than its competitors. I understand. It just means that you can't oofun objects as dictionary keys. Adding a __hash__() method is not enough to make that work. That's quite possibly the source of the bug. Or at least, that's a bug that needs to get fixed first before attempting to debug anything else or attribute bugs to Python or numpy. Also, the lack of a bool-returning __eq__() will prevent proper sorting, which also seems to be used in the code snippet that Dmitrey showed. as I have already mentioned, I ensured via debugger that my __eq__, __le__ etc are not involved from the buggy place of the code, only __hash__ is involved from there. oofun.__lt__() will certainly be called, and it too is problematic. If pointDerivates is a dict mapping oofun objects to other objects as you say, then derivative_items will be a list of (oofun, object) tuples. If you sort derivative_items by the first element, the oofun objects, then oofun.__lt__() *will* be called. That's how list.sort() works. I was wrong: oofun.__eq__() won't be called by the sorting. You are probably not seeing the oofun.__eq__() problem in that code because of an implementation detail in Python: dicts will check identity first before trying to compare with __eq__(). You may be having problems in the construction of the pointDerivates dict or ooVarsIndDict outside of this code snippet, so if you just ran your debugger over this code snippet, you would not detect those calls. However, if you are not seeing the oofun.__lt__() calls from the sorting with your debugger, then your debugger may be missing the oofun.__eq__() calls, too. By all means, if you still think the bug is in someone else's code, please post a short example that other people can run that will demonstrate the problem. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Add ability to disable the autogeneration of the function signature in a ufunc docstring.
On Fri, Mar 15, 2013 at 9:19 PM, Pauli Virtanen p...@iki.fi wrote: 15.03.2013 22:39, Nathaniel Smith kirjoitti: [clip] - Something else... How about: scrap the automatic signatures altogether, and directly use the docstring provided to the ufunc creation function? I suspect ufuncs are not very widely used in 3rd party code, as it requires somewhat tricky messing with the C API. The backwards compatibility issue is also just a documentation issue, so nothing drastic. True enough. I guess a question is how much it bothers us that there are tons of ufunc arguments that are just not mentioned in the interpreter docstrings: http://docs.scipy.org/doc/numpy/reference/ufuncs.html#optional-keyword-arguments Obviously not a huge amount of we'd have altered the auto-generation already to include them :-) But IMHO it would be kind of nice if ?np.add mentioned the existence of things like where= and dtype=... and if we decide that docstrings ought to mention such things, then it's going to be a right hassle updating them all by hand every time some new ufunc feature is added. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
