Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
So, for non-structured arrays, the consens is an Exception. The question is, which one. AttributeError would be fully backwards compatible. Existing code checks for the method and if it exists, the object has fields. ValueError would make more sense, as the value - the array - is in wrong format/structure/type. regards, On 2014-10-01 16:47, Eelco Hoogendoorn wrote: Ah yes; you can use.. from types import MethodType ...to dynamically add methods to specific instances of a type. This may be cleaner or more pythonic than performing a check within the method, I dunno. On Wed, Oct 1, 2014 at 4:41 PM, Benjamin Root ben.r...@ou.edu wrote: Actually, if I remember correctly, special methods show up in the ndarray object when the dtype is datetime64, right? On Wed, Oct 1, 2014 at 10:13 AM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: Well, the method will have to be present on all ndarrays, since structured arrays do not have a different type from regular arrays, only a different dtype. Thus the attribute has to be present regardless, but some Exception will have to be raised depending on the dtype, to make it quack like the kind of duck it is, so to speak. Indeed it seems like an atypical design pattern; but I don't see a problem with it. On Wed, Oct 1, 2014 at 4:08 PM, John Zwinck jzwi...@gmail.com wrote: On 1 Oct 2014 04:30, Stephan Hoyer sho...@gmail.com wrote: On Tue, Sep 30, 2014 at 1:22 PM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: On more careful reading of your words, I think we agree; indeed, if keys() is present is should return an iterable; but I don't think it should be present for non-structured arrays. Indeed, I think we do agree. The attribute can simply be missing (e.g., accessing it raises AttributeError) for non-structured arrays. I'm generally fine with this, though I would like to know if there is precedent for methods being present on structured arrays only. Even if there is no precedent I am still OK with the idea, I just think we should understand if how novel this will be. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion [1] ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion [1] ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion [1] Links: -- [1] http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
On Thu, Oct 2, 2014 at 6:27 PM, Sebastian Wagner se...@sebix.at wrote: So, for non-structured arrays, the consens is an Exception. The question is, which one. AttributeError would be fully backwards compatible. Existing code checks for the method and if it exists, the object has fields. ValueError would make more sense, as the value - the array - is in wrong format/structure/type. If a non-structured array has its keys() raise AttributeError, I think that hasattr(arr, keys) should return False, which implies that getattr(arr, keys) should throw AttributeError. This would require that ndarray.__getattribute__ raise AttributeError, meaning the attribute isn't here, not merely the attribute doesn't have a value now. Otherwise people may rightly complain when they interrogate an array to see if it has keys(), find out that it does, but then get an error upon calling it which could have been known ahead of time. Now, to actually implement it this way would seem to require setting the tp_getattro function pointer (https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_getattro). Currently PyArray_Type has this field as null. If I understand everything correctly, adding a non-null function pointer here would mean some small runtime overhead to resolve every attribute on ndarray. I could easily be missing some detail which would allow us to do what I described above without a performance hit, but someone better versed would need to explain how/why. If I'm right about all that, and if the consensus is that keys() should raise an exception when dtype.names is None, then perhaps raising ValueError is the only viable option. I'd appreciate opinions from those experienced in the details of the C API. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
On 1 Oct 2014 04:30, Stephan Hoyer sho...@gmail.com wrote: On Tue, Sep 30, 2014 at 1:22 PM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: On more careful reading of your words, I think we agree; indeed, if keys() is present is should return an iterable; but I don't think it should be present for non-structured arrays. Indeed, I think we do agree. The attribute can simply be missing (e.g., accessing it raises AttributeError) for non-structured arrays. I'm generally fine with this, though I would like to know if there is precedent for methods being present on structured arrays only. Even if there is no precedent I am still OK with the idea, I just think we should understand if how novel this will be. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
Well, the method will have to be present on all ndarrays, since structured arrays do not have a different type from regular arrays, only a different dtype. Thus the attribute has to be present regardless, but some Exception will have to be raised depending on the dtype, to make it quack like the kind of duck it is, so to speak. Indeed it seems like an atypical design pattern; but I don't see a problem with it. On Wed, Oct 1, 2014 at 4:08 PM, John Zwinck jzwi...@gmail.com wrote: On 1 Oct 2014 04:30, Stephan Hoyer sho...@gmail.com wrote: On Tue, Sep 30, 2014 at 1:22 PM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: On more careful reading of your words, I think we agree; indeed, if keys() is present is should return an iterable; but I don't think it should be present for non-structured arrays. Indeed, I think we do agree. The attribute can simply be missing (e.g., accessing it raises AttributeError) for non-structured arrays. I'm generally fine with this, though I would like to know if there is precedent for methods being present on structured arrays only. Even if there is no precedent I am still OK with the idea, I just think we should understand if how novel this will be. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
Actually, if I remember correctly, special methods show up in the ndarray object when the dtype is datetime64, right? On Wed, Oct 1, 2014 at 10:13 AM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: Well, the method will have to be present on all ndarrays, since structured arrays do not have a different type from regular arrays, only a different dtype. Thus the attribute has to be present regardless, but some Exception will have to be raised depending on the dtype, to make it quack like the kind of duck it is, so to speak. Indeed it seems like an atypical design pattern; but I don't see a problem with it. On Wed, Oct 1, 2014 at 4:08 PM, John Zwinck jzwi...@gmail.com wrote: On 1 Oct 2014 04:30, Stephan Hoyer sho...@gmail.com wrote: On Tue, Sep 30, 2014 at 1:22 PM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: On more careful reading of your words, I think we agree; indeed, if keys() is present is should return an iterable; but I don't think it should be present for non-structured arrays. Indeed, I think we do agree. The attribute can simply be missing (e.g., accessing it raises AttributeError) for non-structured arrays. I'm generally fine with this, though I would like to know if there is precedent for methods being present on structured arrays only. Even if there is no precedent I am still OK with the idea, I just think we should understand if how novel this will be. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
I first proposed this on GitHub: https://github.com/numpy/numpy/issues/5134 ; jaimefrio requested that I bring it to this list for discussion. My proposal is to add a keys() method to NumPy's array class ndarray. The behavior would be to return self.dtype.names, i.e. the column names for a structured array (and None when dtype.names is None, which it is for pure numeric arrays without named columns). I originally proposed to add a values() method also, but I am tabling that for now so we needn't discuss it in this thread. The motivation is to enhance the ability to use duck typing with NumPy arrays, Python dicts, and other types like Pandas DataFrames, h5py Files, and more. It's a fairly common thing to want to get the keys of a container, where keys is understood to be a sequence of values one can pass to __getitem__(), and this is exactly what I'm aiming at. Thoughts? John Zwinck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
Sounds fair to me. Indeed the ducktyping argument makes sense, and I have a hard time imagining any namespace conflicts or other confusion. Should this attribute return none for non-structured arrays, or simply be undefined? On Tue, Sep 30, 2014 at 12:49 PM, John Zwinck jzwi...@gmail.com wrote: I first proposed this on GitHub: https://github.com/numpy/numpy/issues/5134 ; jaimefrio requested that I bring it to this list for discussion. My proposal is to add a keys() method to NumPy's array class ndarray. The behavior would be to return self.dtype.names, i.e. the column names for a structured array (and None when dtype.names is None, which it is for pure numeric arrays without named columns). I originally proposed to add a values() method also, but I am tabling that for now so we needn't discuss it in this thread. The motivation is to enhance the ability to use duck typing with NumPy arrays, Python dicts, and other types like Pandas DataFrames, h5py Files, and more. It's a fairly common thing to want to get the keys of a container, where keys is understood to be a sequence of values one can pass to __getitem__(), and this is exactly what I'm aiming at. Thoughts? John Zwinck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
I am also +1. I have already used structured arrays to do keyword-based string formatting. This makes sense as well. Would this enable keyword argument expansion? On Tue, Sep 30, 2014 at 7:29 AM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: Sounds fair to me. Indeed the ducktyping argument makes sense, and I have a hard time imagining any namespace conflicts or other confusion. Should this attribute return none for non-structured arrays, or simply be undefined? On Tue, Sep 30, 2014 at 12:49 PM, John Zwinck jzwi...@gmail.com wrote: I first proposed this on GitHub: https://github.com/numpy/numpy/issues/5134 ; jaimefrio requested that I bring it to this list for discussion. My proposal is to add a keys() method to NumPy's array class ndarray. The behavior would be to return self.dtype.names, i.e. the column names for a structured array (and None when dtype.names is None, which it is for pure numeric arrays without named columns). I originally proposed to add a values() method also, but I am tabling that for now so we needn't discuss it in this thread. The motivation is to enhance the ability to use duck typing with NumPy arrays, Python dicts, and other types like Pandas DataFrames, h5py Files, and more. It's a fairly common thing to want to get the keys of a container, where keys is understood to be a sequence of values one can pass to __getitem__(), and this is exactly what I'm aiming at. Thoughts? John Zwinck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
I like this idea. But I am -1 on returning None if the array is unstructured. I expect .keys(), if present, to always return an iterable. In fact, this would break some of my existing code, which checks for the existence of keys as a way to do duck typed checks for dictionary like objects (e.g., including pandas.DataFrame): https://github.com/xray/xray/blob/v0.3/xray/core/utils.py#L165 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
So a non-structured array should return an empty list/iterable as its keys? That doesn't seem right to me, but perhaps you have a compelling example to the contrary. I mean, wouldn't we want the duck-typing to fail if it isn't a structured array? Throwing an attributeError seems like the best thing to do, from a duck-typing perspective. On Tue, Sep 30, 2014 at 8:05 PM, Stephan Hoyer sho...@gmail.com wrote: I like this idea. But I am -1 on returning None if the array is unstructured. I expect .keys(), if present, to always return an iterable. In fact, this would break some of my existing code, which checks for the existence of keys as a way to do duck typed checks for dictionary like objects (e.g., including pandas.DataFrame): https://github.com/xray/xray/blob/v0.3/xray/core/utils.py#L165 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
On more careful reading of your words, I think we agree; indeed, if keys() is present is should return an iterable; but I don't think it should be present for non-structured arrays. On Tue, Sep 30, 2014 at 10:21 PM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: So a non-structured array should return an empty list/iterable as its keys? That doesn't seem right to me, but perhaps you have a compelling example to the contrary. I mean, wouldn't we want the duck-typing to fail if it isn't a structured array? Throwing an attributeError seems like the best thing to do, from a duck-typing perspective. On Tue, Sep 30, 2014 at 8:05 PM, Stephan Hoyer sho...@gmail.com wrote: I like this idea. But I am -1 on returning None if the array is unstructured. I expect .keys(), if present, to always return an iterable. In fact, this would break some of my existing code, which checks for the existence of keys as a way to do duck typed checks for dictionary like objects (e.g., including pandas.DataFrame): https://github.com/xray/xray/blob/v0.3/xray/core/utils.py#L165 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal: add ndarray.keys() to return dtype.names
On Tue, Sep 30, 2014 at 1:22 PM, Eelco Hoogendoorn hoogendoorn.ee...@gmail.com wrote: On more careful reading of your words, I think we agree; indeed, if keys() is present is should return an iterable; but I don't think it should be present for non-structured arrays. Indeed, I think we do agree. The attribute can simply be missing (e.g., accessing it raises AttributeError) for non-structured arrays. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
