Re: [Numpy-discussion] Python 3 dict support (issue #5718)

2016-07-21 Thread Joseph Fox-Rabinovitz 
I will do my best. I am not that familiar with rst or numpy docs, but
that's what PRs are for after all.

-Joe

On Thu, Jul 21, 2016 at 12:09 PM, Marten van Kerkwijk
 wrote:
> Yes, indeed, where should this be!?
>
> The logical place would be in the developer documentation, under
> "understanding the code & getting started" [1], but that is right now just a
> single paragraph.
>
> Others should definitely correct me if wrong, but I *think* this simply does
> not exist, and would suggest to move that subsection to its own section, and
> make your contribution a first entry in it ("the path to creating an array"
> or so). Hopefully, someone else will at some point add, e.g., a section on
> how ufuncs work (I always get quite confused following the long chain
> there...).  Or even something as simple as copying an array (where
> transposing axes can have major impacts on speed).
>
> All the best,
>
> Marten
>
> [1]
> http://docs.scipy.org/doc/numpy-dev/dev/development_environment.html#understanding-the-code-getting-started
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Python 3 dict support (issue #5718)

2016-07-21 Thread Marten van Kerkwijk 
I know it is slightly obnoxious to hold the "making a suggestion is to
volunteer for it" -- but usually a PR to the docs is best made by someone
who is trying to understand it rather than someone who already knows
everything
-- Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Python 3 dict support (issue #5718)

2016-07-21 Thread Jaime Fernández del Río 
On Wed, Jul 20, 2016 at 9:39 PM, Hannah  wrote:

> I second (and third & fourth &...) this
>
> Thanks so much for this, Jaime, it's exactly what I was looking for and
> couldn't find. Maybe this can be linked to in the contribute docs as an
> "orienting yourself in the codebase" page?
>
Glad to know it helped, realizing how this works was one of the big leaps
in understanding the code base for myself too.  We (my wife and I) have
sent the kids on vacation with their grandparents, so when we are done
enjoying our shortly recovered freedom hunting for Pokemons around Zurich,
I'll try to find time to write this up in a form suitable for the docs,
hopefully over the next couple of weeks.

I have been known to not keep commitments like this in the past, so don't
hold your breath just in case...

Jaime


>
> On Wed, Jul 20, 2016 at 11:07 AM, Joseph Fox-Rabinovitz <
> jfoxrabinov...@gmail.com> wrote:
>
>> Jaime,
>>
>> This is a great intro for people looking to jump into the C side of
>> things. I have been trying to figure out which bits are the important
>> ones from looking at the code and the docs. Your post cut out most of
>> the confusion. Is there some way you would consider adding something
>> like this this to the docs?
>>
>> -Joe
>>
>>
>> On Wed, Jul 20, 2016 at 8:52 AM, Jaime Fernández del Río
>>  wrote:
>> > On Wed, Jul 20, 2016 at 4:28 AM, Hannah  wrote:
>> >>
>> >> Hi,
>> >> I started venturing down the rabbit hole of trying to write a patch to
>> add
>> >> support for numpy to convert python 3 dictionary keys
>> >> (collections.abc.ViewMapping objects), which is open issue #5718 and am
>> >> having trouble orienting myself. I'm unclear as to where the python
>> entry
>> >> point into array is (basically, what function np.array drops into and
>> if
>> >> this is in Python or C) and where/what language (fine with writing
>> either) a
>> >> patch that supports MappingViews would go. Any help getting oriented
>> would
>> >> be much appreciated.
>> >
>> >
>> > Hi Hannah,
>> >
>> > ǹp.array is written in C, and is part of the multiarray module that is
>> > defined here:
>> >
>> >
>> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c
>> >
>> > The "array" name is mapped here:
>> >
>> >
>> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L4093
>> >
>> > to the function _array_fromobject defined here:
>> >
>> >
>> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L1557
>> >
>> > That functions does some checking and has a couple of fast paths for the
>> > case where the input is already an array or a subclass, but for the
>> general
>> > case it relies on PyArray_CheckFromAny:
>> >
>> >
>> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1848
>> >
>> > which in turn calls Pyarray_FromAny:
>> >
>> >
>> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1674
>> >
>> > You will also haveto take a look at what goes on in
>> > PyArray_GetArrayParamsFromObject, which gets called by PyArray_FromAny;
>> >
>> >
>> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1428
>> >
>> > as well as several other places, but I think they are all (or most of
>> them)
>> > in ctors.c.
>> >
>> > You may also want to take a llok at PyArray_FromIter, which is the
>> function
>> > that ultimately takes care of calls to np.fromiter:
>> >
>> >
>> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L3657
>> >
>> > It's messy, but not that bad once you get used to it: good luck!
>> >
>> > Jaime
>> >
>> >>
>> >>
>> >> The reasoning for the patch is s that dicts are one of the most common
>> >> Python datatypes and this specifically is because of an upstream issue
>> of
>> >> wanting dict support in matplotlib.
>> >>
>> >> Thanks,
>> >> Hannah
>> >>
>> >> ___
>> >> NumPy-Discussion mailing list
>> >> NumPy-Discussion@scipy.org
>> >> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >>
>> >
>> >
>> >
>> > --
>> > (\__/)
>> > ( O.o)
>> > ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus
>> planes de
>> > dominación mundial.
>> >
>> > ___
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion@scipy.org
>> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
(\__/)
( O.o)
( > <) Este es Conejo.

Re: [Numpy-discussion] Python 3 dict support (issue #5718)

2016-07-20 Thread Hannah 
I second (and third & fourth &...) this

Thanks so much for this, Jaime, it's exactly what I was looking for and
couldn't find. Maybe this can be linked to in the contribute docs as an
"orienting yourself in the codebase" page?

On Wed, Jul 20, 2016 at 11:07 AM, Joseph Fox-Rabinovitz <
jfoxrabinov...@gmail.com> wrote:

> Jaime,
>
> This is a great intro for people looking to jump into the C side of
> things. I have been trying to figure out which bits are the important
> ones from looking at the code and the docs. Your post cut out most of
> the confusion. Is there some way you would consider adding something
> like this this to the docs?
>
> -Joe
>
>
> On Wed, Jul 20, 2016 at 8:52 AM, Jaime Fernández del Río
>  wrote:
> > On Wed, Jul 20, 2016 at 4:28 AM, Hannah  wrote:
> >>
> >> Hi,
> >> I started venturing down the rabbit hole of trying to write a patch to
> add
> >> support for numpy to convert python 3 dictionary keys
> >> (collections.abc.ViewMapping objects), which is open issue #5718 and am
> >> having trouble orienting myself. I'm unclear as to where the python
> entry
> >> point into array is (basically, what function np.array drops into and if
> >> this is in Python or C) and where/what language (fine with writing
> either) a
> >> patch that supports MappingViews would go. Any help getting oriented
> would
> >> be much appreciated.
> >
> >
> > Hi Hannah,
> >
> > ǹp.array is written in C, and is part of the multiarray module that is
> > defined here:
> >
> >
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c
> >
> > The "array" name is mapped here:
> >
> >
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L4093
> >
> > to the function _array_fromobject defined here:
> >
> >
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L1557
> >
> > That functions does some checking and has a couple of fast paths for the
> > case where the input is already an array or a subclass, but for the
> general
> > case it relies on PyArray_CheckFromAny:
> >
> >
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1848
> >
> > which in turn calls Pyarray_FromAny:
> >
> >
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1674
> >
> > You will also haveto take a look at what goes on in
> > PyArray_GetArrayParamsFromObject, which gets called by PyArray_FromAny;
> >
> >
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1428
> >
> > as well as several other places, but I think they are all (or most of
> them)
> > in ctors.c.
> >
> > You may also want to take a llok at PyArray_FromIter, which is the
> function
> > that ultimately takes care of calls to np.fromiter:
> >
> >
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L3657
> >
> > It's messy, but not that bad once you get used to it: good luck!
> >
> > Jaime
> >
> >>
> >>
> >> The reasoning for the patch is s that dicts are one of the most common
> >> Python datatypes and this specifically is because of an upstream issue
> of
> >> wanting dict support in matplotlib.
> >>
> >> Thanks,
> >> Hannah
> >>
> >> ___
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion@scipy.org
> >> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> >>
> >
> >
> >
> > --
> > (\__/)
> > ( O.o)
> > ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus
> planes de
> > dominación mundial.
> >
> > ___
> > NumPy-Discussion mailing list
> > NumPy-Discussion@scipy.org
> > https://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Python 3 dict support (issue #5718)

2016-07-20 Thread Joseph Fox-Rabinovitz 
Jaime,

This is a great intro for people looking to jump into the C side of
things. I have been trying to figure out which bits are the important
ones from looking at the code and the docs. Your post cut out most of
the confusion. Is there some way you would consider adding something
like this this to the docs?

-Joe


On Wed, Jul 20, 2016 at 8:52 AM, Jaime Fernández del Río
 wrote:
> On Wed, Jul 20, 2016 at 4:28 AM, Hannah  wrote:
>>
>> Hi,
>> I started venturing down the rabbit hole of trying to write a patch to add
>> support for numpy to convert python 3 dictionary keys
>> (collections.abc.ViewMapping objects), which is open issue #5718 and am
>> having trouble orienting myself. I'm unclear as to where the python entry
>> point into array is (basically, what function np.array drops into and if
>> this is in Python or C) and where/what language (fine with writing either) a
>> patch that supports MappingViews would go. Any help getting oriented would
>> be much appreciated.
>
>
> Hi Hannah,
>
> ǹp.array is written in C, and is part of the multiarray module that is
> defined here:
>
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c
>
> The "array" name is mapped here:
>
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L4093
>
> to the function _array_fromobject defined here:
>
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L1557
>
> That functions does some checking and has a couple of fast paths for the
> case where the input is already an array or a subclass, but for the general
> case it relies on PyArray_CheckFromAny:
>
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1848
>
> which in turn calls Pyarray_FromAny:
>
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1674
>
> You will also haveto take a look at what goes on in
> PyArray_GetArrayParamsFromObject, which gets called by PyArray_FromAny;
>
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1428
>
> as well as several other places, but I think they are all (or most of them)
> in ctors.c.
>
> You may also want to take a llok at PyArray_FromIter, which is the function
> that ultimately takes care of calls to np.fromiter:
>
> https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L3657
>
> It's messy, but not that bad once you get used to it: good luck!
>
> Jaime
>
>>
>>
>> The reasoning for the patch is s that dicts are one of the most common
>> Python datatypes and this specifically is because of an upstream issue of
>> wanting dict support in matplotlib.
>>
>> Thanks,
>> Hannah
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
>
> --
> (\__/)
> ( O.o)
> ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de
> dominación mundial.
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Python 3 dict support (issue #5718)

2016-07-20 Thread Jaime Fernández del Río 
On Wed, Jul 20, 2016 at 4:28 AM, Hannah  wrote:

> Hi,
> I started venturing down the rabbit hole of trying to write a patch to add
> support for numpy to convert python 3 dictionary keys
> (collections.abc.ViewMapping objects), which is open issue #5718 and am
> having trouble orienting myself. I'm unclear as to where the python entry
> point into array is (basically, what function np.array drops into and if
> this is in Python or C) and where/what language (fine with writing either)
> a patch that supports MappingViews would go. Any help getting oriented
> would be much appreciated.
>

Hi Hannah,

ǹp.array is written in C, and is part of the multiarray module that is
defined here:

https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c

The "array" name is mapped here:

https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L4093

to the function _array_fromobject defined here:

https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/multiarraymodule.c#L1557

That functions does some checking and has a couple of fast paths for the
case where the input is already an array or a subclass, but for the general
case it relies on PyArray_CheckFromAny:

https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1848

which in turn calls Pyarray_FromAny:

https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1674

You will also haveto take a look at what goes on in
PyArray_GetArrayParamsFromObject, which gets called by PyArray_FromAny;

https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L1428

as well as several other places, but I think they are all (or most of them)
in ctors.c.

You may also want to take a llok at PyArray_FromIter, which is the function
that ultimately takes care of calls to np.fromiter:

https://github.com/numpy/numpy/blob/maintenance/1.11.x/numpy/core/src/multiarray/ctors.c#L3657

It's messy, but not that bad once you get used to it: good luck!

Jaime


>
> The reasoning for the patch is s that dicts are one of the most common
> Python datatypes and this specifically is because of an upstream issue of
> wanting dict support in matplotlib.
>
> Thanks,
> Hannah
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación mundial.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Python 3 dict support (issue #5718)

2016-07-19 Thread Hannah 
Hi,
I started venturing down the rabbit hole of trying to write a patch to add
support for numpy to convert python 3 dictionary keys
(collections.abc.ViewMapping objects), which is open issue #5718 and am
having trouble orienting myself. I'm unclear as to where the python entry
point into array is (basically, what function np.array drops into and if
this is in Python or C) and where/what language (fine with writing either)
a patch that supports MappingViews would go. Any help getting oriented
would be much appreciated.

The reasoning for the patch is s that dicts are one of the most common
Python datatypes and this specifically is because of an upstream issue of
wanting dict support in matplotlib.

Thanks,
Hannah
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion