Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Ondřej Čertík]

Hi Travis,

On Sun, Sep 13, 2015 at 4:51 PM, Travis Oliphant  wrote:
> Hey all,
>
> I just wanted to clarify, that I am very excited about a few ideas I have
> --- but I don't have time myself to engage in the community process to get
> these changes into NumPy. However, those are real processes --- I've
> been coaching a few people in those processes for the past several years
> already.
>
> So, rather than do nothing, what I'm looking to do is to work with a few
> people who I can share my ideas with, get excited about the ideas, and then
> who will work with the community to get them implemented.   That's what I
> was announcing and talking about yesterday --- looking for interested people
> who want to work on NumPy *with* the NumPy community.
>
> In my enthusiasm, I realize that some may have mis-understood my intention.
> There is no 'imminent' fork, nor am I planning on doing some crazy amount of
> work that I then try to force on other developers of NumPy.
>
> What I'm planning to do is find people to train on NumPy code base (people
> to increase the diversity of the developers would be ideal -- but hard to
> accomplish).  I plan to train them on NumPy based on my experience, and on
> what I think should be done --- and then have *them* work through the
> community process and engage with others to get consensus (hopefully not
> losing too much in translation in the process --- but instead getting even
> better).
>
> During that process I will engage as a member of the community and help
> write NEPs and other documents and help clarify where it makes sense as I
> can.   I will be filtering for people that actually want to see NumPy get
> better.Until I identify the people and work with them, it will be hard
> to tell how this will best work.   So, stay tuned.
>
> If all goes well, what you should see in a few weeks time are specific
> proposals, a branch or two, and the beginnings of some pull requests.If
> you don't see that, then I will not have found the right people to help me,
> and we will all continue to go back to searching.
>
> While I'm expecting the best, in the worst case, we get additional people
> who know the NumPy code base and can help squash bugs as well as implement
> changes that are desired.Three things are needed if you want to
> participate in this:  1) A willingness to work with the open source
> community, 2) a deep knowledge of C and in-particular CPython's brand of C,
> and 3) a willingness to engage with me, do a mind-meld and dump around the
> NumPy code base, and then improve on what is in my head with the rest of the
> community.


I don't have time to do the work myself, but I'll be happy to help in
terms of the community. Travis has paid me in the past to do the NumPy
1.7 release, and implement/fix some things that Travis had in mind. I
had no prior involvement with the NumPy codebase / community, and
within few months I had to get myself familiar with the code, fix
release critical bugs, get my PRs accepted by the community,
eventually get push access (from the core developers, not from Travis)
so that I can merge other people's patches, and most importantly, make
the community allow me to actually do the release, even though I came
from outside. As far as I know, Travis didn't have to pull any strings
for me, and as far as I know, there was no friction in the community
either --- there was a job to be done, we got it done, and that was
it.

So I'll be happy to share my experience and help the person (if coming
from outside) how to do this. In this case a push access is not
necessary, so it's just about sending high quality PRs and good
communication.

Ondrej
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Nathaniel Smith]

Hi all,

Thanks, Travis, for the followup. I know some people were confused or
concerned by some points in Travis’s recent emails (as was I,
initially), but after checking in with Travis and the NumPy steering
council, it sounds like the main points of possible confusion are
actually things where we at least are actually on the same page. So to
avoid any uncertainty or miscommunication, I just want to reiterate
some hopefully-uncontroversial points here:

- There is currently no plan whatsoever for NumPy to drop Python 2
support; something like ~80% of our users are still using Python 2,
and we anticipate that both Python 2 and Python 3 will continue to
receive full support for the foreseeable future.

- While it is possible that there will eventually be a
compatibility-breaking “NumPy 2.0” release, it won’t happen without a
community consensus that this is a good idea, and that consensus has
not yet been reached.

- As a clarification on how NumPy governance works: While we all
continue to be grateful to Travis for his past contributions, in terms
of formal authority he stepped down from his leadership role in the
project several years ago, and the project has since switched to a
community-driven governance model [1]. Under this model, all
contributors participate as equal peers, and (outside of specific
exceptional circumstances) all contributors should be assumed to be
speaking only on behalf of themselves, not the project as a whole.

- Discussion about the best way to improve NumPy’s dtype system is
ongoing, and several approaches are under consideration. No proposal
will be accepted without review and consensus by the broader NumPy
community. We welcome anyone who’s interested in these issues to join
us on the mailing list -- the more input we have, the better the
result will be :-).

Apologies for the wide distribution of this message; I suggest any
followups be directed to numpy-discussion@scipy.org only.

Thanks,
- Nathaniel

[1] http://thread.gmane.org/gmane.comp.python.numeric.general/61106
(we’ll move it into the repository soon, we promise!)

On Sun, Sep 13, 2015 at 3:51 PM, Travis Oliphant  wrote:
> Hey all,
>
> I just wanted to clarify, that I am very excited about a few ideas I have
> --- but I don't have time myself to engage in the community process to get
> these changes into NumPy. However, those are real processes --- I've
> been coaching a few people in those processes for the past several years
> already.
>
> So, rather than do nothing, what I'm looking to do is to work with a few
> people who I can share my ideas with, get excited about the ideas, and then
> who will work with the community to get them implemented.   That's what I
> was announcing and talking about yesterday --- looking for interested people
> who want to work on NumPy *with* the NumPy community.
>
> In my enthusiasm, I realize that some may have mis-understood my intention.
> There is no 'imminent' fork, nor am I planning on doing some crazy amount of
> work that I then try to force on other developers of NumPy.
>
> What I'm planning to do is find people to train on NumPy code base (people
> to increase the diversity of the developers would be ideal -- but hard to
> accomplish).  I plan to train them on NumPy based on my experience, and on
> what I think should be done --- and then have *them* work through the
> community process and engage with others to get consensus (hopefully not
> losing too much in translation in the process --- but instead getting even
> better).
>
> During that process I will engage as a member of the community and help
> write NEPs and other documents and help clarify where it makes sense as I
> can.   I will be filtering for people that actually want to see NumPy get
> better.Until I identify the people and work with them, it will be hard
> to tell how this will best work.   So, stay tuned.
>
> If all goes well, what you should see in a few weeks time are specific
> proposals, a branch or two, and the beginnings of some pull requests.If
> you don't see that, then I will not have found the right people to help me,
> and we will all continue to go back to searching.
>
> While I'm expecting the best, in the worst case, we get additional people
> who know the NumPy code base and can help squash bugs as well as implement
> changes that are desired.Three things are needed if you want to
> participate in this:  1) A willingness to work with the open source
> community, 2) a deep knowledge of C and in-particular CPython's brand of C,
> and 3) a willingness to engage with me, do a mind-meld and dump around the
> NumPy code base, and then improve on what is in my head with the rest of the
> community.
>
> Thanks,
>
> -Travis
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
Nathaniel J. Smith -- http://vorpus.org

Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Chris Barker]

Travis,

I'm sure you appreciate that this might all look a bit scary, given the
recent discussion about numpy governance.

But it's an open-source project, and I, at least, fully understand that
going through a big process is NOT the way to get a new idea tried out and
implemented. So I think think this is a great development -- I know I want
to see something like this dtype work done.

So, as someone who has been around this community for a long time, and
dependent on Numeric, numarray, and numpy over the years, this looks like a
great development.

And, in fact, with the new governance effort -- I think less scary --
people can go off and work on a branch or fork, do good stuff, and we, as a
community, can be assured that API (or even ABI) changes won't be thrust
upon us unawares :-)

As for the technical details -- I get a bit lost, not fully understanding
the current dtype system either, but do your ideas take us in the direction
of having dtypes independent of the container and ufunc machinery -- and
thus easier to create new dtypes (even in Python?) 'cause that would be
great.

I hope you find the partner you're looking for -- that's a challenge!

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Travis Oliphant]

Hey Chris (limiting to NumPy only),

I've had some great conversations with Nathaniel in the past few days and
I'm glad he posted his thoughts so that there is no confusion about
governance or what I was implying.

With respect to governance, I'm very supportive of what everyone is doing
in organizing a governance document and approach and appreciate the effort
of Nathaniel and others to move this forward.   Nothing I said was meant to
imply differently.   I'm sorry if it made anyone nervous.

I'm a very enthusiastic person when I get an idea of what to do.   I like
to see things implemented.   In this case, it also turns out that in terms
of overall architecture, my ideas are actually very similar to Nathaniel's
ideas.   That's a good sign.We have different tactical approaches as to
how to move forward, but I think it's a good thing to note that we see a
very similar path forward.Nothing will be done in NumPy itself except
via pull-request and review.

My approach for the ideas I'm pursuing will be to organize people around
two new prototype packages I'm calling memtype and gufunc.   The purpose of
these is to allow playing with the design and ideas quickly before looking
at how to put them into NumPy itself --- there will also be some training
involved in getting people up to speed.There was a long discussion
today at this BIDS data-structures for data-science summit part of which
talked about how to improve NumPy's dtype system. I would love to these
independent objects evolve into independent packages that could even go
into Python standard library. Not everyone agrees that is the best
idea, but regardless of whether this happens or not, the intent is to do
work that could go into NumPy now.

I look forward to the activity.

-Travis




On Mon, Sep 14, 2015 at 10:46 AM, Chris Barker 
wrote:

> Travis,
>
> I'm sure you appreciate that this might all look a bit scary, given the
> recent discussion about numpy governance.
>
> But it's an open-source project, and I, at least, fully understand that
> going through a big process is NOT the way to get a new idea tried out and
> implemented. So I think think this is a great development -- I know I want
> to see something like this dtype work done.
>
> So, as someone who has been around this community for a long time, and
> dependent on Numeric, numarray, and numpy over the years, this looks like a
> great development.
>
> And, in fact, with the new governance effort -- I think less scary --
> people can go off and work on a branch or fork, do good stuff, and we, as a
> community, can be assured that API (or even ABI) changes won't be thrust
> upon us unawares :-)
>
> As for the technical details -- I get a bit lost, not fully understanding
> the current dtype system either, but do your ideas take us in the direction
> of having dtypes independent of the container and ufunc machinery -- and
> thus easier to create new dtypes (even in Python?) 'cause that would be
> great.
>
> I hope you find the partner you're looking for -- that's a challenge!
>
> -Chris
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Travis Oliphant]

Hey all,

I just wanted to clarify, that I am very excited about a few ideas I have
--- but I don't have time myself to engage in the community process to get
these changes into NumPy. However, those are real processes --- I've
been coaching a few people in those processes for the past several years
already.

So, rather than do nothing, what I'm looking to do is to work with a few
people who I can share my ideas with, get excited about the ideas, and then
who will work with the community to get them implemented.   That's what I
was announcing and talking about yesterday --- looking for interested
people who want to work on NumPy *with* the NumPy community.

In my enthusiasm, I realize that some may have mis-understood my
intention.  There is no 'imminent' fork, nor am I planning on doing some
crazy amount of work that I then try to force on other developers of NumPy.


What I'm planning to do is find people to train on NumPy code base (people
to increase the diversity of the developers would be ideal -- but hard to
accomplish).  I plan to train them on NumPy based on my experience, and on
what I think should be done --- and then have *them* work through the
community process and engage with others to get consensus (hopefully not
losing too much in translation in the process --- but instead getting even
better).

During that process I will engage as a member of the community and help
write NEPs and other documents and help clarify where it makes sense as I
can.   I will be filtering for people that actually want to see NumPy get
better.Until I identify the people and work with them, it will be hard
to tell how this will best work.   So, stay tuned.

If all goes well, what you should see in a few weeks time are specific
proposals, a branch or two, and the beginnings of some pull requests.If
you don't see that, then I will not have found the right people to help me,
and we will all continue to go back to searching.

While I'm expecting the best, in the worst case, we get additional people
who know the NumPy code base and can help squash bugs as well as implement
changes that are desired.Three things are needed if you want to
participate in this:  1) A willingness to work with the open source
community, 2) a deep knowledge of C and in-particular CPython's brand of C,
and 3) a willingness to engage with me, do a mind-meld and dump around the
NumPy code base, and then improve on what is in my head with the rest of
the community.

Thanks,

-Travis
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion