Re: [Numpy-discussion] Governance model request

[Ryan May]

This has to be one of the most bizarre threads I've ever read in my life.
Somehow companies are lurking around like the boogeyman and academics are
completely free of ulterior motives and conflicts of interest? This is just
asinine--we're all people and have various motivations. (Having just gotten
out of my university after 15 years, the idea that academics are somehow
immune to ulterior motives and conflicts of interest makes me laugh
hysterically.)

The sad part is that this worry completely unnecessary. This is an open
source project, not international politics, and the end goal is to produce
software. Therefore, 99.9% of the time motives (ulterior, profit, or
otherwise) are completely orthogonal to the question of: IS IT A GOOD
TECHNICAL IDEA? It's really that simple: does the proposed change move the
project in a direction that we want to go?

Now, for the 0.01% of the time, where nobody can agree on that answer, or
the question is non-technical, and there is concern about the motives of
members of the "council" (or whatnot), it's again simple: RECUSAL. It's a
simple concept that I learned in the godawful ethics class NSF forced grad
students to take: if you have a conflict of interest, you don't vote. It's
how the grownups from the Supreme Court to the college football playoff
deal with the fact that people WILL have conflicts; potential conflicts
don't disbar qualified individuals from being included in the group, just
from weighing in when their decisions can be clouded.

So how about we stop making up reasons to discourage participation by
(over-)qualified individuals, and actually take advantage of the fact that
people actually want to move numpy forward?

Ryan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Matthew Brett]

Hi,

On Tue, Sep 22, 2015 at 11:20 AM, Stefan van der Walt
 wrote:
> Hi Travis
>
> On 2015-09-22 03:44:12, Travis Oliphant  wrote:
>> I'm actually offended that so many at BIDS seem eager to crucify my
>> intentions when I've done nothing but give away my time, my energy, my
>> resources, and my sleep to NumPy for many, many years.I guess if your
>> intent is to drive me away, then you are succeeding.
>
> I guess we've gone off the rails pretty far at this point, so let me at
> least take a step back, and make sure that you know that:
>
> - I have never doubted that your intensions for NumPy are anything but
>   good (I know they are!),
> - I *want* the community to be a welcoming place for companies to
>   contribute (otherwise, I guess I'd not be such a fervent supporter of
>   the scientific eco-system using the BSD license), and
> - I love your enthusiasm for the project.  After all, that is a big part
>   of what inspired me to become involved in the first place.
>
> My goal is not to spread uncertainty, fear nor doubt—if that was the
> perception left, I apologize.
>
> I'll re-iterate that I wanted to highlight a concern about the
> interactions of a (somewhat weakly cohesive) community and strong,
> driven personalities such as yourself backed by a formidable amount of
> development power.  No matter how good your intensions are, there are
> risks involved in this kind of interaction, and if we fail to even
> *admit* that, we are in trouble.
>
> Lest the above be read in a negative light again, let me state it
> up-front: *I don't think you will hijack the project, use it for your
> own gain, or attempt to do anything you don't believe to be in the best
> interest of NumPy.* What I'm saying is that we absolutely need to move
> forward in a way that brings everyone along, and makes everyone rest
> assured that their voice will be heard.
>
> Also, please know that I have not discussed these matters with Nathaniel
> behind the scenes, other than an informal hour-long discussion about his
> original governance proposal.  There is no BIDS conspiracy or attempts
> at crucifixion.  After all, you were an invited guest speaker at an
> event I organized this weekend, since I value your opinion and insights.
>
> Either way, let me again apologize if my suggested lack of insight hurt
> people's feelings.  I can only hope that, in educating me, we all learn
> a few lessons.

I'm also in favor of taking a step back.

The point is, that a sensible organization and a sensible leader has
to take the possibility of conflict of interest into account.  They
also have to consider the perception of a conflict of interest.

It is the opposite of sensible, to respond to this with 'how dare you"
or by asserting that this could never happen or by saying that we
shouldn't talk about that in case people get frightened.  I point you
again to Linus' interview [1].  He is not upset that he has been
insulted by the implication of conflict of interest, he soberly
accepts that this will always be an issue, with companies in
particular, and goes out of his way to address that in an explicit and
reasonable way.

Cheers,

Matthew

[1] http://www.bbc.com/news/technology-18419231
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[James E.H. Turner]

I don't think I've contributed code to NumPy itself, but as someone
involved in the scientific python ecosystem for a while, I can't see
why people would consider Continuum less of a legitimate participant
or community member than individual contributors, especially if the
person behind it has had the opportunity to control things previously
and instead passed NumPy onto the community. I'd be wary of commercial
interests dominating the agenda, but that's different from them having
a proportionate (in this case minor) say when they have something to
offer. And that's all true *even if* Travis were heavily biased to his
own commercial ends, which is not consistent with my understanding of
his wider efforts and sacrifices over most of a decade that I've been
paying attention.

Remember this is free/open source software and if enough people don't
like the committee at some point, the project can be forked as an
option of last resort. Nothing is set in stone, nor code lost.

Just saying (I probably won't reply to any criticism or corrections,
to avoid adding peripheral noise/heat to the thread).

Cheers,

James (from, but not on behalf of, a non-profit research facility).

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Benjamin Root]

To expand on Ryan's point a bit about recusal... this is why we have a
general policy against self-merging and why peer review is so valuable. A
ban on self-merging is much like recusal, and I think it is a fantastic
policy.

As for a BDFL, I used to like that idea having seen it work well for Linux
and Python, but I have found it at odds with the policy of recusal and no
self-merging. That said, as I am sure Thomas Caswell can attest, a
non-self-merging policy can result in a lot of ideas getting stalled,
waiting for review that may or may not happen. I don't know what the
solution is, but I am sympathetic to those who are apprehensive about a
BDFL -- regardless of who is in that role.

Ben Root


On Tue, Sep 22, 2015 at 2:20 PM, Stefan van der Walt 
wrote:

> Hi Travis
>
> On 2015-09-22 03:44:12, Travis Oliphant  wrote:
> > I'm actually offended that so many at BIDS seem eager to crucify my
> > intentions when I've done nothing but give away my time, my energy, my
> > resources, and my sleep to NumPy for many, many years.I guess if your
> > intent is to drive me away, then you are succeeding.
>
> I guess we've gone off the rails pretty far at this point, so let me at
> least take a step back, and make sure that you know that:
>
> - I have never doubted that your intensions for NumPy are anything but
>   good (I know they are!),
> - I *want* the community to be a welcoming place for companies to
>   contribute (otherwise, I guess I'd not be such a fervent supporter of
>   the scientific eco-system using the BSD license), and
> - I love your enthusiasm for the project.  After all, that is a big part
>   of what inspired me to become involved in the first place.
>
> My goal is not to spread uncertainty, fear nor doubt—if that was the
> perception left, I apologize.
>
> I'll re-iterate that I wanted to highlight a concern about the
> interactions of a (somewhat weakly cohesive) community and strong,
> driven personalities such as yourself backed by a formidable amount of
> development power.  No matter how good your intensions are, there are
> risks involved in this kind of interaction, and if we fail to even
> *admit* that, we are in trouble.
>
> Lest the above be read in a negative light again, let me state it
> up-front: *I don't think you will hijack the project, use it for your
> own gain, or attempt to do anything you don't believe to be in the best
> interest of NumPy.* What I'm saying is that we absolutely need to move
> forward in a way that brings everyone along, and makes everyone rest
> assured that their voice will be heard.
>
> Also, please know that I have not discussed these matters with Nathaniel
> behind the scenes, other than an informal hour-long discussion about his
> original governance proposal.  There is no BIDS conspiracy or attempts
> at crucifixion.  After all, you were an invited guest speaker at an
> event I organized this weekend, since I value your opinion and insights.
>
> Either way, let me again apologize if my suggested lack of insight hurt
> people's feelings.  I can only hope that, in educating me, we all learn
> a few lessons.
>
> Stéfan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Commit rights for Allan Haldane

[Charles R Harris]

Hi All,

Allan Haldane has been given commit rights. Here's to the new member of the
team.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Commit rights for Allan Haldane

[Jaime Fernández del Río]

Congrats Allan!

Jaime

On Tue, Sep 22, 2015 at 11:54 AM, Charles R Harris <
charlesr.har...@gmail.com> wrote:

> Hi All,
>
> Allan Haldane has been given commit rights. Here's to the new member of
> the team.
>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación mundial.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

Thank you for posting that draft as it is a useful comparison to borrow
from.   I think Nathaniel's original document is a great start.   Perhaps
some tweaks along the lines of what you and Matt have suggested could also
be useful.

I agree that my proposal is mostly about altering the governance model,
mixed with some concern about being "automatically disqualified" from a
council that can decide the future of NumPy if things don't move forward.

-Travis


On Tue, Sep 22, 2015 at 12:57 AM, Stefan van der Walt 
wrote:

> On 2015-09-20 11:20:28, Travis Oliphant  wrote:
> > I would recommend three possible adjustments to the steering council
> > concept.
> >
> > 1 - define a BDFL for the council.  I would nominate chuck Harris
> >
> > 2 - limit the council to 3 people.  I would nominate chuck, nathaniel,
> and
> > pauli.
> >
> > 3 - add me as a permanent member of the steering council.
>
> I would split the above into two parts: a suggestion on how to change
> the governance model (first half of 1 and 2) and then some thoughts on
> what to do once those changes have been made (latter half of 1 and 2, as
> well as 3).
>
> For now, since those changes are not in place yet, it's probably best
> to focus on the governance model.
>
> I would agree that one person (or a very small group) is best suited to
> "getting things unstuck".  And, personally, I believe it best for that
> person/persons to be elected by the community (whatever we define "the
> community" to be)---which is what I presume you suggested when you
> mentioned nominating candidates.
>
> Since Matthew mentioned the governance proposal we're working on, here
> is a very early draft:
>
>
> https://github.com/stefanv/skimage-org/blob/governance_proposal/governance.md
>
> As I said, this is still a work-in-progress--comments are welcome.
> E.g., the weighting element in the voting has to be fine tuned (but was
> put in place to prevent rapid take-overs).
>
> Essentially, we need:
>
> - a way for community members to express disagreement without being
>   ousted,
> - protection against individuals who want to exert disproportional
>   influence,
> - protection against those in leadership roles who cause the project
>   long-term harm,
> - and a way for the community to change the direction of the project if
>   they so wished.
>
> Stéfan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Stefan van der Walt]

Hi Travis

On 2015-09-21 23:29:12, Travis Oliphant  wrote:
>   1) nobody believes that the community should be forced to adopt numba as
> part of ufunc core yet --- but this could happen someday just as Cython is
> now being adopted but was proposed 8 years ago that it "could be adopted"
> That's a red-hearing.

Yes, I'd like to clarify: I was not against including any specific
technology in NumPy.  I was highlighting that there may be different
motivations for members of the general community and those working for,
say, Continuum, to get certain features adopted.

>   2) I have stated that breaking the ABI is of little consequence because
> of conda as well as other tools.I still believe that.  This has nothing
> to do with any benefit Continuum might or might not receive because of
> conda.   Everyone else who wants to make a conda-based distribution also
> benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits.
> I don't think the community realizes the damange that is done with FUD like
> this.  There are real implications.  It halts progress, creates confusion,
> and I think ultimately damages the community.

This is an old argument, and the reason why we have extensive measures
in place to guard against ABI breakage.  But, reading what you wrote
above, I would like to understand better what FUD you are referring to,
because I, rightly or wrongly, believe there is a real concern here that
is being glossed over.

> I don't see how.None of these have been proposed for integrating into
> NumPy.I don't see how integrating numba into NumPy benefits Continuum
> at all.  It's much easier for us to keep it separate.   At this point
> Continuum doesn't have an opinion about integrating DyND into NumPy or
> not.

I think that touches, tangentially at least, on the problem.  If an
employee of Continuum were steering NumPy, and the company developed an
opinion on those integrations, would such a person not feel compelled to
toe the company line?  (Whether the company is Continuum or another is
besides the point—I am only trying to understand the dynamics of working
for a company and leading an open source project that closely interacts
with their producs.)

> I know that you were responding to specific question by Brian as to how
> their could be a conflict of interest for Continuum and NumPy development.
> I don't think this is a useful conversation --- we could dream up all
> kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS
> really wants Spark to take over and for NumPy to have special connections
> to Spark).   Are we to not allow anyone at BIDS to participate in the
> steering council because of their other interests?

I guess that's an interesting example, but BIDS (which sits inside a
university and is funded primarily by foundations) has no financial, and
very few other, incentives to do so.

> But remember, the original point is whether or not someone from Continuum
> (or I presume any company and not just singling out Continuum for special
> treatment) should be on the steering council.Are you really arguing
> that they shouldn't because there are other projects Continuum is working
> on that have some overlap with NumPy.I really hope you don't actually
> believe that.

Here's what I'm trying to say (and I apologise for ruffling feathers in
the process):

There are concerns amongst members of the community that (will) arise
when strong players from industry try / hint at exerting (some)
executive control over NumPy.  We can say that these concerns amount to
spreading FUD, that they are uninformed, unrealistic, etc., but
ultimately they are still out there, and until they are discussed and
addressed, I find it hard to see how we can move forward with ease.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.10.0rc1 coming tomorrow, 22 Sept.

[Travis Oliphant]

Of course it will be 1.10.0 final where all the problems will show up
suddenly :-)

Perhaps we can get to where we are testing Anaconda against beta releases
better.

-Travis


On Mon, Sep 21, 2015 at 5:19 PM, Charles R Harris  wrote:

> Hi All,
>
> Just a heads up. The lack of reported problems in 1.10.0b1 has been
> stunning.
>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Stefan van der Walt]

On 2015-09-21 22:15:55, Bryan Van de Ven  wrote:
> Beyond that, what (even in a broad sense) is an example of a goal that
> "Continuum might need" that would conceivably do detriment to the
> NumPy community? That it be faster? Simpler to maintain? Easier to
> extend? Integrate better with more OS projects? Attract new active
> developers? Receive more financial support? Grow its user base even
> more?

I don't know how productive it is to dream up examples, but it's not
very hard to do.  Currently, e.g., the community is not ready to adopt
numba as part of the ufunc core.  But it's been stated by some that,
with so many people running Conda, breaking the ABI is of little
consequence.  And then it wouldn't be much of a leap to think that numba
is an acceptable dependency.

There's a broad range of Continuum projects that intersect with what
NumPy does: numba, DyND, dask and Odo to name a few.  Integrating them
into NumPy may make a lot more sense for someone from Continuum than for
other members of the community.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Stefan van der Walt]

Hi Brian

On 2015-09-21 23:28:48, Bryan Van de Ven  wrote:
>> very hard to do.  Currently, e.g., the community is not ready to adopt
>> numba as part of the ufunc core.  But it's been stated by some that,
>
> Who are you speaking for? The entire community? Under what mandate?

I am speaking on behalf of myself, under no mandate.

> The current somewhat concrete proposal I am aware of involves funding
> cleaning up dtypes. Is there another concrete, credible proposal to
> make Numba a dependency of NumPy that you can refer to? If not, why
> are we mired in hypotheticals?

I'm sorry if I misunderstood, but I thought you wanted us to explore
hypothetical confictual situations.

> May? Can you elaborate? More speculation. My own position is that
> these projects want to integrate with NumPy, not the
> converse. Regardless of my opinion, can you actually make any specific
> arguements, one way or the otehr? What if if some integrations
> actually make more sense for the community? Is this simply a dogmatic
> ideological position that anything whatsoever that benefits both NumPy
> and Continuum simultaneously is bad, on principle? That's fine, as
> such, but let's make that position explicit if that's all it is.

No, I don't have such a dogmatic ideological position.  I think,
however, that it is somewhat unimaginative to propose that there are no
potential conflicts whatsoever.

I am happy if we can find solutions that benefit both numpy and any
company out there.  But in the end, I'm sure you'd agree that we want
the decisions that lead to such solutions to be taken in the best
interest of the project, and not be weighed by alterior motivations of
any sorts.  In the end, even the *perception* that that is not the case
can be very harmful.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Nathaniel Smith]

On Mon, Sep 21, 2015 at 9:20 AM, Travis Oliphant  wrote:
>
> I wrote my recommendations quickly before heading on a plane.I hope the 
> spirit of them was caught correctly.I also want to re-emphasize that I 
> completely understand that the Steering Council is not to be making decisions 
> that often and almost all activity will be similar to it is now --- 
> discussion, debate, proposals, and pull-requests --- that is a good thing.
>
> However, there is a need for leadership to help unstick things and move the 
> project forward from time to time because quite often doing *something* can 
> be better than trying to please everyone with a voice.   My concerns about 
> how to do this judgment have 2 major components:
>
> 1) The need for long-term consistency --- a one-year horizon on defining this 
> group is too short in my mind for a decades-old project like NumPy.
> 2) The group that helps unstick things needs to be small (1, 3, or 5 at the 
> most)

For reference, the rules for steering council membership were taken
directly from those used by the Jupyter project, and their steering
council currently has 10 people, making it larger than the "seed
council" proposed in the numpy document:
https://github.com/jupyter/governance/blob/master/people.md

> We could call this group the "adjudication group" rather than the "Steering 
> Council" as well.   I could see that having a formal method of changing that 
> "adjudication group" would be a good idea as well (and perhaps that formal 
> vote could be made by a vote of a group of active contributors.   In that 
> case, I would define active as having a time-window of 5 years instead of 
> just 1).

I may be misreading things, but I'm getting the impression that the
active "adjudication group" you envision is radically different from
the "steering council" as envisioned by the current governance
document. It also, I think, radically different from anything I've
ever seen in a functioning community-run FOSS project and frankly it's
something where if I saw a project using this model, it would make me
extremely wary about contributing.

The key point that I think differs is that you envision that this
"adjudication group" will actually intervene into discussions and make
formal decisions in situations other than true irreconcilable crises,
which in my estimation happen approximately never. The only two kinds
of F/OSS projects that I can think of that run like this are (a)
projects that are not really community driven at all, but rather run
as internal company projects that happen to have a public repository,
(b) massive projects like Debian and Fedora that have to manage
literally thousands of contributors, and thus have especially robust
backstop procedures to handle the rare truly irreconcilable situation.

E.g., the Debian CTTE acts as an "adjudication group" in the way it
sounds like you envision it: on a regular basis, irreconcilable
arguments in Debian get taken to them to decide, and they issue a
ruling. By some back of the envelope calculations, it looks like they
issue approximately ~0.002 rulings per debian-contributor-year [1][2].
If we assume crudely that irreconcilable differences scale linearly
with the size of a project, this suggests that a ~20 person project
like NumPy should require a ruling ~once every 20 years.

Or quoting myself from the last thread about this [3]:
] Or on the other end of things, you have e.g. Subversion, which had an
] elaborate defined governance system with different levels of
] "core-ness", a voting system, etc. -- and they were 6 years into the
] project before they had their first vote. (The vote was on the crucial
] technical decision of whether to write function calls like "f ()" or
] "f()".)

These are two real projects and how they really work. And even in
projects that do have a BDFL, the successful ones almost never use
this power to actually "unstick things" (i.e., use their formal power
to resolve a discussion). Consider PEP 484, Guido's somewhat
controversial type hints proposal: rather than use his power to move
the debate along, he explicitly delegated his power to one of the
idea's strongest critics [4].

Of course, things to get stuck. But the only time that getting them
unstuck needs or even benefits from the existence of a formal
"unsticking things" group is if the group is actually using some
powers that they have. In 99.9% of cases, though, the correct way to
get things unstuck is for a new idea to be introduced, or for someone
respected to act as a mediator, or for someone to summarize the
situation and isolate some core of agreement that allows for forward
progress. But all of *these* things can and should be done by anyone
and everyone who can contribute them -- restricting them to a small
"adjudication group" makes no sense.

In terms of real-world examples: From my point of view, the worst
parts of the NA fiasco was caused by the decision to cut off debate
with a 

Re: [Numpy-discussion] Governance model request

[Marten van Kerkwijk]

Hi All,

I've been reading this thread with amazement and a bit of worry. It seems
Nathaniel's proposal is clearly an improvement, even if it is not perfect.
But it is in the end for a project where, at least as seen from the
outside, the main challenge is not in governance, but rather in having only
a small group of people who understand the code well enough that they are
able to make and judge modifications. As such, this discussion doesn't seem
worthy of the effort, and even less of needless heat and irritation, of the
type that seems unlikely would have arisen if this conversation had been in
person instead of per e-mail.

Might it be an idea to accept the proposal provisionally, returning to it a
year from now with practical experience? This certainly has the benefit of
allowing to switch focus to the more pressing and fortunately also more
interesting work to be done on interfacing numpy/ndarray nicely with other
classes (i.e., __numpy_ufunc__ and/or similar, and the dtype
generalisations, which have me quite intrigued -- either might be very
interesting for the Quantity class in astropy, as well as for a
work-in-progress Variable class [which propagates uncertainties including
covariances]).
​
All the best,

Marten

-- 
Prof. M. H. van Kerkwijk
Dept. of Astronomy & Astroph., 50 St George St., Toronto, ON, M5S 3H4,
Canada
McLennan Labs, room 1203B, tel: +1(416)9467288, fax: +1(416)9467287
m...@astro.utoronto.ca, http://www.astro.utoronto.ca/~mhvk
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Stefan van der Walt]

Hi Travis

On 2015-09-22 03:44:12, Travis Oliphant  wrote:
> I'm actually offended that so many at BIDS seem eager to crucify my
> intentions when I've done nothing but give away my time, my energy, my
> resources, and my sleep to NumPy for many, many years.I guess if your
> intent is to drive me away, then you are succeeding.

I guess we've gone off the rails pretty far at this point, so let me at
least take a step back, and make sure that you know that:

- I have never doubted that your intensions for NumPy are anything but
  good (I know they are!),
- I *want* the community to be a welcoming place for companies to
  contribute (otherwise, I guess I'd not be such a fervent supporter of
  the scientific eco-system using the BSD license), and
- I love your enthusiasm for the project.  After all, that is a big part
  of what inspired me to become involved in the first place.

My goal is not to spread uncertainty, fear nor doubt—if that was the
perception left, I apologize.

I'll re-iterate that I wanted to highlight a concern about the
interactions of a (somewhat weakly cohesive) community and strong,
driven personalities such as yourself backed by a formidable amount of
development power.  No matter how good your intensions are, there are
risks involved in this kind of interaction, and if we fail to even
*admit* that, we are in trouble.

Lest the above be read in a negative light again, let me state it
up-front: *I don't think you will hijack the project, use it for your
own gain, or attempt to do anything you don't believe to be in the best
interest of NumPy.* What I'm saying is that we absolutely need to move
forward in a way that brings everyone along, and makes everyone rest
assured that their voice will be heard.

Also, please know that I have not discussed these matters with Nathaniel
behind the scenes, other than an informal hour-long discussion about his
original governance proposal.  There is no BIDS conspiracy or attempts
at crucifixion.  After all, you were an invited guest speaker at an
event I organized this weekend, since I value your opinion and insights.

Either way, let me again apologize if my suggested lack of insight hurt
people's feelings.  I can only hope that, in educating me, we all learn
a few lessons.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Commit rights for Allan Haldane

[Allan Haldane]

Thanks all. I'm very happy to contribute back to a project which has
been so useful to me over many years!

On 09/22/2015 04:53 PM, Travis Oliphant wrote:
> Excellent news!   Welcome Allan.  
> 
> -Travis
> 
> 
> On Tue, Sep 22, 2015 at 1:54 PM, Charles R Harris
> > wrote:
> 
> Hi All,
> 
> Allan Haldane has been given commit rights. Here's to the new member
> of the team.
> 
> Chuck
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org 
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> 
> 
> 
> -- 
> *
> Travis Oliphant*
> /Co-founder and CEO/
> /
> /
> 
> @teoliphant
> 512-222-5440
> http://www.continuum.io
> 
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Commit rights for Allan Haldane

[Travis Oliphant]

Excellent news!   Welcome Allan.

-Travis


On Tue, Sep 22, 2015 at 1:54 PM, Charles R Harris  wrote:

> Hi All,
>
> Allan Haldane has been given commit rights. Here's to the new member of
> the team.
>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] facebook, twitter, and g+

[Bryan Van de Ven]

> On Sep 22, 2015, at 9:58 PM, Charles R Harris  
> wrote:
> 
> Hi All,
> 
> Just posting to elicit thoughts about scipy.org having a presence in social 
> media for announcements.

Of the ones listed in the subject, I would suggest Twitter is the most 
valuable. It has been great for release and other announcements. I find 
Facebook and G+ a little odd for software projects personally, though I guess 
they could be useful if you wanted to have somewhat longer "posts". But of 
course that's also yet more for someone to have maintain.

One additional consideration is that some people will inevitably attempt to 
elicit technical support over these channels. I generally have to steer people 
to the mailing list immediately. 

Bryan 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ANN: Numpy 1.10.0rc1 released.

[Charles R Harris]

Hi all,

I'm pleased to announce the availability of Numpy 1.10.0rc1. Sources and 32
bit binary packages for Windows may be found at Sourceforge
.
Please test this out, as I would like to move to the final release as
rapidly as possible and the lack of error reports from the beta has left me
nervous. It's been quiet, too quiet. In the movies, we would all die in the
next five minutes.

Cheers

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Charles R Harris]

On Tue, Sep 22, 2015 at 8:55 PM, Bryan Van de Ven 
wrote:

>
> > On Sep 22, 2015, at 1:48 PM, Matthew Brett 
> wrote:
> >
> > The point is, that a sensible organization and a sensible leader has
> > to take the possibility of conflict of interest into account.  They
> > also have to consider the perception of a conflict of interest.
>
> Of course, and the policies to deal with conflicts have deal with the
> possibility that *anyone* at all might have conflict. But that was not your
> suggestion. Your suggestion was to make one individual be subject to
> additional scrutiny that no one else would be subject to. Please explain
> why should one person be singled out for a special "six-month waiting
> period" when the exact same possibility for conflict exists with anyone who
> is ever on the committee?
>
> > It is the opposite of sensible, to respond to this with 'how dare you"
> > or by asserting that this could never happen or by saying that we
> > shouldn't talk about that in case people get frightened.  I point you
>
> Red herring. The objection (as many people have now voiced) is the double
> standard you proposed.
>
> > again to Linus' interview [1].  He is not upset that he has been
> > insulted by the implication of conflict of interest, he soberly
> > accepts that this will always be an issue, with companies in
> > particular, and goes out of his way to address that in an explicit and
> > reasonable way.
>
> Your selective quotation is impressive. Also in that interview, Linus
> points out that his employment contract is probably "unique in the entire
> world". Which means in practical terms that the details of what he has does
> are fairly well irrelevant to any other situation. So what is the point in
> bringing it up, at all, except to try and diminish someone else by
> comparison?
>
> (I'm done.)
>

Andrew Morton would be a prominent counter example in the Linux community.
He is employed by Google. Many other Linux developers are employed by Red
Hat, Intel, and other companies. At this point Linus relies on those people
for decisions on what goes into the kernel; it is much too big for a single
person to deal with even in review. I also expect the Linux community would
be overjoyed if every company provided drivers for their hardware. Subject
to review, of course.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Bryan Van de Ven]

> On Sep 22, 2015, at 1:48 PM, Matthew Brett  wrote:
> 
> The point is, that a sensible organization and a sensible leader has
> to take the possibility of conflict of interest into account.  They
> also have to consider the perception of a conflict of interest.

Of course, and the policies to deal with conflicts have deal with the 
possibility that *anyone* at all might have conflict. But that was not your 
suggestion. Your suggestion was to make one individual be subject to additional 
scrutiny that no one else would be subject to. Please explain why should one 
person be singled out for a special "six-month waiting period" when the exact 
same possibility for conflict exists with anyone who is ever on the committee? 

> It is the opposite of sensible, to respond to this with 'how dare you"
> or by asserting that this could never happen or by saying that we
> shouldn't talk about that in case people get frightened.  I point you

Red herring. The objection (as many people have now voiced) is the double 
standard you proposed. 

> again to Linus' interview [1].  He is not upset that he has been
> insulted by the implication of conflict of interest, he soberly
> accepts that this will always be an issue, with companies in
> particular, and goes out of his way to address that in an explicit and
> reasonable way.

Your selective quotation is impressive. Also in that interview, Linus points 
out that his employment contract is probably "unique in the entire world". 
Which means in practical terms that the details of what he has does are fairly 
well irrelevant to any other situation. So what is the point in bringing it up, 
at all, except to try and diminish someone else by comparison? 

(I'm done.) 

Bryan 
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] facebook, twitter, and g+

[Jaime Fernández del Río]

+1 for twitter
+0 for the others

On Tue, Sep 22, 2015 at 8:14 PM, Bryan Van de Ven 
wrote:

>
> > On Sep 22, 2015, at 9:58 PM, Charles R Harris 
> wrote:
> >
> > Hi All,
> >
> > Just posting to elicit thoughts about scipy.org having a presence in
> social media for announcements.
>
> Of the ones listed in the subject, I would suggest Twitter is the most
> valuable. It has been great for release and other announcements. I find
> Facebook and G+ a little odd for software projects personally, though I
> guess they could be useful if you wanted to have somewhat longer "posts".
> But of course that's also yet more for someone to have maintain.
>
> One additional consideration is that some people will inevitably attempt
> to elicit technical support over these channels. I generally have to steer
> people to the mailing list immediately.
>
> Bryan
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación mundial.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

[Nathaniel Smith]

On Tue, Sep 22, 2015 at 3:43 PM, Charles R Harris
 wrote:
>
>
> On Mon, Sep 21, 2015 at 10:23 PM, Nathaniel Smith  wrote:
[...]
>> When it comes to evolving these APIs in general: one unfortunate thing
>> about the PyArrayObject changes in 1.7 is that because they were
>> implemented using *inline* functions (/macros) they haven't affected
>
>
> One thing we might consider along the way is separating numpy.multiarray and
> friends into an actual library plus a module. That way the new numpy api
> would be exposed in the library rather than by importing an array of
> pointers from the module.
>

I'm not sure whether we'll be able to pull this off at the technical
level? Partly because anything involving cross-platform linker
behavior is a recipe for unpleasantness, but mostly because doing
sliding-window API/ABI tracking requires that we have some way to
check which of multiple APIs a given third-party package is
requesting, and provide a nice error if the one they want isn't
available, and I'm not certain how to accomplish that via the regular
linker. But sure, something to look into when we reach that point :-)

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

[Charles R Harris]

On Tue, Sep 22, 2015 at 10:19 PM, Nathaniel Smith  wrote:

> On Tue, Sep 22, 2015 at 3:43 PM, Charles R Harris
>  wrote:
> >
> >
> > On Mon, Sep 21, 2015 at 10:23 PM, Nathaniel Smith  wrote:
> [...]
> >> When it comes to evolving these APIs in general: one unfortunate thing
> >> about the PyArrayObject changes in 1.7 is that because they were
> >> implemented using *inline* functions (/macros) they haven't affected
> >
> >
> > One thing we might consider along the way is separating numpy.multiarray
> and
> > friends into an actual library plus a module. That way the new numpy api
> > would be exposed in the library rather than by importing an array of
> > pointers from the module.
> >
>
> I'm not sure whether we'll be able to pull this off at the technical
> level? Partly because anything involving cross-platform linker
> behavior is a recipe for unpleasantness, but mostly because doing
> sliding-window API/ABI tracking requires that we have some way to
> check which of multiple APIs a given third-party package is
> requesting, and provide a nice error if the one they want isn't
> available, and I'm not certain how to accomplish that via the regular
> linker. But sure, something to look into when we reach that point :-)
>

I'd recommend the Henry Ford approach,  "Any customer can have a car
painted any color that he wants so long as it is *black*".  Essentially, an
ABI break split between a backward compatible layer on top, and a bare
metal layer below, with the latter recommended. We would still need to
solve the 'hide the structure" problem, but that is probably unavoidable
whatever approach we take. In any case, it might be worthwhile making a
list of the functions such a library would expose. I'm not sure how big a
problem linking would be, likely Windows would continue to be the largest
source of problems if we go the shared library route.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[josef.pktd]

On Tue, Sep 22, 2015 at 10:55 PM, Bryan Van de Ven 
wrote:

>
> > On Sep 22, 2015, at 1:48 PM, Matthew Brett 
> wrote:
> >
> > The point is, that a sensible organization and a sensible leader has
> > to take the possibility of conflict of interest into account.  They
> > also have to consider the perception of a conflict of interest.
>
> Of course, and the policies to deal with conflicts have deal with the
> possibility that *anyone* at all might have conflict. But that was not your
> suggestion. Your suggestion was to make one individual be subject to
> additional scrutiny that no one else would be subject to. Please explain
> why should one person be singled out for a special "six-month waiting
> period" when the exact same possibility for conflict exists with anyone who
> is ever on the committee?
>


I don't quite understand where the discussion went. The question was not
whether Travis is singled out but whether he is "singled in".

>From my perspectives (7 to 8 years) the situation has changed a lot. Most
of the discussion and consensus building happens on github issues, pull
requests and mailing lists. Merge policy has changed a lot since the svn
days.

Based on all the comments, Travis doesn't have time for this. And I think
the final (last resort) decisions about code should be left to the active
developers that know and participate in the actual work.

If Travis is treated as developer but doesn't have a special status, then
there is also no reason to "single out" him and Continuum for possibly too
much influence.

This is already the current status quo as it developed over the last
several years, AFAICS.

In my view, in a narrow sense, Travis is a "hit and run" contributor. good
ideas and providing good contributions, but somebody has to integrate it,
maintain it and fit it into the development plan.
(Given my experience I would compare it more with GSOC contributions that
need the "core developers" to provide the continuity in the development to
absorb the good work.)
Travis has too many other obligations and interests to provide this day to
day continuity.

Travis is still very important for providing ideas, pushing projects
forward and as one of the community leaders, but I would leave the final
decisions for the development of numpy to the developers in the trenches.

I pretty much agree completely with Nathanial, and Sebastian, (except that
I don't know much about any other FOSS communities)

And to repeat my point from another thread on this: I'm very skeptical
about any committee or board that is based on "outside members" and that is
involved in the decision about code development.

Josef




>
> > It is the opposite of sensible, to respond to this with 'how dare you"
> > or by asserting that this could never happen or by saying that we
> > shouldn't talk about that in case people get frightened.  I point you
>
> Red herring. The objection (as many people have now voiced) is the double
> standard you proposed.
>
> > again to Linus' interview [1].  He is not upset that he has been
> > insulted by the implication of conflict of interest, he soberly
> > accepts that this will always be an issue, with companies in
> > particular, and goes out of his way to address that in an explicit and
> > reasonable way.
>
> Your selective quotation is impressive. Also in that interview, Linus
> points out that his employment contract is probably "unique in the entire
> world". Which means in practical terms that the details of what he has does
> are fairly well irrelevant to any other situation. So what is the point in
> bringing it up, at all, except to try and diminish someone else by
> comparison?
>
> (I'm done.)
>
> Bryan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] facebook, twitter, and g+

[Charles R Harris]

Hi All,

Just posting to elicit thoughts about scipy.org having a presence in social
media for announcements.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: Numpy 1.10.0rc1 released.

[Nathaniel Smith]

On Tue, Sep 22, 2015 at 8:12 PM, Charles R Harris
 wrote:
> Hi all,
>
> I'm pleased to announce the availability of Numpy 1.10.0rc1. Sources and 32
> bit binary packages for Windows may be found at Sourceforge.  Please test
> this out, as I would like to move to the final release as rapidly as
> possible and the lack of error reports from the beta has left me nervous.
> It's been quiet, too quiet. In the movies, we would all die in the next five
> minutes.

The release was coming from INSIDE THE HOUSE!

Thanks Chuck!

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

On Tue, Sep 22, 2015 at 1:20 PM, Stefan van der Walt 
wrote:

>
>
> I guess we've gone off the rails pretty far at this point, so let me at
> least take a step back, and make sure that you know that:
>
> - I have never doubted that your intensions for NumPy are anything but
>   good (I know they are!),
> - I *want* the community to be a welcoming place for companies to
>   contribute (otherwise, I guess I'd not be such a fervent supporter of
>   the scientific eco-system using the BSD license), and
> - I love your enthusiasm for the project.  After all, that is a big part
>   of what inspired me to become involved in the first place.
>
> My goal is not to spread uncertainty, fear nor doubt—if that was the
> perception left, I apologize.
>
> I'll re-iterate that I wanted to highlight a concern about the
> interactions of a (somewhat weakly cohesive) community and strong,
> driven personalities such as yourself backed by a formidable amount of
> development power.  No matter how good your intensions are, there are
> risks involved in this kind of interaction, and if we fail to even
> *admit* that, we are in trouble.
>
> Lest the above be read in a negative light again, let me state it
> up-front: *I don't think you will hijack the project, use it for your
> own gain, or attempt to do anything you don't believe to be in the best
> interest of NumPy.* What I'm saying is that we absolutely need to move
> forward in a way that brings everyone along, and makes everyone rest
> assured that their voice will be heard.
>
>
Thank you for the clarification.   I'm sorry that I started to question
your intentions.I agree that everyone should rest assured that their
voice will be heard.   I have been and continue to be a staunch advocate
for the voices that are not even on this mailing list.


> Also, please know that I have not discussed these matters with Nathaniel
> behind the scenes, other than an informal hour-long discussion about his
> original governance proposal.  There is no BIDS conspiracy or attempts
> at crucifixion.  After all, you were an invited guest speaker at an
> event I organized this weekend, since I value your opinion and insights.
>
>
Thank you.   I'm sorry for implying otherwise.   That was wrong of me.   I
know we are just trying to bring all the voices to the table.

Best,

-Travis
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

I am not upset nor was I ever upset about discussing the possibility of
conflict of interest.   Of course it can be discussed --- but it should be
discussed directly about specific things --- and as others have said it is
generally easily handled when it actually could arise.   The key is to
understand affiliations.   We should not do things in the community that
actually encourage people to hide their affiliations for fear of backlash
or bias.

I was annoyed at the insinuation that conflict of interest is a
company-only problem that academics are somehow immune to.

I was upset about accusations and mis-interpretations of my activities and
those of my colleagues in behalf of the community.

On Tue, Sep 22, 2015 at 1:48 PM, Matthew Brett 
wrote:

> Hi,
>
> On Tue, Sep 22, 2015 at 11:20 AM, Stefan van der Walt
>  wrote:
> > Hi Travis
> >
> > On 2015-09-22 03:44:12, Travis Oliphant  wrote:
> >> I'm actually offended that so many at BIDS seem eager to crucify my
> >> intentions when I've done nothing but give away my time, my energy, my
> >> resources, and my sleep to NumPy for many, many years.I guess if
> your
> >> intent is to drive me away, then you are succeeding.
> >
> > I guess we've gone off the rails pretty far at this point, so let me at
> > least take a step back, and make sure that you know that:
> >
> > - I have never doubted that your intensions for NumPy are anything but
> >   good (I know they are!),
> > - I *want* the community to be a welcoming place for companies to
> >   contribute (otherwise, I guess I'd not be such a fervent supporter of
> >   the scientific eco-system using the BSD license), and
> > - I love your enthusiasm for the project.  After all, that is a big part
> >   of what inspired me to become involved in the first place.
> >
> > My goal is not to spread uncertainty, fear nor doubt—if that was the
> > perception left, I apologize.
> >
> > I'll re-iterate that I wanted to highlight a concern about the
> > interactions of a (somewhat weakly cohesive) community and strong,
> > driven personalities such as yourself backed by a formidable amount of
> > development power.  No matter how good your intensions are, there are
> > risks involved in this kind of interaction, and if we fail to even
> > *admit* that, we are in trouble.
> >
> > Lest the above be read in a negative light again, let me state it
> > up-front: *I don't think you will hijack the project, use it for your
> > own gain, or attempt to do anything you don't believe to be in the best
> > interest of NumPy.* What I'm saying is that we absolutely need to move
> > forward in a way that brings everyone along, and makes everyone rest
> > assured that their voice will be heard.
> >
> > Also, please know that I have not discussed these matters with Nathaniel
> > behind the scenes, other than an informal hour-long discussion about his
> > original governance proposal.  There is no BIDS conspiracy or attempts
> > at crucifixion.  After all, you were an invited guest speaker at an
> > event I organized this weekend, since I value your opinion and insights.
> >
> > Either way, let me again apologize if my suggested lack of insight hurt
> > people's feelings.  I can only hope that, in educating me, we all learn
> > a few lessons.
>
> I'm also in favor of taking a step back.
>
> The point is, that a sensible organization and a sensible leader has
> to take the possibility of conflict of interest into account.  They
> also have to consider the perception of a conflict of interest.
>
> It is the opposite of sensible, to respond to this with 'how dare you"
> or by asserting that this could never happen or by saying that we
> shouldn't talk about that in case people get frightened.  I point you
> again to Linus' interview [1].  He is not upset that he has been
> insulted by the implication of conflict of interest, he soberly
> accepts that this will always be an issue, with companies in
> particular, and goes out of his way to address that in an explicit and
> reasonable way.
>
> Cheers,
>
> Matthew
>
> [1] http://www.bbc.com/news/technology-18419231
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Tentative NumPy Tutorial inaccessible

[Andriy Yurchuk]

Hi!

The Tentative NumPy Tutorial is no longer accessible by the URL 
http://wiki.scipy.org/Tentative_NumPy_Tutorial, it returns a 403. The 
link to this page is still on NumPy homepage though. Has the page been 
moved somewhere else?


---
Regards,
Andriy Yurchuk
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Nathaniel Smith]

On Mon, Sep 21, 2015 at 10:15 PM, Bryan Van de Ven  wrote:
>
>> On Sep 21, 2015, at 9:42 PM, David Cournapeau  wrote:
>> There is ample history of such things happening in OSS history, so I think 
>> that's a fair concern, even if that has not happened for numpy yet.
>
> Specific examples to support that claim would be appreciated. In particular, 
> examples where an OSS project was corrupted (is that the word?) by a company 
> specifically at the hand of the project's original creator would be 
> especially relevant.

I have no expectation that continuum will follow any of these paths,
and in most cases am not even sure what that would mean, BUT just
because I think it is useful to have a wide variety of concrete
examples to draw on -- data is good! -- there actually are *lots* of
examples of "community revolts" wresting projects from their original
founders, in a variety of corporate and non-corporate contexts. Some
examples include the nodejs->iojs fork and merge (which was about
wresting control of the project from the founding company), the
gcc->egcs fork and merge (which removed RMS's control over day-to-day
running of the project), the openoffice->libreoffice fork, the
xfree86->x.org fork (where the original core team decided to change
the license and all the developers left), the mambo->joomla fork, the
xchat->hexchat fork (triggered partially by people's annoyance at the
original developer for trying to monetize the project), ... Along
somewhat similar lines, there's also the fraught history of Qt and
Trolltech and the conflicts between the community and commercial
interests there.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

On Tue, Sep 22, 2015 at 2:16 AM, Stefan van der Walt 
wrote:

> Hi Travis
>
> On 2015-09-21 23:29:12, Travis Oliphant  wrote:
> >   1) nobody believes that the community should be forced to adopt numba
> as
> > part of ufunc core yet --- but this could happen someday just as Cython
> is
> > now being adopted but was proposed 8 years ago that it "could be adopted"
> > That's a red-hearing.
>
> Yes, I'd like to clarify: I was not against including any specific
> technology in NumPy.  I was highlighting that there may be different
> motivations for members of the general community and those working for,
> say, Continuum, to get certain features adopted.
>

This is what I'm calling you out on.  Why?   I think that is an unfair
statement and inaccurate.   The general community includes Continuum,
Enthought, Microsoft, Intel, various hedge funds, investment banks and
companies large and small.   Are you saying that people should not be
upfront about their affiliations with a company?  That if they are not
academics, then they should not participate in the discussion?   It is hard
enough to be at a company and get time to contribute effort back to an open
source project.We should not be questioning people's motives just
*because* they are at a company.   We should not assume people cannot think
in terns of the success of the project, just because they are at a company.


Their proposals and contributions can be evaluated on their merits and
value --- so this whole discussion seems to be just revealing an
anti-company paranoia rather than helping understand the actual concern.


> >   2) I have stated that breaking the ABI is of little consequence because
> > of conda as well as other tools.I still believe that.  This has
> nothing
> > to do with any benefit Continuum might or might not receive because of
> > conda.   Everyone else who wants to make a conda-based distribution also
> > benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits.
> > I don't think the community realizes the damange that is done with FUD
> like
> > this.  There are real implications.  It halts progress, creates
> confusion,
> > and I think ultimately damages the community.
>
> This is an old argument, and the reason why we have extensive measures
> in place to guard against ABI breakage.  But, reading what you wrote
> above, I would like to understand better what FUD you are referring to,
> because I, rightly or wrongly, believe there is a real concern here that
> is being glossed over.
>

I don't know which is the "old argument".   Anyway, old arguments can still
be right.  The fact is that not breaking the ABI has caused real damage to
the community.  NumPy was never designed to not have it's ABI broken for
over a decade. We have some attempts to guard against ABI breakage ---
but they are not perfect.

We have not moved the code-base forward for fear of breaking the ABI.
When it was hard to update your Python installation that was a concern.
There are very few cases where this is still the concern (conda is a big
part of it but not the only part as other distros and approaches for easily
updating the install exist) --- having this drive major architecture
decisions is a serious mistake in my mind, and causes a lot more work than
it should.

The FUD I'm talking about is the anti-company FUD that has influenced
discussions in the past.I really hope that we can move past this.


>
> > I don't see how.None of these have been proposed for integrating into
> > NumPy.I don't see how integrating numba into NumPy benefits Continuum
> > at all.  It's much easier for us to keep it separate.   At this point
> > Continuum doesn't have an opinion about integrating DyND into NumPy or
> > not.
>
> I think that touches, tangentially at least, on the problem.  If an
> employee of Continuum were steering NumPy, and the company developed an
> opinion on those integrations, would such a person not feel compelled to
> toe the company line?  (Whether the company is Continuum or another is
> besides the point—I am only trying to understand the dynamics of working
> for a company and leading an open source project that closely interacts
> with their producs.)
>

O.K.  if you are honestly asking this question out of inexperience, then I
can at least help you understand because perhaps that is the problem
(creating a straw-man that doesn't exist).I have never seen a motivated
open source developer at a company who "tows the company line" within a
community project that is accepted long term.All that would do is drive
the developer out of the company and be a sure-fire way to make sure their
contributions are not accepted.   I know that at Continuum, for example, if
someone disagreed with me about an open source technology, didn't tell me
and just "towed the company line" (whatever they thought that was) --- they
would be fired.   Most software companies that participate in 

Re: [Numpy-discussion] Governance model request

[Bryan Van de Ven]

> I have no expectation that continuum will follow any of these paths,
> and in most cases am not even sure what that would mean, BUT just
> because I think it is useful to have a wide variety of concrete
> examples to draw on -- data is good! -- there actually are *lots* of
> examples of "community revolts" wresting projects from their original
> founders, in a variety of corporate and non-corporate contexts. Some
> examples include the nodejs->iojs fork and merge (which was about
> wresting control of the project from the founding company), the
> gcc->egcs fork and merge (which removed RMS's control over day-to-day
> running of the project), the openoffice->libreoffice fork, the
> xfree86->x.org fork (where the original core team decided to change
> the license and all the developers left), the mambo->joomla fork, the
> xchat->hexchat fork (triggered partially by people's annoyance at the
> original developer for trying to monetize the project), ... Along
> somewhat similar lines, there's also the fraught history of Qt and
> Trolltech and the conflicts between the community and commercial
> interests there.

All of there are exactly the opposite of what I asked about, and what was 
suggested as the threat: an original founder and corporate interest wresting 
control from the community. 

Bryan 
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Nathaniel Smith]

Hi Bryan,

I understand where you're coming from, but I'd appreciate it if we
could keep the discussion on a less visceral level? Nobody's personal
integrity is being impugned, but it's the nature of this kind of
governance discussion that we have to consider unlikely-and-unpleasant
hypotheticals. It's like when you talk to a lawyer about a contract or
a will or whatever: they'll make you think about all kinds of horrible
possibilities, not because any of them are likely, but because sooner
or later *something* will go wrong, and the point of having a
contract/will/governance document is to provide some framework to
handle whichever unlikely edge case does arise.

And the other purpose of this kind of framework is to avoid the
*perception* (whether justified or not) of these kinds of conflicts of
interest -- if not handled well then this can easily scare away
volunteers, contributions from other companies, etc. Obviously you
know Travis and Continuum well as an employee there, but to most
people without that personal connection, Continuum is just a large
corporate entity with unknown motives. Imagine if instead of Continuum
we were talking about it was Google or RandomAggressiveStartup -- some
company that you didn't have any personal connection or insight into.
For someone in this position, it's not unreasonable to want more of a
reassurance that things will work out then just being asked to trust
that the CEO is personally committed to not being evil and they can
trust him.

Also, in these messages you seem to be working from a framework where
people working in good faith will always agree, and so any suggestion
of a possible conflict of interest can only arise through bad faith.
But this isn't true. Is it really so difficult to imagine that, say,
Continuum and Enthought might at some point have conflicting needs for
numpy, or for Continuum's vision of the future of numpy could be
less-than-perfectly-representative with every bit of numpy's entire
giant userbase? Continuum is a company that has in the past submitted
rather obscure patches to numpy that AFAICT are used exclusively by a
particular contracting customer (e.g. [1]), and that is currently
investing a substantial multiple of numpy's development budget on
building a direct competitor to numpy.

To emphasize: I personally am not concerned by these facts -- we did
merge that patch I linked, and there's no animosity between the numpy
and dynd teams -- but reasonable people could reasonably be concerned
that tricky situations might emerge in the future, and I've talked to
plenty of people who are nervous about Continuum's influence in
general. And with my numpy developer hat on, I don't even care which
"side" is right, that's irrelevant to me, because my concern is with
providing a space where both "sides" feel comfortable working
together. This is why it's crucial that numpy-the-project be clearly
distinguishable as an independent entity that is beholden only to its
own community: it's *exactly because* we *value* the contribution of
companies like Continuum, and want to be able to freely foster those
relationships without creating suspicion and bad blood.

Also to emphasize: none of this means that Travis can't be on the
steering council -- I think that's a more complex issue that I'll
address separately. All I'm saying is that pretending that you aren't
going to reassure people by pretending this elephant isn't in the
room, or by taking a reasonable set of concerns and aggressively
turning them into a referendum on individual people's integrity.

Finally, can I point out... anyone who has some wariness around the
possible influence of financial interests on the community (whether
justified or not!) is in particular not going to be reassured if you
keep aggressively attempting to shut down any perceived criticism of
*your own employer*. I know that your paycheck is not dictating your
opinions, and probably the hypothetical people I'm talking about are
being totally unfair to you for even considering such a thing, but...
strictly as a matter of practical rhetoric, I don't think this is the
most effective way to accomplish your goals.

-n

[1] https://github.com/numpy/numpy/pull/359


On Mon, Sep 21, 2015 at 11:28 PM, Bryan Van de Ven  wrote:
>
>> I don't know how productive it is to dream up examples, but it's not
>
> Well, agreed, to be honest.
>
>> very hard to do.  Currently, e.g., the community is not ready to adopt
>> numba as part of the ufunc core.  But it's been stated by some that,
>
> Who are you speaking for? The entire community? Under what mandate?
>
>> with so many people running Conda, breaking the ABI is of little
>> consequence.  And then it wouldn't be much of a leap to think that numba
>> is an acceptable dependency.
>
> The current somewhat concrete proposal I am aware of involves funding 
> cleaning up dtypes. Is there another concrete, credible proposal to make 
> Numba a dependency of NumPy that you can 

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

[Antoine Pitrou]

On Mon, 21 Sep 2015 21:38:36 -0700
Nathaniel Smith  wrote:
> Hi Antoine,
> 
> On Mon, Sep 21, 2015 at 2:44 AM, Antoine Pitrou  wrote:
> >
> > Hi Nathaniel,
> >
> > On Sun, 20 Sep 2015 21:13:30 -0700
> > Nathaniel Smith  wrote:
> >> Given this, I propose that for 1.11 we:
> >> 1) go ahead and hide/disable the problematic parts of the ABI/API,
> >> 2) coordinate with the known affected projects to minimize disruption
> >> to their users (which is made easier since they are all projects that
> >> are almost exclusively distributed via conda, which enforces strict
> >> NumPy ABI versioning),
> >> 3) publicize these changes widely so as to give any private code that
> >> might be affected a chance to speak up or adapt, and
> >> 4) leave the "ABI version tag" as it is, so as not to force rebuilds
> >> of the vast majority of projects that will be unaffected by these
> >> changes.
> >
> > Thanks for a detailed and clear explanation of the proposed changes.
> > As far as Numba is concerned, making changes is ok for us provided
> > Numpy provides APIs to do what we want.
> 
> Good to hear, thanks!
> 
> Any interest in designing those new APIs that will do what you want?
> :-)

I'll take a look.

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

I actually do agree with your view of the steering council as being usually
not really being needed.You are creating a straw-man by indicating
otherwise.I don't believe a small council should do anything *except*
resolve disputes that cannot be resolved without one.  Like you, I would
expect that would almost never happen --- but I would argue that
extrapolating from Debian's experience is not actually relevant here.

So, if the steering council is not really needed then why have it at all?
Let's just eliminate the concept entirely.

But there are real questions that have to have an answer or an approach to
making a decision.  The answer to these questions cannot really be a vague
notion of "lack of vigorous opposition by people who read the mailing list"
which then gets parried about as "the community decided this."   The NumPy
user base is far, far larger than the number of people that read this list.


For better or for worse, we will always be subject to the "tyranny of who
has time to contribute lately".Fundamentally, I would argue that this
kind of "tyranny" should at least be tempered by additional considerations
from long-time contributors who may also be acting more indirectly than is
measured by a simple git log.

So, what are the questions that have to have an answer that even calls for
some kind of governance?   I know Nathaniel's document listed some of them
(and perhaps all of them).   Here are mine:

1) who gets commit rights to the repo (and who has them removed)?
2) who tags the release of NumPy?
3) how is it decided where money is spent if it's donated to the project
(and not just to people directly)?
4) how is it decided if someone needs to be removed from participation in
the group because they are not adding to the conversation (we have been
fortunate that this hasn't happened in NumPy before --- but it could)?
5) how is it decided what goes on the NumPy website --- i.e. will
advertisers be able to put their logos or book-covers there?

If I understand what you are proposing, then basically the "steering
council" decides these things.Perhaps rather than a steering council,
though, we just need clear answers to questions like the above --- which
might be handled differently for different questions.I don't think
these questions have very easy answers.

Ultimately NumPy has relied on and continues to rely on the mutual respect
of all the people that have worked on the code and tried to make it better.
We all have opinions about how things have gone in the past, and what
has gone well and what hasn't.   But, nothing you have said persuades me
that you have a full picture of past history with respect to a lot of the
difficult kinds of conversations that have happened and the different modes
of activity that have tried to help move NumPy along.In fact, I think
you mis-understand and mis-interpret that history quite often.

I'm convinced you are well-intentioned and doing the very best you can, and
I'm very grateful that you are passionate and eager about moving NumPy
forward.   Ultimately I hope it will help things.

Here is my attempt at a proposal for how to answer the above questions:

1) who gets commit rights to the repo (and who has them removed)?

  * people who contribute regularly are granted commit rights by another
committer with at least two additional nominations and the lack of a veto
within 1 week of the proposal.
  * nobody has commit rights removed except by unanimous consent of all the
other committers (with consent being implied if not responded to within 2
weeks).

2) who tags the release of NumPy?

   * whomever volunteers to be release manager and if there is no veto from
committers.

3) how is it decided where money is spent if it's donated to the project
(and not just to people directly)?

   * three people who self-select represent NumPy to Numfocus following the
rules of Numfocus (that there is only one representative from any
organization).
   * If 3 committers oppose one of those people and nominate another in
place, then that person is replaced.

4) how is it decided if someone needs to be removed from participation in
the group because they are not adding to the conversation (we have been
fortunate that this hasn't happened in NumPy before --- but it could)?

* unanimous consent of all committers (with a 2 week period given for
consent to be given --- and it is assumed given if they are not heard
from).

5) how is it decided what goes on the NumPy website --- i.e. will
advertisers be able to put their logos or book-covers there?

* only Numfocus can advertise and put their logo on the website


Now, I'm sure one can poke holes in the above --- and I would welcome
better answers to the above questions.Perhaps we should just decide how
specific decisions get made and make a document that lists that and only
talks about committers instead of inventing another "bit" to differentiate
people in the community.

-Travis

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

>
>
>
> > May? Can you elaborate? More speculation. My own position is that
> > these projects want to integrate with NumPy, not the
> > converse. Regardless of my opinion, can you actually make any specific
> > arguements, one way or the otehr? What if if some integrations
> > actually make more sense for the community? Is this simply a dogmatic
> > ideological position that anything whatsoever that benefits both NumPy
> > and Continuum simultaneously is bad, on principle? That's fine, as
> > such, but let's make that position explicit if that's all it is.
>
> No, I don't have such a dogmatic ideological position.  I think,
> however, that it is somewhat unimaginative to propose that there are no
> potential conflicts whatsoever.
>
> I am happy if we can find solutions that benefit both numpy and any
> company out there.  But in the end, I'm sure you'd agree that we want
> the decisions that lead to such solutions to be taken in the best
> interest of the project, and not be weighed by alterior motivations of
> any sorts.  In the end, even the *perception* that that is not the case
> can be very harmful.
>

I will only comment on the last point.   I completely agree that the
*perception* that this is not the case can be harmful.

But, what concerns me is where this perception comes from --- from actual
evidence of anything that is not in the best interests of the project ---
or just ideological differences of opinion about the way the world works
and the perceptions around open source and markets.   It is quite easy for
someone to spread FUD about companies that contribute to open source ---
and it has the effect of discouraging companies from continuing to
contribute to community projects.This removes a huge amount of
potential support from projects.

In NumPy's case in particular, this kind of attitude basically guarantees
that I won't be able to contribute effectively and potentially even people
I fund to contribute might not be accepted --- not because we can't
faithfully participate in the same spirit that we have always contributed
to SciPy and NumPy and other open source projects --- but because people
are basically going to question things just because.

What exactly do you need me to say to get you to believe that I have
nothing but the best interests of array computing in Python at heart?

The only thing that is different between me today and me 18 years ago is
that 1) I have more resources now, 2) I have more knowledge about computer
science and software architecture and 3) I have more experience with how
NumPy gets used.All I can do is continue to try and make things better
the best way I know how.

-Travis


-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.10.0rc1 coming tomorrow, 22 Sept.

[Nathaniel Smith]

On Sep 21, 2015 11:51 PM, "Travis Oliphant"  wrote:
>
> Of course it will be 1.10.0 final where all the problems will show up
suddenly :-)
>
> Perhaps we can get to where we are testing Anaconda against beta releases
better.

The most useful thing would actually not even involve you doing any more
testing, but just if you could make builds available so that end-users
could easily conda install the prereleases and do their own testing against
their own choice. In principle I guess we could provide our own binstar
channel for this, but it's difficult given that AFAIK rebuilding numpy in
conda requires also rebuilding the whole stack, and the numpy build recipes
are still proprietary.

-n
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Nathaniel Smith]

On Tue, Sep 22, 2015 at 1:24 AM, Travis Oliphant 
wrote:

> I actually do agree with your view of the steering council as being
> usually not really being needed.You are creating a straw-man by
> indicating otherwise.I don't believe a small council should do anything
> *except* resolve disputes that cannot be resolved without one.  Like you, I
> would expect that would almost never happen --- but I would argue that
> extrapolating from Debian's experience is not actually relevant here.
>

To be clear, Debian was only one example -- what I'm extrapolating from is
every community-driven F/OSS project that I'm aware of.

It's entirely possible my data set is incomplete -- if you have some other
examples that you think would be better to extrapolate from, then I'd be
genuinely glad to hear them. You may have noticed that I'm a bit of an
enthusiast on this topic :-).


>
>
So, if the steering council is not really needed then why have it at all?
> Let's just eliminate the concept entirely.
>
>
In my view, the reasons for having such a council are:
1) The framework is useful even if you never use it, because it means
people can run "what if" scenarios in their mind and make decisions on that
basis. In the US legal system, only a vanishingly small fraction of cases
go to the Supreme Court -- but the rules governing the Supreme Court have a
huge effect on all cases, because people can reason about what would happen
*if* they tried to appeal to the Supreme Court.
2) It provides a formal structure for interfacing with the outside world.
E.g., one can't do anything with money or corporate contributions without
having some kind of written-down and enforceable rules for making decisions
(even if in practice you always stick to the "everyone is equal and we
govern by consensus" part of the rules).
3) There are rare but important cases where discussions have to be had in
private. The main one is "personnel decisions" like inviting people to join
the council; another example Fernando has mentioned to me is that when they
need to coordinate a press release between the project and a funding body,
the steering council reviews the press release before it goes public.

That's pretty much it, IMO.

The framework we all worked out at the dev meeting in Austin seems to
handle these cases well AFAICT.


> But there are real questions that have to have an answer or an approach to
> making a decision.  The answer to these questions cannot really be a vague
> notion of "lack of vigorous opposition by people who read the mailing list"
> which then gets parried about as "the community decided this."   The NumPy
> user base is far, far larger than the number of people that read this list.
>

According to the dev meeting rules, no particularly "vigorous opposition"
is required -- anyone who notices that something bad is happening can write
a single email and stop an idea dead in its tracks, with only the steering
council able to overrule. We expect this will rarely if ever happen,
because the threat will be enough to keep everyone honest and listening,
but about the only way we could possibly be *more* democratic is if we
started phoning up random users at home to ask their opinion.

This is actually explicitly designed to prevent the situation where whoever
talks the loudest and longest wins, and to put those with more and less
time available on an equal footing.


> For better or for worse, we will always be subject to the "tyranny of who
> has time to contribute lately".Fundamentally, I would argue that this
> kind of "tyranny" should at least be tempered by additional considerations
> from long-time contributors who may also be acting more indirectly than is
> measured by a simple git log.
>

I guess I am missing something fundamental here. Who are these long-time
contributors who will sit on your council of 1/3/5 but who don't even read
the mailing list? How will they know when their special insight is
necessary?


> So, what are the questions that have to have an answer that even calls for
> some kind of governance?   I know Nathaniel's document listed some of them
> (and perhaps all of them).   Here are mine:
>
> 1) who gets commit rights to the repo (and who has them removed)?
> 2) who tags the release of NumPy?
> 3) how is it decided where money is spent if it's donated to the project
> (and not just to people directly)?
> 4) how is it decided if someone needs to be removed from participation in
> the group because they are not adding to the conversation (we have been
> fortunate that this hasn't happened in NumPy before --- but it could)?
> 5) how is it decided what goes on the NumPy website --- i.e. will
> advertisers be able to put their logos or book-covers there?
>
> If I understand what you are proposing, then basically the "steering
> council" decides these things.
>

No, absolutely not. The proposal is that these issues are decided by open
discussion on the mailing list. (With the 

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

On Tue, Sep 22, 2015 at 1:07 AM, Stefan van der Walt 
wrote:

> On 2015-09-21 22:15:55, Bryan Van de Ven  wrote:
> > Beyond that, what (even in a broad sense) is an example of a goal that
> > "Continuum might need" that would conceivably do detriment to the
> > NumPy community? That it be faster? Simpler to maintain? Easier to
> > extend? Integrate better with more OS projects? Attract new active
> > developers? Receive more financial support? Grow its user base even
> > more?
>
> I don't know how productive it is to dream up examples, but it's not
> very hard to do.  Currently, e.g., the community is not ready to adopt
> numba as part of the ufunc core.  But it's been stated by some that,
> with so many people running Conda, breaking the ABI is of little
> consequence.  And then it wouldn't be much of a leap to think that numba
> is an acceptable dependency.
>

A couple of things to help clarify:

  1) nobody believes that the community should be forced to adopt numba as
part of ufunc core yet --- but this could happen someday just as Cython is
now being adopted but was proposed 8 years ago that it "could be adopted"
That's a red-hearing.

  2) I have stated that breaking the ABI is of little consequence because
of conda as well as other tools.I still believe that.  This has nothing
to do with any benefit Continuum might or might not receive because of
conda.   Everyone else who wants to make a conda-based distribution also
benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits.
I don't think the community realizes the damange that is done with FUD like
this.  There are real implications.  It halts progress, creates confusion,
and I think ultimately damages the community.

Numba being an acceptable dependency means a lot more than conda --- it's
dependent on LLVM compiled support which would have to be carefully tested
--- first as only an optional dependency for many years.


>
> There's a broad range of Continuum projects that intersect with what
> NumPy does: numba, DyND, dask and Odo to name a few.  Integrating them
> into NumPy may make a lot more sense for someone from Continuum than for
> other members of the community.
>

I don't see how.None of these have been proposed for integrating into
NumPy.I don't see how integrating numba into NumPy benefits Continuum
at all.  It's much easier for us to keep it separate.   At this point
Continuum doesn't have an opinion about integrating DyND into NumPy or not.


These projects will all succeed or fail on their own based on users needs.
  Whether or not they every become a part of NumPy will depend on whether
they are useful as such not because a person at Continuum is part of a
steering committee (with other people on it).

I know that you were responding to specific question by Brian as to how
their could be a conflict of interest for Continuum and NumPy development.
I don't think this is a useful conversation --- we could dream up all
kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS
really wants Spark to take over and for NumPy to have special connections
to Spark).   Are we to not allow anyone at BIDS to participate in the
steering council because of their other interests?

But remember, the original point is whether or not someone from Continuum
(or I presume any company and not just singling out Continuum for special
treatment) should be on the steering council.Are you really arguing
that they shouldn't because there are other projects Continuum is working
on that have some overlap with NumPy.I really hope you don't actually
believe that.

-Travis




> Stéfan
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Bryan Van de Ven]

> I don't know how productive it is to dream up examples, but it's not

Well, agreed, to be honest. 

> very hard to do.  Currently, e.g., the community is not ready to adopt
> numba as part of the ufunc core.  But it's been stated by some that,

Who are you speaking for? The entire community? Under what mandate?

> with so many people running Conda, breaking the ABI is of little
> consequence.  And then it wouldn't be much of a leap to think that numba
> is an acceptable dependency.

The current somewhat concrete proposal I am aware of involves funding cleaning 
up dtypes. Is there another concrete, credible proposal to make Numba a 
dependency of NumPy that you can refer to? If not, why are we mired in 
hypotheticals?

> There's a broad range of Continuum projects that intersect with what
> NumPy does: numba, DyND, dask and Odo to name a few.  Integrating them
> into NumPy may make a lot more sense for someone from Continuum than for
> other members of the community.

May? Can you elaborate? More speculation. My own position is that these 
projects want to integrate with NumPy, not the converse. Regardless of my 
opinion, can you actually make any specific arguements, one way or the otehr? 
What if if some integrations actually make more sense for the community? Is 
this simply a dogmatic ideological position that anything whatsoever that 
benefits both NumPy and Continuum simultaneously is bad, on principle? That's 
fine, as such, but let's make that position explicit if that's all it is. 

Bryan 
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Stephan Hoyer]

On Tue, Sep 22, 2015 at 2:33 AM, Travis Oliphant 
wrote:

> The FUD I'm talking about is the anti-company FUD that has influenced
> discussions in the past.I really hope that we can move past this.
>

I have mostly stayed out of the governance discussion, in deference to how
new I am in this community, but I do want to take a moment to speak up here
to echo Travis's concern about anti-company FUD.

Everyone invested in NumPy has their own projects, priorities and employers
which shape their agenda. As far as I can tell, Travis and Continuum have
only ever acted with the long term health of the scipy ecosystem in mind.

Stephan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.10.0rc1 coming tomorrow, 22 Sept.

[Travis Oliphant]

Absolutely it would be good if others can test.  All I was suggesting is
that we do run a pretty decent set of tests upon build and that would be
helpful.

If the numpy build recipes are not available, it is only because they have
not been updated to use conda-build yet.  If somebody wants to volunteer to
convert all of our internal recipes to conda-build recipes so they could be
open source --- we would welcome the help.

But, it's not just the numpy recipes, it's the downstream binaries and
their test-suite as well that is useful to run.   I am hoping we will have
something automatic here in the next few months on anaconda.org that will
make this easier -- but no promises at this point.

-Travis


On Tue, Sep 22, 2015 at 2:19 AM, Nathaniel Smith  wrote:

> On Sep 21, 2015 11:51 PM, "Travis Oliphant"  wrote:
> >
> > Of course it will be 1.10.0 final where all the problems will show up
> suddenly :-)
> >
> > Perhaps we can get to where we are testing Anaconda against beta
> releases better.
>
> The most useful thing would actually not even involve you doing any more
> testing, but just if you could make builds available so that end-users
> could easily conda install the prereleases and do their own testing against
> their own choice. In principle I guess we could provide our own binstar
> channel for this, but it's difficult given that AFAIK rebuilding numpy in
> conda requires also rebuilding the whole stack, and the numpy build recipes
> are still proprietary.
>
> -n
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

On Tue, Sep 22, 2015 at 2:33 AM, Nathaniel Smith  wrote:

> Hi Bryan,
>
> I understand where you're coming from, but I'd appreciate it if we
> could keep the discussion on a less visceral level? Nobody's personal
> integrity is being impugned, but it's the nature of this kind of
> governance discussion that we have to consider unlikely-and-unpleasant
> hypotheticals. It's like when you talk to a lawyer about a contract or
> a will or whatever: they'll make you think about all kinds of horrible
> possibilities, not because any of them are likely, but because sooner
> or later *something* will go wrong, and the point of having a
> contract/will/governance document is to provide some framework to
> handle whichever unlikely edge case does arise.
>
> And the other purpose of this kind of framework is to avoid the
> *perception* (whether justified or not) of these kinds of conflicts of
> interest -- if not handled well then this can easily scare away
> volunteers, contributions from other companies, etc. Obviously you
> know Travis and Continuum well as an employee there, but to most
> people without that personal connection, Continuum is just a large
> corporate entity with unknown motives. Imagine if instead of Continuum
> we were talking about it was Google or RandomAggressiveStartup -- some
> company that you didn't have any personal connection or insight into.
> For someone in this position, it's not unreasonable to want more of a
> reassurance that things will work out then just being asked to trust
> that the CEO is personally committed to not being evil and they can
> trust him.
>

Anybody who comes to NumPy should know that I wrote it and then gave it to
the community -- creating an independent foundation at the same time as I
created a Company to create a division between them so that my actions
could be understood.   I really don't know how to give more reassurance.

I'm actually offended that so many at BIDS seem eager to crucify my
intentions when I've done nothing but give away my time, my energy, my
resources, and my sleep to NumPy for many, many years.I guess if your
intent is to drive me away, then you are succeeding.


>
> Also, in these messages you seem to be working from a framework where
> people working in good faith will always agree, and so any suggestion
> of a possible conflict of interest can only arise through bad faith.
> But this isn't true. Is it really so difficult to imagine that, say,
> Continuum and Enthought might at some point have conflicting needs for
> numpy, or for Continuum's vision of the future of numpy could be
> less-than-perfectly-representative with every bit of numpy's entire
> giant userbase?


Of course not, but this is no different than anyone else and a company
should not be singled out.   All you are doing is forcing any contribution
to be either only from a non-profit or have individuals hide their actual
allegiances.


> Continuum is a company that has in the past submitted
> rather obscure patches to numpy that AFAICT are used exclusively by a
> particular contracting customer (e.g. [1]), and that is currently
> investing a substantial multiple of numpy's development budget on
> building a direct competitor to numpy.
>

Good grief!   These comments are so much bunk that I have to call you on it
emphatically.  You claim below that you are unconcerned but yet you
insinuate some crazy, ulterior motivations for Continuum actually helping
people upgrade their NumPy installation that broke their old code because
of changes to NumPy.This kind of comment really upsets me.   You
dismiss real things and real problems that happen and brush it away because
it's *commercial*.

That patch you reference was actually an attempt to fix a problem that the
community pushed on the world --- therefore breaking people's code (but
good thing the ABI didn't change...).   We were up front about it and
worked with the community to get a change into NumPy to accommodate a user
of the tool.  It was hard work to figure out how to do that.   To have you
use that as some kind of argument against Continuum is not only unfair, it
is exactly the kind of mis-characterization and mis-interpretation of
events that I refer to in other posts.

And to say that we are investing a substantial multiple of Numpy's
development budget in a competitor is also incorrect.Continuum invests
in competent people who want to do interesting things.  We don't have a
rule that says things are "off-limits" including NumPy.   If competent
people feel like an alternative to NumPy is a good idea, then a certain
amount of exploration in that direction is allowed. DyND does not have
to be a competitor to NumPy.   It might compete with *your* vision of
NumPy, but it doesn't have to compete with NumPy.


>
> To emphasize: I personally am not concerned by these facts -- we did
> merge that patch I linked, and there's no animosity between the numpy
> and dynd teams -- but reasonable 

Re: [Numpy-discussion] Governance model request

[Sebastian Berg]

On Di, 2015-09-22 at 05:44 -0500, Travis Oliphant wrote:
> 
> 
> On Tue, Sep 22, 2015 at 2:33 AM, Nathaniel Smith 
> wrote:
> Hi Bryan,
> 
> I understand where you're coming from, but I'd appreciate it
> if we
> could keep the discussion on a less visceral level? Nobody's
> personal
> integrity is being impugned, but it's the nature of this kind
> of
> governance discussion that we have to consider
> unlikely-and-unpleasant
> hypotheticals. It's like when you talk to a lawyer about a
> contract or
> a will or whatever: they'll make you think about all kinds of
> horrible
> possibilities, not because any of them are likely, but because
> sooner
> or later *something* will go wrong, and the point of having a
> contract/will/governance document is to provide some framework
> to
> handle whichever unlikely edge case does arise.
> 
> And the other purpose of this kind of framework is to avoid
> the
> *perception* (whether justified or not) of these kinds of
> conflicts of
> interest -- if not handled well then this can easily scare
> away
> volunteers, contributions from other companies, etc. Obviously
> you
> know Travis and Continuum well as an employee there, but to
> most
> people without that personal connection, Continuum is just a
> large
> corporate entity with unknown motives. Imagine if instead of
> Continuum
> we were talking about it was Google or RandomAggressiveStartup
> -- some
> company that you didn't have any personal connection or
> insight into.
> For someone in this position, it's not unreasonable to want
> more of a
> reassurance that things will work out then just being asked to
> trust
> that the CEO is personally committed to not being evil and
> they can
> trust him.
> 
> 
> Anybody who comes to NumPy should know that I wrote it and then gave
> it to the community -- creating an independent foundation at the same
> time as I created a Company to create a division between them so that
> my actions could be understood.   I really don't know how to give more
> reassurance.
> 
> 
> I'm actually offended that so many at BIDS seem eager to crucify my
> intentions when I've done nothing but give away my time, my energy, my
> resources, and my sleep to NumPy for many, many years.I guess if
> your intent is to drive me away, then you are succeeding.   
>  
> 


Frankly, I am bit surprised at how this is developing. Why?! Nobody,
except being silly, really would doubt your intentions. That said, I
will be honest with you. I do not feel I know you well enough (or the
mode you operate) to for example accept you as BDFL of numpy [1] and I
frankly do not think that -- just like anyone else -- you should have
any special place in the governance document (which obviously does not
mean you should be blocked from being on the steering council) [2].

I just think there should probably not be any specific name in the NumPy
governance document at all.

The thing is, NOBODY really seems suggests that [3]. I have dislike
giving any special "power" to ANYONE. Frankly, I think if some people
are nervous (not so much those active in the discussion), it is probably
because they perceive that you have some direct power over numpy. As you
have said, this is just not true. But *because* of the lack of a
governance document, I would not be surprised if it *appears* to many
like you could wield a lot of power if you so wished. Simply because few
people really know how things are decided currently. And I think this
wrong perception is what makes some nervous.
If you say "maybe we should stop worrying about ABI" it may sound like
"two years from now we will definitely break ABI", the curse being that
it does not matter that you and those who know numpy's well know that it
is just you stressing strongly that we should seriously discuss it. I
have to admit, you sometimes sound a bit too definite in your
suggestions given your former position ;).

I hope I have not been rude here,

Sebastian


[1] I am sure you were *exactly* the right person to start numpy and be
the de-facto BDFL then, but today this is not on the table anyway.

[2] Also, lets be honest, you probably do have quite a bit of soft
influence, just by knowing the community, being at the conferences,
having NumFOCUS close by, etc.

[3] I guess you have in some sense, but I now understand that as a
suggestion to approach a different problem. And we can find another
solution for that problem!

> Also, in these messages you seem to be working from a
> framework where
> people working in good faith will always agree, and so any
> suggestion
> 

Re: [Numpy-discussion] Governance model request

[Travis Oliphant]

On Tue, Sep 22, 2015 at 4:33 AM, Nathaniel Smith  wrote:

> On Tue, Sep 22, 2015 at 1:24 AM, Travis Oliphant 
> wrote:
>
>> I actually do agree with your view of the steering council as being
>> usually not really being needed.You are creating a straw-man by
>> indicating otherwise.I don't believe a small council should do anything
>> *except* resolve disputes that cannot be resolved without one.  Like you, I
>> would expect that would almost never happen --- but I would argue that
>> extrapolating from Debian's experience is not actually relevant here.
>>
>
> To be clear, Debian was only one example -- what I'm extrapolating from is
> every community-driven F/OSS project that I'm aware of.
>
> It's entirely possible my data set is incomplete -- if you have some other
> examples that you think would be better to extrapolate from, then I'd be
> genuinely glad to hear them. You may have noticed that I'm a bit of an
> enthusiast on this topic :-).
>
>

Yes, you are much better at that than I am.   I'm not even sure where I
would look for this kind of data.


>
>>
> So, if the steering council is not really needed then why have it at all?
>> Let's just eliminate the concept entirely.
>>
>>
> In my view, the reasons for having such a council are:
> 1) The framework is useful even if you never use it, because it means
> people can run "what if" scenarios in their mind and make decisions on that
> basis. In the US legal system, only a vanishingly small fraction of cases
> go to the Supreme Court -- but the rules governing the Supreme Court have a
> huge effect on all cases, because people can reason about what would happen
> *if* they tried to appeal to the Supreme Court.
>

O.K.  That is a good point.   I can see the value in that.



> 2) It provides a formal structure for interfacing with the outside world.
> E.g., one can't do anything with money or corporate contributions without
> having some kind of written-down and enforceable rules for making decisions
> (even if in practice you always stick to the "everyone is equal and we
> govern by consensus" part of the rules).
>

O.K.


> 3) There are rare but important cases where discussions have to be had in
> private. The main one is "personnel decisions" like inviting people to join
> the council; another example Fernando has mentioned to me is that when they
> need to coordinate a press release between the project and a funding body,
> the steering council reviews the press release before it goes public.
>

O.K.



> That's pretty much it, IMO.
>
> The framework we all worked out at the dev meeting in Austin seems to
> handle these cases well AFAICT.
>

How did we "all" work it out when not everyone was there?   This is where I
get lost.   You talk about community decision making and yet any actual
decision is always a subset of the community.I suppose you just rely on
the "if nobody complains than it's o.k." rule?   That really only works if
the project is moving slowly.


> But there are real questions that have to have an answer or an approach to
>> making a decision.  The answer to these questions cannot really be a vague
>> notion of "lack of vigorous opposition by people who read the mailing list"
>> which then gets parried about as "the community decided this."   The NumPy
>> user base is far, far larger than the number of people that read this list.
>>
>
> According to the dev meeting rules, no particularly "vigorous opposition"
> is required -- anyone who notices that something bad is happening can write
> a single email and stop an idea dead in its tracks, with only the steering
> council able to overrule. We expect this will rarely if ever happen,
> because the threat will be enough to keep everyone honest and listening,
> but about the only way we could possibly be *more* democratic is if we
> started phoning up random users at home to ask their opinion.
>

O.K.  so how long is the time allowed for this kind of opposition to be
noted?



>
> This is actually explicitly designed to prevent the situation where
> whoever talks the loudest and longest wins, and to put those with more and
> less time available on an equal footing.
>
>
>> For better or for worse, we will always be subject to the "tyranny of who
>> has time to contribute lately".Fundamentally, I would argue that this
>> kind of "tyranny" should at least be tempered by additional considerations
>> from long-time contributors who may also be acting more indirectly than is
>> measured by a simple git log.
>>
>
> I guess I am missing something fundamental here. Who are these long-time
> contributors who will sit on your council of 1/3/5 but who don't even read
> the mailing list? How will they know when their special insight is
> necessary?
>

The long-time contributors wouldn't necessarily sit on that council.   But,
 I would support the idea of an advisory council that could act if it saw
things going the wrong direction.  This is where 

Re: [Numpy-discussion] Governance model request

[Sturla Molden]

On 20/09/15 20:20, Travis Oliphant wrote:


1 - define a BDFL for the council.  I would nominate chuck Harris

2 - limit the council to 3 people.  I would nominate chuck, nathaniel,
and pauli.

3 - add me as a permanent member of the steering council.



I have stayed out of this governance debate until now.

Personally I would prefer if you were BDFL.



Sturla

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Perry Greenfield]

I’ve also stayed out of this until now. I’m surprised and disheartened at the 
amount of suspicion and distrust directed towards Travis. I don’t think anyone 
has invested as much personal time and resources (e.g., money) towards 
supporting numpy, and not just in creating it but through efforts at Continuum 
and Numfocus.

So much of this distrust appears based on what Continuum might do rather than 
what the actual record is. I agree with Travis that virtually everyone has 
their own interests. I don’t think non-profit institutions (academia or 
otherwise) are any more virtuous than for-profit companies when it comes to 
possible conflicts of interest. As long as everyone understands the interests 
involved, that shouldn’t bar anyone from participating in governance.

If anyone deserves a special seat at the table it is Travis. I’m completely 
convinced he has the community’s greater interests at heart (not to say that he 
always understand all the interests and may need input to help in that).

Perry

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Governance model request

[Sturla Molden]

On 22/09/15 14:31, Perry Greenfield wrote:


I’ve also stayed out of this until now. I’m surprised and disheartened at the 
amount of suspicion and distrust directed towards Travis.


I have no idea where this distrust comes from. Nobody has invested so 
much of their time in NumPy. Without Travis there would not even be a NumPy.




So much of this distrust appears based on what Continuum might do rather than 
what the actual record is.


Being CEO of Continuum should not disqualify him in any way.



I agree with Travis that virtually everyone has their own interests.


Even Guido and Linus have employers. Should we distrust Guido as Python 
BDFL because some day Dropbox will be evil? It is just nonsense.



Sturla










___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] 1.10.0rc1 coming tomorrow, 22 Sept.

[Antoine Pitrou]

On Tue, 22 Sep 2015 04:43:18 -0500
Travis Oliphant  wrote:
> Absolutely it would be good if others can test.  All I was suggesting is
> that we do run a pretty decent set of tests upon build and that would be
> helpful.
> 
> If the numpy build recipes are not available, it is only because they have
> not been updated to use conda-build yet.  If somebody wants to volunteer to
> convert all of our internal recipes to conda-build recipes so they could be
> open source --- we would welcome the help.

Note there's a skeleton recipe for numpy here:
https://github.com/conda/conda-recipes/tree/master/numpy-openblas

If there's interest I could try to come up with a slightly better one,
although I can only promise to make it work on Linux.

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] draft NEP for breaking ufunc ABI in a controlled way

[Antoine Pitrou]

Hi,

This e-mail is an attempt at proposing an API to solve Numba's needs.

Attribute access


int PyUFunc_Nin(PyUFuncObject *)

  Replaces ufunc->nin.

int PyUFunc_Nout(PyUFuncObject *)

  Replaces ufunc->nout.

int PyUFunc_Nargs(PyUFuncObject *)

  Replaces ufunc->nargs.

PyObject *PyUFunc_Name(PyUFuncObject *)

  Replaces ufunc->name, returns a unicode object.
  (alternative: return a const char *)

For introspection, the following would be nice too:

int PyUFunc_Identity(PyFuncObject *)

  Replaces ufunc->identity.

const char *PyUFunc_Signature(PyUFuncObject *, int i)

  Gives a pointer to the types of the i'th signature.
  (equivalent today to >ntypes[i * ufunc->nargs])


Lifetime control


PyObject *PyUFunc_SetObject(PyUFuncObject *, PyObject *)

  Sets the ufunc's "object" to the given object.  The object has no
  special semantics except that it is DECREF'ed when the ufunc is
  deallocated (this is today's ufunc->obj).  The DECREF should happen
  only after the ufunc has accessed any internal resources (since the
  DECREF could deallocate some of those resources).

PyObject *PyUFunc_GetObject(PyUFuncObject *)

  Return the ufunc's current "object".


Loop registration
-

int PyUFunc_RegisterLoopForSignature(
PyUFuncObject* ufunc,
PyUFuncGenericFunction function, int *arg_types,
void *data, PyObject *obj)

  Register a loop implementation for the given arg_types (built-in
  types, presumably). This either appends the loop to the types and
  functions array (reallocating it if necessary), or replaces an
  existing one with the same signature.

  A copy of arg_types is done, such that the caller does not have to
  manage its lifetime. The optional "PyObject *obj" is an object which
  gets DECREF'ed when the loop is relinquished (for example when the
  ufunc is destroyed, or when the loop gets replaced with another by
  calling this function again).


I cannot say I'm 100% sure this is sufficient, but this seems it should
cover our current needs.

Note this is a minimal proposal. For example, Numpy could instead decide
to pass and return all argument types as PyArray_Descr pointers rather
than raw integers, and that would probably work for us too.

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion