Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Nathaniel Smith]

On Sat, Sep 5, 2015 at 8:08 PM, Matthew Brett  wrote:
> On Sat, Sep 5, 2015 at 7:14 PM, Nathaniel Smith  wrote:
>> On Sat, Sep 5, 2015 at 4:20 PM, Matthew Brett  
>> wrote:
>>> Hi,
>>>
>>> Thanks for replying, that is helpful.
>>>
>>> I'm going to write carefully, because I suspect we've only got this
>>> one shot at getting it right.
>>
>> Not sure why? "Change the governance document" is explicitly listed as
>> one of the steering council's powers, so I don't see what we'd lose by
>> adopting what we have now.
>
> In practice, I doubt that there will be any proposal to change the
> governance in the medium term, especially if the governance does turn
> out to be risk averse and conservative.

Whether there will be a proposal depends entirely on whether someone
(perhaps you) makes one :-).

Whether the proposal will be successful depends on many things, but
I'm guessing you'd have *more* luck getting traction in this
hypothetical future where you can point to how the project is
suffering from risk aversion and over-conservative decision making.
Right now everyone seems to be getting excited about getting stuff
done and trying out this proposed model, which seems like the worst
possible time to try and convince them that they're doomed to
stagnation.

[...]
>> No, by drama I mean situations where technical issues become personal,
>> and where people end up feeling emotionally hurt and disrespected and
>> so forth. I agree that vigorous debate is important, but strongly
>> disagree that veins bulging and violence are necessary or helpful to
>> success. (This is tangential to your main point, but I wanted to point
>> it out.) Since I happened to be reading Kate Heddleston's blog
>> yesterday, here's another relevant link...
>> 
>> https://www.kateheddleston.com/blog/argument-cultures-and-unregulated-aggression
>
> The distinction is between a culture that welcomes expressions of
> disagreement with one that does not.  In practice, if a group is
> easily alarmed by expressions of strong feeling, most people will be
> careful about expressing disagreement.   This is of course a rather
> subtle point.  It is easy to say that we do welcome disagreement, when
> in practice what happens is that the person disagreeing will in fact
> be labeled as being impolite.

I worry about this too. But this is a culture thing, not something
that seems amenable to quick fixes; it requires buy-in from everyone
individually. In particular I don't see what it has to do with whether
we have a named leader or not -- being named "project leader" doesn't
make someone's emails particularly more or less likely to change how
anyone in particular thinks about things.

[...]
>> Yep. And like you said above re: IPython and so forth, the solution is
>> to confront that problem head-on and have the debate. Having a
>> dictator come in and dictate the outcome wouldn't help anything.
>
> Yes, I see I am not being clear.  I was claiming that it is difficult
> to have this debate without someone who will make the final call.
> That person doesn't 'dictate' but they have the final say when all the
> arguments are in.   If I am right, then, with the no-leader consensus
> model you propose, it will be hard to have that debate because it is
> harder to get to the final decision.

I suggest we try it and see :-)

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Nathaniel Smith]

On Fri, Sep 4, 2015 at 1:53 PM, Matthew Brett  wrote:
> On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett  wrote:
>> Hi,
>>
>> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker  wrote:
>>> 1) I very much agree that governance can make or break a project. However,
>>> the actual governance approach often ends up making less difference than the
>>> people involved.
>>>
>>> 2) While the FreeBSD and XFree examples do point to some real problems with
>>> the "core" model it seems that there are many other projects that are using
>>> it quite successfully.
>
> I was just rereading the complaints about the 'core' structure from
> high-level NetBSD project leaders:
>
> "[the "core" and "board of directors"] teams are dysfunctional because
> they do not provide leadership: all they do is act reactively to
> requests from users and/or to resolve internal disputes. In other
> words: there is no initiative nor vision emerging from these teams
> (and, for that matter, from anybody)." [1]
>
> "There is no high-level direction; if you ask "what about the problems
> with threads" or "will there be a flash-friendly file system", the
> best you'll get is "we'd love to have both" -- but no work is done to
> recruit people to code these things, or encourage existing developers
> to work on them." [2]
>
> I imagine we will have to reconcile ourselves to similar problems, if
> we adopt the same structures.

I guess I just don't see how you think we can legislate ourselves into
having a vision. Like... we'll elect a volunteer to produce a roadmap,
and then if they don't we'll fire them (accountability!), leaving us
with no volunteer and no roadmap? How will that help?

The Jupyter/IPython project probably has one of the the most developed
roadmaps in all of open-source, since their grant-funded development
model requires them to actually commit to plans ahead of time. And
AFAIK the way they accomplish this has nothing to do with Fernando
sitting down and having visions; it involves dragging the "core" team
in front of a whiteboard 2x a year for a week, and seeing what they
can come up with. And I guess we have some tentative evidence from the
other thread that this strategy may work for us too...

I'm not sure how useful this focus on NetBSD is, given the elephant in
the room: the fundamental challenge for NetBSD is that NetBSD has no
compelling reason to exist; even its old distinctive feature of being
the most portable free OS (which is already an intrinsically niche
appeal) has been taken over by Linux. (And Linux, notably, has always
explicitly refused to have a roadmap [1][2]... it seems to be doing
okay.) Nonetheless, looking at the Hannum [3] post in particular, I'm
struck by how much it *doesn't* apply to us. In particular, his main
two prescriptions aside from "have leadership" are #6 and #7. #6 is
that the legal Foundation part of the project should restrict itself
to administrative activities and get out of technical decision making;
the draft governance document says "[The NumFOCUS] Subcommittee shall
NOT make decisions about the direction, scope or technical direction
of the Project.". #7 is:

   The "core" group must be replaced with people who are actually
   competent and dedicated enough to review proposals, accept feedback,
   and make good decisions.  More to the point, though, the "core" group
   must only act when *needed* -- most technical decisions should be
   left to the community to hash out; it must not preempt the community
   from developing better solutions.  (This is how the "core" group
   worked during most of the project's growth period.)

So he has nothing at all against the idea of a "core group", he just
thinks it should be competent and... basically follow the rules that
we attempted to codify in the draft governance document?

-n

[1] http://www.cnn.com/TECH/computing/9906/21/linus.idg/
[2] https://en.wikipedia.org/wiki/Linux_kernel#Development
[3] http://mail-index.netbsd.org/netbsd-users/2006/08/30/0016.html

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Nathaniel Smith]

On Sat, Sep 5, 2015 at 4:20 PM, Matthew Brett  wrote:
> Hi,
>
> Thanks for replying, that is helpful.
>
> I'm going to write carefully, because I suspect we've only got this
> one shot at getting it right.

Not sure why? "Change the governance document" is explicitly listed as
one of the steering council's powers, so I don't see what we'd lose by
adopting what we have now.

> I first wanted to come back to an earlier email of yours:
>
> On Thu, Aug 27, 2015 at 3:05 AM, Nathaniel Smith  wrote:
>> At least personally, I have to admit that the idea of a governance
>> model involving elections fills me with creeping horror. The reason is
>> that whole point of having a governance model (IMHO) is to (a)
>> minimize the rise of interpersonal drama, (b) when some amount of
>> interpersonal drama does inevitably arise anyway, provide some
>> regulated channel for it, hopefully one that leads to a drama sink.
>
> I think this is the wrong way to think of the problem.   There is no drama 
> when:
>
> * everyone agrees (problem is trivial);
> * no one cares (problem is not important);
> * everyone who does care has come to the conclusion that it is not
> worth arguing.
>
> The opposite of drama is not efficiency, but boredom and frustration.
>
> As evidence, vigorous, substantial and bad-tempered debates are
> characteristic of successful companies:
>
> """
> "They were chaos. We would stay there for hours, ironing out the
> issues, until we came to something ... At times the meetings would get
> so violent that people almost went across the table at each other
> People yelled. They waved their arms around and pounded on tables.
> Faces would get red and veins bulged out".
> """ [1]

No, by drama I mean situations where technical issues become personal,
and where people end up feeling emotionally hurt and disrespected and
so forth. I agree that vigorous debate is important, but strongly
disagree that veins bulging and violence are necessary or helpful to
success. (This is tangential to your main point, but I wanted to point
it out.) Since I happened to be reading Kate Heddleston's blog
yesterday, here's another relevant link...

https://www.kateheddleston.com/blog/argument-cultures-and-unregulated-aggression

> I'm sure you have hung out with the IPython team as I have.  You will
> then hear their description of how much the team disagree with each
> other, and often, how important that disagreement is, to finding the
> right answer.

Indeed, that's... why we have a document that is designed to provide a
framework for exactly that sort of debate? I don't really understand
the criticism here.

My objection to elections is that they create a whole *extra* source
of drama that runs the risk of compromising technical debates.
Suddenly people have to campaign and have fraught arguments over who
gets to vote and how to run the election and it's just borrowing
trouble IMO. (Are you by chance on the PSF members mailing list? If
not then ask someone how things have been going there vis-a-vis
elections -- I'm not myself, but I've heard stories...) Obviously it's
a fallacy to think that we can entirely ignore social issues and focus
only on technical problems, but still -- I'd rather we spend our
difficult debates on technical issues as much as possible.

Even worse, elections mean that someone might think "Hmm, should I
give in on this point, even though I know I'm right, so as to avoid
annoying this person who might otherwise retaliate by telling their
friends not to vote for me next year?" Or "hmm, did they give in on
that point because they actually agree or because the person they work
for wants the person they're agreeing with to win the next election?"
Or etc. Hopefully no-one would actually start playing politics to that
degree, but IMHO it's poisonous to even set up a situation where
people are tempted to perform such calculations.

> On the other hand, I don't think it is our culture in numpy to value
> that kind of debate.  That brings us to my next comment:
>
> On Fri, Sep 4, 2015 at 8:35 PM, Nathaniel Smith  wrote:
>> On Fri, Aug 28, 2015 at 1:46 AM, Matthew Brett  
>> wrote:
>>> Hi,
>>>
>>> On Fri, Aug 28, 2015 at 5:59 AM, Jaime Fernández del Río
>>>  wrote:
>> [...]
 
 Are you trying to prove the point that consensus doesn't work by making it
 impossible to reach a consensus on this? ;-)
 
>>>
>>> Forgive me if I use this joke to see if I can get us any further.
>>>
>>> If this was code, I think this joke would not be funny, because we
>>> wouldn't expect to reach consensus without considering all the
>>> options, and discussing their pros and cons.
>>>
>>> Why would that not be useful in the case of forms of governance?
>>>
>>> One reason might be that the specific form of governance can have no
>>> influence on the long-term health of the project.
>>>
>>> I am convinced that that is 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

Thanks for replying, that is helpful.

I'm going to write carefully, because I suspect we've only got this
one shot at getting it right.

I first wanted to come back to an earlier email of yours:

On Thu, Aug 27, 2015 at 3:05 AM, Nathaniel Smith  wrote:
> At least personally, I have to admit that the idea of a governance
> model involving elections fills me with creeping horror. The reason is
> that whole point of having a governance model (IMHO) is to (a)
> minimize the rise of interpersonal drama, (b) when some amount of
> interpersonal drama does inevitably arise anyway, provide some
> regulated channel for it, hopefully one that leads to a drama sink.

I think this is the wrong way to think of the problem.   There is no drama when:

* everyone agrees (problem is trivial);
* no one cares (problem is not important);
* everyone who does care has come to the conclusion that it is not
worth arguing.

The opposite of drama is not efficiency, but boredom and frustration.

As evidence, vigorous, substantial and bad-tempered debates are
characteristic of successful companies:

"""
"They were chaos. We would stay there for hours, ironing out the
issues, until we came to something ... At times the meetings would get
so violent that people almost went across the table at each other
People yelled. They waved their arms around and pounded on tables.
Faces would get red and veins bulged out".
""" [1]

I'm sure you have hung out with the IPython team as I have.  You will
then hear their description of how much the team disagree with each
other, and often, how important that disagreement is, to finding the
right answer.

On the other hand, I don't think it is our culture in numpy to value
that kind of debate.  That brings us to my next comment:

On Fri, Sep 4, 2015 at 8:35 PM, Nathaniel Smith  wrote:
> On Fri, Aug 28, 2015 at 1:46 AM, Matthew Brett  
> wrote:
>> Hi,
>>
>> On Fri, Aug 28, 2015 at 5:59 AM, Jaime Fernández del Río
>>  wrote:
> [...]
>>> 
>>> Are you trying to prove the point that consensus doesn't work by making it
>>> impossible to reach a consensus on this? ;-)
>>> 
>>
>> Forgive me if I use this joke to see if I can get us any further.
>>
>> If this was code, I think this joke would not be funny, because we
>> wouldn't expect to reach consensus without considering all the
>> options, and discussing their pros and cons.
>>
>> Why would that not be useful in the case of forms of governance?
>>
>> One reason might be that the specific form of governance can have no
>> influence on the long-term health of the project.
>>
>> I am convinced that that is wrong - that the form of governance has a
>> large influence on the long-term health of a project.
>>
>> If there is some possibility that this is true, then it seems to me
>> that we would be foolish not to try and come to some reasoned choice
>> about the form of governance.
>
> That seems fair, and I also think that even if we end up not changing
> anything then there's still some utility in having the discussion.
> Certainly one of my secret fears in writing all this up is that it
> would end up being just some words that only I cared about and
> everyone else ignored, so seeing other people engaging with it is
> heartening :-).
>
> I'm not feeling inspired this Friday evening to write a full white
> paper reviewing the universe of possible governance, but I'll
> summarize a bit what the reasoning was that led us to what we ended up
> with (at least as I remember it and from my perspective).
>
> Some of the sources that were mentioned for reference and positive or
> negative inspiration include Karl Fogel's chapter on governance, the
> Debian constitution, the gcc/egcs fork+reverse-fork, glibc/eglibc
> ditto, subversion, Jupyter/ipython, the Linux kernel, python,
> matplotlib, the Apache foundation's processes, the gnome foundation,
> nodejs, ... I'm certainly forgetting some.
>
> The basic observation that we started from was that all the successful
> projects seemed to fit a similar template:
> (1) basically all actual decision making is done by informal and
> maximally inclusive consensus-based processes,
> (2) there are some sort of more formal "backstop" rules that are
> called into play if informal processes break down (which hopefully
> never even happens).
> These projects don't tend to look that similar on paper, because
> people tend to only write down the "backstop" rules, and those vary a
> lot.
>
> So e.g., for some projects like Python or IPython, the formal rule is
> "the BDFL decides". But then if you look at what successful BDFLs do
> with this power, they mostly enforce rule #1 above. (Karl discusses
> this: http://producingoss.com/en/producingoss.html#benevolent-dictator).
> E.g., if you watch Guido, I don't think I've ever seen him say "I know
> people don't like X but we're doing it anyway". He mostly sits back to
> watch the debate, steps in 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

On Sat, Sep 5, 2015 at 7:14 PM, Nathaniel Smith  wrote:
> On Sat, Sep 5, 2015 at 4:20 PM, Matthew Brett  wrote:
>> Hi,
>>
>> Thanks for replying, that is helpful.
>>
>> I'm going to write carefully, because I suspect we've only got this
>> one shot at getting it right.
>
> Not sure why? "Change the governance document" is explicitly listed as
> one of the steering council's powers, so I don't see what we'd lose by
> adopting what we have now.

In practice, I doubt that there will be any proposal to change the
governance in the medium term, especially if the governance does turn
out to be risk averse and conservative.

>> I first wanted to come back to an earlier email of yours:
>>
>> On Thu, Aug 27, 2015 at 3:05 AM, Nathaniel Smith  wrote:
>>> At least personally, I have to admit that the idea of a governance
>>> model involving elections fills me with creeping horror. The reason is
>>> that whole point of having a governance model (IMHO) is to (a)
>>> minimize the rise of interpersonal drama, (b) when some amount of
>>> interpersonal drama does inevitably arise anyway, provide some
>>> regulated channel for it, hopefully one that leads to a drama sink.
>>
>> I think this is the wrong way to think of the problem.   There is no drama 
>> when:
>>
>> * everyone agrees (problem is trivial);
>> * no one cares (problem is not important);
>> * everyone who does care has come to the conclusion that it is not
>> worth arguing.
>>
>> The opposite of drama is not efficiency, but boredom and frustration.
>>
>> As evidence, vigorous, substantial and bad-tempered debates are
>> characteristic of successful companies:
>>
>> """
>> "They were chaos. We would stay there for hours, ironing out the
>> issues, until we came to something ... At times the meetings would get
>> so violent that people almost went across the table at each other
>> People yelled. They waved their arms around and pounded on tables.
>> Faces would get red and veins bulged out".
>> """ [1]
>
> No, by drama I mean situations where technical issues become personal,
> and where people end up feeling emotionally hurt and disrespected and
> so forth. I agree that vigorous debate is important, but strongly
> disagree that veins bulging and violence are necessary or helpful to
> success. (This is tangential to your main point, but I wanted to point
> it out.) Since I happened to be reading Kate Heddleston's blog
> yesterday, here's another relevant link...
> 
> https://www.kateheddleston.com/blog/argument-cultures-and-unregulated-aggression

The distinction is between a culture that welcomes expressions of
disagreement with one that does not.  In practice, if a group is
easily alarmed by expressions of strong feeling, most people will be
careful about expressing disagreement.   This is of course a rather
subtle point.  It is easy to say that we do welcome disagreement, when
in practice what happens is that the person disagreeing will in fact
be labeled as being impolite.

>> then hear their description of how much the team disagree with each
>> other, and often, how important that disagreement is, to finding the
>> right answer.
>
> Indeed, that's... why we have a document that is designed to provide a
> framework for exactly that sort of debate? I don't really understand
> the criticism here.

Er - I was arguing that, by attempting to formalize a system that
doesn't have a leader, that will lead to a culture where debate is
more difficult and less effective.

> My objection to elections is that they create a whole *extra* source
> of drama that runs the risk of compromising technical debates.
> Suddenly people have to campaign and have fraught arguments over who
> gets to vote and how to run the election and it's just borrowing
> trouble IMO. (Are you by chance on the PSF members mailing list? If
> not then ask someone how things have been going there vis-a-vis
> elections -- I'm not myself, but I've heard stories...) Obviously it's
> a fallacy to think that we can entirely ignore social issues and focus
> only on technical problems, but still -- I'd rather we spend our
> difficult debates on technical issues as much as possible.
>
> Even worse, elections mean that someone might think "Hmm, should I
> give in on this point, even though I know I'm right, so as to avoid
> annoying this person who might otherwise retaliate by telling their
> friends not to vote for me next year?" Or "hmm, did they give in on
> that point because they actually agree or because the person they work
> for wants the person they're agreeing with to win the next election?"
> Or etc. Hopefully no-one would actually start playing politics to that
> degree, but IMHO it's poisonous to even set up a situation where
> people are tempted to perform such calculations.

I suppose these things might happen.  The benefit of elections is that
it forces us to be explicit about what the issues are, and what is at
stake.   It seems 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Eric Firing]

On 2015/09/04 10:53 AM, Matthew Brett wrote:
> On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett  wrote:
>> Hi,
>>
>> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker  wrote:
>>> 1) I very much agree that governance can make or break a project. However,
>>> the actual governance approach often ends up making less difference than the
>>> people involved.
>>>
>>> 2) While the FreeBSD and XFree examples do point to some real problems with
>>> the "core" model it seems that there are many other projects that are using
>>> it quite successfully.
>
> I was just rereading the complaints about the 'core' structure from
> high-level NetBSD project leaders:
>
> "[the "core" and "board of directors"] teams are dysfunctional because
> they do not provide leadership: all they do is act reactively to
> requests from users and/or to resolve internal disputes. In other
> words: there is no initiative nor vision emerging from these teams
> (and, for that matter, from anybody)." [1]
>
> "There is no high-level direction; if you ask "what about the problems
> with threads" or "will there be a flash-friendly file system", the
> best you'll get is "we'd love to have both" -- but no work is done to
> recruit people to code these things, or encourage existing developers
> to work on them." [2]


This is consistent with Chris's first point.

>
> I imagine we will have to reconcile ourselves to similar problems, if
> we adopt the same structures.

Do you have suggestions as to who would make a good numpy president or 
BDFL and potentially has the time and inclination to do it, or how to 
identify and recruit such a person?

Eric

>
> Cheers,
>
> Matthew
>
> [1] 
> http://julipedia.meroh.net/2013/06/self-interview-after-leaving-netbsd.html
> [2] http://mail-index.netbsd.org/netbsd-users/2006/08/30/0016.html
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett  wrote:
> Hi,
>
> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker  wrote:
>> 1) I very much agree that governance can make or break a project. However,
>> the actual governance approach often ends up making less difference than the
>> people involved.
>>
>> 2) While the FreeBSD and XFree examples do point to some real problems with
>> the "core" model it seems that there are many other projects that are using
>> it quite successfully.

I was just rereading the complaints about the 'core' structure from
high-level NetBSD project leaders:

"[the "core" and "board of directors"] teams are dysfunctional because
they do not provide leadership: all they do is act reactively to
requests from users and/or to resolve internal disputes. In other
words: there is no initiative nor vision emerging from these teams
(and, for that matter, from anybody)." [1]

"There is no high-level direction; if you ask "what about the problems
with threads" or "will there be a flash-friendly file system", the
best you'll get is "we'd love to have both" -- but no work is done to
recruit people to code these things, or encourage existing developers
to work on them." [2]

I imagine we will have to reconcile ourselves to similar problems, if
we adopt the same structures.

Cheers,

Matthew

[1] http://julipedia.meroh.net/2013/06/self-interview-after-leaving-netbsd.html
[2] http://mail-index.netbsd.org/netbsd-users/2006/08/30/0016.html
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Nathaniel Smith]

On Fri, Aug 28, 2015 at 1:46 AM, Matthew Brett  wrote:
> Hi,
>
> On Fri, Aug 28, 2015 at 5:59 AM, Jaime Fernández del Río
>  wrote:
[...]
>> 
>> Are you trying to prove the point that consensus doesn't work by making it
>> impossible to reach a consensus on this? ;-)
>> 
>
> Forgive me if I use this joke to see if I can get us any further.
>
> If this was code, I think this joke would not be funny, because we
> wouldn't expect to reach consensus without considering all the
> options, and discussing their pros and cons.
>
> Why would that not be useful in the case of forms of governance?
>
> One reason might be that the specific form of governance can have no
> influence on the long-term health of the project.
>
> I am convinced that that is wrong - that the form of governance has a
> large influence on the long-term health of a project.
>
> If there is some possibility that this is true, then it seems to me
> that we would be foolish not to try and come to some reasoned choice
> about the form of governance.

That seems fair, and I also think that even if we end up not changing
anything then there's still some utility in having the discussion.
Certainly one of my secret fears in writing all this up is that it
would end up being just some words that only I cared about and
everyone else ignored, so seeing other people engaging with it is
heartening :-).

I'm not feeling inspired this Friday evening to write a full white
paper reviewing the universe of possible governance, but I'll
summarize a bit what the reasoning was that led us to what we ended up
with (at least as I remember it and from my perspective).

Some of the sources that were mentioned for reference and positive or
negative inspiration include Karl Fogel's chapter on governance, the
Debian constitution, the gcc/egcs fork+reverse-fork, glibc/eglibc
ditto, subversion, Jupyter/ipython, the Linux kernel, python,
matplotlib, the Apache foundation's processes, the gnome foundation,
nodejs, ... I'm certainly forgetting some.

The basic observation that we started from was that all the successful
projects seemed to fit a similar template:
(1) basically all actual decision making is done by informal and
maximally inclusive consensus-based processes,
(2) there are some sort of more formal "backstop" rules that are
called into play if informal processes break down (which hopefully
never even happens).
These projects don't tend to look that similar on paper, because
people tend to only write down the "backstop" rules, and those vary a
lot.

So e.g., for some projects like Python or IPython, the formal rule is
"the BDFL decides". But then if you look at what successful BDFLs do
with this power, they mostly enforce rule #1 above. (Karl discusses
this: http://producingoss.com/en/producingoss.html#benevolent-dictator).
E.g., if you watch Guido, I don't think I've ever seen him say "I know
people don't like X but we're doing it anyway". He mostly sits back to
watch the debate, steps in occasionally to prune off unproductive
lines of conversation (e.g. in the PEP 465 debate where he watched for
a while, stepped in to tell people to knock it off with the pointless
bikeshedding about which character to use, and then went back to
watching) or encourage people to keep thinking about something, is
always very careful not to actually make any strong statements, etc.
Or, e.g., the bit in the draft governance about how the steering
committee's secondary job is to make decisions, and their primary job
is to prevent things reaching the point where they have to? That's a
straight paraphrase of Fernando talking about his BDFL philosophy.

Or on the other end of things, you have e.g. Subversion, which had an
elaborate defined governance system with different levels of
"core-ness", a voting system, etc. -- and they were 6 years into the
project before they had their first vote. (The vote was on the crucial
technical decision of whether to write function calls like "f ()" or
"f()".) Or e.g. the massive and famously fractious Debian has their
CTTE, which theoretically is the final arbiter of technical decisions
in Debian and has immense powers... but they issue like ~5 rulings a
year on average, and have gone years without issuing a ruling at all.

So, our thought process was: for our purposes, since we don't have a
BDFL, we can't depend on them to be the unwritten mechanism for
enforcing rule #1, so first we should write down rule #1. And then we
should attach some kind of workable backstop procedure. "Consensus
among a core group" seemed like a good bet. It does mean you need to
have a formal rule for defining who is in the "core group", because
the point of having a formal backstop rule is that it be... formal...
but otherwise it's about as simple as you can get: it requires minimal
voting logistics (it's trivial to tell whether you have consensus,
because if someone disagrees, there's your answer), it's minimally
gameable (no 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

On Sat, Sep 5, 2015 at 12:04 AM,   wrote:
>
>
> On Fri, Sep 4, 2015 at 5:55 PM, Matthew Brett 
> wrote:
>>
>> Hi,
>>
>> On Fri, Sep 4, 2015 at 10:22 PM, Eric Firing  wrote:
>> > On 2015/09/04 10:53 AM, Matthew Brett wrote:
>> >> On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett 
>> >> wrote:
>> >>> Hi,
>> >>>
>> >>> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker 
>> >>> wrote:
>>  1) I very much agree that governance can make or break a project.
>>  However,
>>  the actual governance approach often ends up making less difference
>>  than the
>>  people involved.
>> 
>>  2) While the FreeBSD and XFree examples do point to some real
>>  problems with
>>  the "core" model it seems that there are many other projects that are
>>  using
>>  it quite successfully.
>> >>
>> >> I was just rereading the complaints about the 'core' structure from
>> >> high-level NetBSD project leaders:
>> >>
>> >> "[the "core" and "board of directors"] teams are dysfunctional because
>> >> they do not provide leadership: all they do is act reactively to
>> >> requests from users and/or to resolve internal disputes. In other
>> >> words: there is no initiative nor vision emerging from these teams
>> >> (and, for that matter, from anybody)." [1]
>> >>
>> >> "There is no high-level direction; if you ask "what about the problems
>> >> with threads" or "will there be a flash-friendly file system", the
>> >> best you'll get is "we'd love to have both" -- but no work is done to
>> >> recruit people to code these things, or encourage existing developers
>> >> to work on them." [2]
>> >
>> >
>> > This is consistent with Chris's first point.
>>
>> Do you mean Chris' point that "I very much agree that governance can
>> make or break a project"?   Charles Hannum's complaints about NetBSD
>> are very specific in blaming the model rather than the people.   I
>> think the XFree86 story supports the same conclusion - that the
>> governance model caused a sense of diffused responsibility that lead
>> to bad decisions and lack of direction.
>>
>> >> I imagine we will have to reconcile ourselves to similar problems, if
>> >> we adopt the same structures.
>> >
>> > Do you have suggestions as to who would make a good numpy president or
>> > BDFL and potentially has the time and inclination to do it, or how to
>> > identify and recruit such a person?
>>
>> That's a good question, and the answer is that in the current
>> situation (zero interest in this discussion from the three current
>> members of the numpy leadership team) - no reasonable person would be
>> interested in that job.   That's the situation we're in, and so we
>> have to accept that nothing is going to change, with the consequences
>> that implies.   If the situation were different, and we had the
>> interest or commitment to explore this problem, then I guess we could
>> discuss other options including the one I suggested further up the
>> thread.
>
>
> "
>
> Today, the project is run by a different cabal.  This is the result of a
> coup that took place in 2000-2001, in which The NetBSD Foundation was
> taken over by a fraudulent change of the board of directors.  (Note:
> It's probably too late for me to pursue any legal remedy for this,
> unfortunately.)  Although "The NetBSD Project" and "The NetBSD
> Foundation" were intended from the start to be separate entities -- the
> latter supplying support infrastructure for the former -- this
> distinction has been actively blurred since, so that the current "board"
> of TNF has rather tight control over many aspects of TNP.
>
> "
>
> "
>
> The existing NetBSD Foundation must be disbanded, and replaced with
>an organization that fulfills its original purpose: to merely handle
>administrative issues, and not to manage day-to-day affairs.
>
> "
>
>
> It doesn't sound to me like a developer and community driven governance
> structure to me.

I think that's a separate issue - the distinction between the 'board'
and the 'core'.   It would be great if the 'core' concept was fine as
long as there is no 'board' but I think that's a hard argument to
make.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[josef.pktd]

On Fri, Sep 4, 2015 at 7:19 PM, Matthew Brett 
wrote:

> On Sat, Sep 5, 2015 at 12:04 AM,   wrote:
> >
> >
> > On Fri, Sep 4, 2015 at 5:55 PM, Matthew Brett 
> > wrote:
> >>
> >> Hi,
> >>
> >> On Fri, Sep 4, 2015 at 10:22 PM, Eric Firing 
> wrote:
> >> > On 2015/09/04 10:53 AM, Matthew Brett wrote:
> >> >> On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett <
> matthew.br...@gmail.com>
> >> >> wrote:
> >> >>> Hi,
> >> >>>
> >> >>> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker  >
> >> >>> wrote:
> >>  1) I very much agree that governance can make or break a project.
> >>  However,
> >>  the actual governance approach often ends up making less difference
> >>  than the
> >>  people involved.
> >> 
> >>  2) While the FreeBSD and XFree examples do point to some real
> >>  problems with
> >>  the "core" model it seems that there are many other projects that
> are
> >>  using
> >>  it quite successfully.
> >> >>
> >> >> I was just rereading the complaints about the 'core' structure from
> >> >> high-level NetBSD project leaders:
> >> >>
> >> >> "[the "core" and "board of directors"] teams are dysfunctional
> because
> >> >> they do not provide leadership: all they do is act reactively to
> >> >> requests from users and/or to resolve internal disputes. In other
> >> >> words: there is no initiative nor vision emerging from these teams
> >> >> (and, for that matter, from anybody)." [1]
> >> >>
> >> >> "There is no high-level direction; if you ask "what about the
> problems
> >> >> with threads" or "will there be a flash-friendly file system", the
> >> >> best you'll get is "we'd love to have both" -- but no work is done to
> >> >> recruit people to code these things, or encourage existing developers
> >> >> to work on them." [2]
> >> >
> >> >
> >> > This is consistent with Chris's first point.
> >>
> >> Do you mean Chris' point that "I very much agree that governance can
> >> make or break a project"?   Charles Hannum's complaints about NetBSD
> >> are very specific in blaming the model rather than the people.   I
> >> think the XFree86 story supports the same conclusion - that the
> >> governance model caused a sense of diffused responsibility that lead
> >> to bad decisions and lack of direction.
> >>
> >> >> I imagine we will have to reconcile ourselves to similar problems, if
> >> >> we adopt the same structures.
> >> >
> >> > Do you have suggestions as to who would make a good numpy president or
> >> > BDFL and potentially has the time and inclination to do it, or how to
> >> > identify and recruit such a person?
> >>
> >> That's a good question, and the answer is that in the current
> >> situation (zero interest in this discussion from the three current
> >> members of the numpy leadership team) - no reasonable person would be
> >> interested in that job.   That's the situation we're in, and so we
> >> have to accept that nothing is going to change, with the consequences
> >> that implies.   If the situation were different, and we had the
> >> interest or commitment to explore this problem, then I guess we could
> >> discuss other options including the one I suggested further up the
> >> thread.
> >
> >
> > "
> >
> > Today, the project is run by a different cabal.  This is the result of a
> > coup that took place in 2000-2001, in which The NetBSD Foundation was
> > taken over by a fraudulent change of the board of directors.  (Note:
> > It's probably too late for me to pursue any legal remedy for this,
> > unfortunately.)  Although "The NetBSD Project" and "The NetBSD
> > Foundation" were intended from the start to be separate entities -- the
> > latter supplying support infrastructure for the former -- this
> > distinction has been actively blurred since, so that the current "board"
> > of TNF has rather tight control over many aspects of TNP.
> >
> > "
> >
> > "
> >
> > The existing NetBSD Foundation must be disbanded, and replaced with
> >an organization that fulfills its original purpose: to merely handle
> >administrative issues, and not to manage day-to-day affairs.
> >
> > "
> >
> >
> > It doesn't sound to me like a developer and community driven governance
> > structure to me.
>
> I think that's a separate issue - the distinction between the 'board'
> and the 'core'.   It would be great if the 'core' concept was fine as
> long as there is no 'board' but I think that's a hard argument to
> make.
>

there is an "esprit de corps" pronounced "esprit de core" but not an
"esprit de board"

I trust the core developers, but not ...

But maybe I don't understand some definitions

"

The "core" group must be replaced with people who are actually
   competent and dedicated enough to review proposals, accept feedback,
   and make good decisions.

"

I thought that's what the "core" group is.

Josef



>
> Cheers,
>
> Matthew
> 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[josef.pktd]

On Fri, Sep 4, 2015 at 5:55 PM, Matthew Brett 
wrote:

> Hi,
>
> On Fri, Sep 4, 2015 at 10:22 PM, Eric Firing  wrote:
> > On 2015/09/04 10:53 AM, Matthew Brett wrote:
> >> On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett 
> wrote:
> >>> Hi,
> >>>
> >>> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker 
> wrote:
>  1) I very much agree that governance can make or break a project.
> However,
>  the actual governance approach often ends up making less difference
> than the
>  people involved.
> 
>  2) While the FreeBSD and XFree examples do point to some real
> problems with
>  the "core" model it seems that there are many other projects that are
> using
>  it quite successfully.
> >>
> >> I was just rereading the complaints about the 'core' structure from
> >> high-level NetBSD project leaders:
> >>
> >> "[the "core" and "board of directors"] teams are dysfunctional because
> >> they do not provide leadership: all they do is act reactively to
> >> requests from users and/or to resolve internal disputes. In other
> >> words: there is no initiative nor vision emerging from these teams
> >> (and, for that matter, from anybody)." [1]
> >>
> >> "There is no high-level direction; if you ask "what about the problems
> >> with threads" or "will there be a flash-friendly file system", the
> >> best you'll get is "we'd love to have both" -- but no work is done to
> >> recruit people to code these things, or encourage existing developers
> >> to work on them." [2]
> >
> >
> > This is consistent with Chris's first point.
>
> Do you mean Chris' point that "I very much agree that governance can
> make or break a project"?   Charles Hannum's complaints about NetBSD
> are very specific in blaming the model rather than the people.   I
> think the XFree86 story supports the same conclusion - that the
> governance model caused a sense of diffused responsibility that lead
> to bad decisions and lack of direction.
>
> >> I imagine we will have to reconcile ourselves to similar problems, if
> >> we adopt the same structures.
> >
> > Do you have suggestions as to who would make a good numpy president or
> > BDFL and potentially has the time and inclination to do it, or how to
> > identify and recruit such a person?
>
> That's a good question, and the answer is that in the current
> situation (zero interest in this discussion from the three current
> members of the numpy leadership team) - no reasonable person would be
> interested in that job.   That's the situation we're in, and so we
> have to accept that nothing is going to change, with the consequences
> that implies.   If the situation were different, and we had the
> interest or commitment to explore this problem, then I guess we could
> discuss other options including the one I suggested further up the
> thread.
>

"

Today, the project is run by a different cabal.  This is the result of a
coup that took place in 2000-2001, in which The NetBSD Foundation was
taken over by a fraudulent change of the board of directors.  (Note:
It's probably too late for me to pursue any legal remedy for this,
unfortunately.)  Although "The NetBSD Project" and "The NetBSD
Foundation" were intended from the start to be separate entities -- the
latter supplying support infrastructure for the former -- this
distinction has been actively blurred since, so that the current "board"
of TNF has rather tight control over many aspects of TNP.

"

"

The existing NetBSD Foundation must be disbanded, and replaced with
   an organization that fulfills its original purpose: to merely handle
   administrative issues, and not to manage day-to-day affairs.

"


It doesn't sound to me like a developer and community driven governance
structure to me.

Cheers

Josef
https://jeb2016.com/?lang=es



>
> Cheers,
>
> Matthew
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Fri, Sep 4, 2015 at 10:22 PM, Eric Firing  wrote:
> On 2015/09/04 10:53 AM, Matthew Brett wrote:
>> On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett  
>> wrote:
>>> Hi,
>>>
>>> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker  wrote:
 1) I very much agree that governance can make or break a project. However,
 the actual governance approach often ends up making less difference than 
 the
 people involved.

 2) While the FreeBSD and XFree examples do point to some real problems with
 the "core" model it seems that there are many other projects that are using
 it quite successfully.
>>
>> I was just rereading the complaints about the 'core' structure from
>> high-level NetBSD project leaders:
>>
>> "[the "core" and "board of directors"] teams are dysfunctional because
>> they do not provide leadership: all they do is act reactively to
>> requests from users and/or to resolve internal disputes. In other
>> words: there is no initiative nor vision emerging from these teams
>> (and, for that matter, from anybody)." [1]
>>
>> "There is no high-level direction; if you ask "what about the problems
>> with threads" or "will there be a flash-friendly file system", the
>> best you'll get is "we'd love to have both" -- but no work is done to
>> recruit people to code these things, or encourage existing developers
>> to work on them." [2]
>
>
> This is consistent with Chris's first point.

Do you mean Chris' point that "I very much agree that governance can
make or break a project"?   Charles Hannum's complaints about NetBSD
are very specific in blaming the model rather than the people.   I
think the XFree86 story supports the same conclusion - that the
governance model caused a sense of diffused responsibility that lead
to bad decisions and lack of direction.

>> I imagine we will have to reconcile ourselves to similar problems, if
>> we adopt the same structures.
>
> Do you have suggestions as to who would make a good numpy president or
> BDFL and potentially has the time and inclination to do it, or how to
> identify and recruit such a person?

That's a good question, and the answer is that in the current
situation (zero interest in this discussion from the three current
members of the numpy leadership team) - no reasonable person would be
interested in that job.   That's the situation we're in, and so we
have to accept that nothing is going to change, with the consequences
that implies.   If the situation were different, and we had the
interest or commitment to explore this problem, then I guess we could
discuss other options including the one I suggested further up the
thread.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

On Sat, Sep 5, 2015 at 12:47 AM,   wrote:
>
>
> On Fri, Sep 4, 2015 at 7:19 PM, Matthew Brett 
> wrote:
>>
>> On Sat, Sep 5, 2015 at 12:04 AM,   wrote:
>> >
>> >
>> > On Fri, Sep 4, 2015 at 5:55 PM, Matthew Brett 
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> On Fri, Sep 4, 2015 at 10:22 PM, Eric Firing 
>> >> wrote:
>> >> > On 2015/09/04 10:53 AM, Matthew Brett wrote:
>> >> >> On Fri, Sep 4, 2015 at 2:33 AM, Matthew Brett
>> >> >> 
>> >> >> wrote:
>> >> >>> Hi,
>> >> >>>
>> >> >>> On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker
>> >> >>> 
>> >> >>> wrote:
>> >>  1) I very much agree that governance can make or break a project.
>> >>  However,
>> >>  the actual governance approach often ends up making less
>> >>  difference
>> >>  than the
>> >>  people involved.
>> >> 
>> >>  2) While the FreeBSD and XFree examples do point to some real
>> >>  problems with
>> >>  the "core" model it seems that there are many other projects that
>> >>  are
>> >>  using
>> >>  it quite successfully.
>> >> >>
>> >> >> I was just rereading the complaints about the 'core' structure from
>> >> >> high-level NetBSD project leaders:
>> >> >>
>> >> >> "[the "core" and "board of directors"] teams are dysfunctional
>> >> >> because
>> >> >> they do not provide leadership: all they do is act reactively to
>> >> >> requests from users and/or to resolve internal disputes. In other
>> >> >> words: there is no initiative nor vision emerging from these teams
>> >> >> (and, for that matter, from anybody)." [1]
>> >> >>
>> >> >> "There is no high-level direction; if you ask "what about the
>> >> >> problems
>> >> >> with threads" or "will there be a flash-friendly file system", the
>> >> >> best you'll get is "we'd love to have both" -- but no work is done
>> >> >> to
>> >> >> recruit people to code these things, or encourage existing
>> >> >> developers
>> >> >> to work on them." [2]
>> >> >
>> >> >
>> >> > This is consistent with Chris's first point.
>> >>
>> >> Do you mean Chris' point that "I very much agree that governance can
>> >> make or break a project"?   Charles Hannum's complaints about NetBSD
>> >> are very specific in blaming the model rather than the people.   I
>> >> think the XFree86 story supports the same conclusion - that the
>> >> governance model caused a sense of diffused responsibility that lead
>> >> to bad decisions and lack of direction.
>> >>
>> >> >> I imagine we will have to reconcile ourselves to similar problems,
>> >> >> if
>> >> >> we adopt the same structures.
>> >> >
>> >> > Do you have suggestions as to who would make a good numpy president
>> >> > or
>> >> > BDFL and potentially has the time and inclination to do it, or how to
>> >> > identify and recruit such a person?
>> >>
>> >> That's a good question, and the answer is that in the current
>> >> situation (zero interest in this discussion from the three current
>> >> members of the numpy leadership team) - no reasonable person would be
>> >> interested in that job.   That's the situation we're in, and so we
>> >> have to accept that nothing is going to change, with the consequences
>> >> that implies.   If the situation were different, and we had the
>> >> interest or commitment to explore this problem, then I guess we could
>> >> discuss other options including the one I suggested further up the
>> >> thread.
>> >
>> >
>> > "
>> >
>> > Today, the project is run by a different cabal.  This is the result of a
>> > coup that took place in 2000-2001, in which The NetBSD Foundation was
>> > taken over by a fraudulent change of the board of directors.  (Note:
>> > It's probably too late for me to pursue any legal remedy for this,
>> > unfortunately.)  Although "The NetBSD Project" and "The NetBSD
>> > Foundation" were intended from the start to be separate entities -- the
>> > latter supplying support infrastructure for the former -- this
>> > distinction has been actively blurred since, so that the current "board"
>> > of TNF has rather tight control over many aspects of TNP.
>> >
>> > "
>> >
>> > "
>> >
>> > The existing NetBSD Foundation must be disbanded, and replaced with
>> >an organization that fulfills its original purpose: to merely handle
>> >administrative issues, and not to manage day-to-day affairs.
>> >
>> > "
>> >
>> >
>> > It doesn't sound to me like a developer and community driven governance
>> > structure to me.
>>
>> I think that's a separate issue - the distinction between the 'board'
>> and the 'core'.   It would be great if the 'core' concept was fine as
>> long as there is no 'board' but I think that's a hard argument to
>> make.
>
>
> there is an "esprit de corps" pronounced "esprit de core" but not an "esprit
> de board"
>
> I trust the core developers, but not ...
>
> But maybe I don't understand some definitions
>
> 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Wed, Sep 2, 2015 at 5:41 PM, Chris Barker  wrote:
> 1) I very much agree that governance can make or break a project. However,
> the actual governance approach often ends up making less difference than the
> people involved.
>
> 2) While the FreeBSD and XFree examples do point to some real problems with
> the "core" model it seems that there are many other projects that are using
> it quite successfully.
>
> 3) at least in the XFree case, maybe the biggest issue was not the core (or
> the board), but the fact that discussions and decision making process were
> kept secret (and restriction to CVS, EVEN read-only) -- this does not seem
> to be inherent to the "core" model at all, and not how nupy will even
> operate.

This was an issue, but not the central one.  I would argue that the
central problem was that XFree86 had grown up slowly and organically,
to find itself at the heart of the Linux ecosystem.  At this point the
private developers-club aspect of XFree86 became a massive hindrance
to the huge pressure from outside to develop new stuff.

> 4) the biggest issue numpy has faced in years is a lack of people that
> can/will/do actually contribute code itself. there simply isn't a big group
> f folks waiting to step up and be the president. And the "leader", president
> of BDFL, etc, while primarily being a management role, needs to be fully
> technically competent -- there are very few people qualified at this point,
> and I suspect none of them want that job. -- in theory, one can manage
> without the technical competency, but does anyone have a single example of a
> successful open-source software project run by a good manager that isn't a
> top technical expert? All the BDFLs I know of are absolutely top of the heap
> technically.

I can't easily imagine someone without reasonable technical ability
wanting to lead the project - although it appears that technical
ability is overrated for technical management [1].

> 5) Sadly, at the end of the day, democracy is often a way to make very poor
> decisions.

I don't think the question here is whether to have democracy or not,
but what kind of democracy.   In fact the current proposal is a sort
of consensus democracy.  My interest is whether this really is a good
model for numpy.   For example, will this model lead to a lack of
accountability and coherent direction?  Will it make it harder for
other projects to interact with numpy and negotiate about numpy
direction?

> So -- it seems there is consensus that we need to formalize the governance
> of numpy. And honestly, I don't think there are any other options that would
> work at this point with the current community.

It is true, that this discussion needs some commitment.  That either
has to come from {Chuck, Nathaniel, Ralf} or very strongly from the
rest of us, and we don't have either of those conditions, so I think
we are going to fall back to the status quo by default.

Cheers,

Matthew


[1] http://www.nytimes.com/2011/03/13/business/13hire.html?_r=0
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Chris Barker]

1) I very much agree that governance can make or break a project. However,
the actual governance approach often ends up making less difference than
the people involved.

2) While the FreeBSD and XFree examples do point to some real problems with
the "core" model it seems that there are many other projects that are using
it quite successfully.

3) at least in the XFree case, maybe the biggest issue was not the core (or
the board), but the fact that discussions and decision making process were
kept secret (and restriction to CVS, EVEN read-only) -- this does not seem
to be inherent to the "core" model at all, and not how nupy will even
operate.

4) the biggest issue numpy has faced in years is a lack of people that
can/will/do actually contribute code itself. there simply isn't a big group
f folks waiting to step up and be the president. And the "leader",
president of BDFL, etc, while primarily being a management role, needs to
be fully technically competent -- there are very few people qualified at
this point, and I suspect none of them want that job. -- in theory, one can
manage without the technical competency, but does anyone have a single
example of a successful open-source software project run by a good manager
that isn't a top technical expert? All the BDFLs I know of are absolutely
top of the heap technically.

5) Sadly, at the end of the day, democracy is often a way to make very poor
decisions.

So -- it seems there is consensus that we need to formalize the governance
of numpy. And honestly, I don't think there are any other options that
would work at this point with the current community.


And the biggest point: while forking may well demonstrate a complete
failure of the governance of a project -- it also represents a success of
open-source. In fact, when I read Mathew's history of X11 -- it seems the
biggest impediment at each stage were the efforts to make it less that
truly open-source.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Francesc Alted]

+10  Very well written down ideas Jaime.

2015-08-28 6:59 GMT+02:00 Jaime Fernández del Río jaime.f...@gmail.com:

 On Thu, Aug 27, 2015 at 11:06 AM, Matthew Brett matthew.br...@gmail.com
 wrote:

 Hi,

 On Thu, Aug 27, 2015 at 6:23 PM,  josef.p...@gmail.com wrote:
 
 
  On Thu, Aug 27, 2015 at 12:22 PM, Matthew Brett 
 matthew.br...@gmail.com
  wrote:
 
  Hi
 
  On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:
  
  
   On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett
   matthew.br...@gmail.com
   wrote:
  
   Hi,
  
   On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
   [snip]
I don't really see a problem with codifying the status quo.
  
   That's an excellent point.If we believe that the current
 situation
   is the best possible, both now and in the future, then codifying the
   status quo is an excellent idea.
  
   So, we should probably first start by asking ourselves:
  
   * what numpy is doing well;
   * what numpy could do better;
  
   and then ask, is there some way we could make it more likely we will
   improve over time.
  
   [snip]
  
As the current debate shows it's possible to have a public
 discussion
about
the direction of the project without having to delegate providing
 a
vision
to a president.
  
   The idea of a president that I had in mind, was not someone who
 makes
   all decisions, but the person who holds themselves responsible for
 the
   performance of the project.  If the project has a coherent vision
   already, the president has no need to provide one, but it's the
   president's job to worry about whether we have vision or not, and do
   what they need to, to make sure we don't lose track of that.   If
 you
   don't know it already, I highly recommend Jim Collins' work on
 'level
   5 leadership' [1]
  
  
   Still doesn't sound like the need for a president to me
  
the person who holds themselves responsible for the
   performance of the project
  
   sounds more like the role of the core group (adding plural to
 persons)
   to
   me, and cannot be pushed of to an official president.
 
  Except that, in the past, having multiple people taking decisions has
  led to the situation where no-one feels themselves accountable for the
  result, hence this situation tends to lead to stagnation.
 
 
  Is there any evidence for this?

 Oh - dear - that's the key point, but I'm obviously not making it
 clearly enough.  Yes there is, and that was the evidence I was
 pointing to before.

 But anyway - Sebastian is right - this discussion isn't going anywhere
 useful.

 So - let's step back.

 In thinking about governance, we first need to ask what we want to
 achieve.  This includes considering the risks ahead for the project.

 So, in the spirit of fruitful discussion, can I ask what y'all
 consider to be the current problems with working on numpy (other than
 the technical ones).   What is numpy doing well, and what is it doing
 badly? What risks do we have to plan for in the future?


 joke
 Are you trying to prove the point that consensus doesn't work by making it
 impossible to reach a consensus on this? ;-)
 /joke

 One thing we are doing very badly is leveraging resources outside of
 contributions of work and time from individuals.  Getting sponsors to
 finance work on what is the cornerstone of just about any Python package
 that has to add two numbers together shouldn't be too hard, especially
 seeing success stories like Jupyter's, who I believe has several paid
 developers working full time.  That requires formalizing governance,
 because apparently sponsors are a little wary of giving money to people on
 the internet. ;-)  Fernando Pérez was extremely emphatic about the size of
 the opportunity NumPy was letting slip by not formalizing *any* governance
 model.  And it is a necessary first step so that e.g. we have the money to,
 say a year from now, get the right people together for a couple of days to
 figure out a better governance model.  I'd argue that money would be better
 spent financing a talented developer to advance e.g. Nathaniel's new dtype
 system to end all dtype systems, but that's a different story.

 Largely because of the above, even if Nathaniel's document involved
 tossing a coin to resolve disputes, I'd rather have that now than something
 much better never. Because there really is no alternative to Nathaniel's
 write-up of the status quo, other than the status quo without a write-up:
 it has taken him two months to put this draft together, **after** we agreed
 over several hours of face to face discussion on what the model should be.
 And I'm sure he has hated every minute he has had to put into it.  So if we
 keep going around this in circles, after a few days we will all grow tired
 and go back to fighting over whether indexing should transpose subspaces or
 not, and all that other cool stuff we really enjoy. And a year from now we
 will be in the same place we are now, only a year older 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Sebastian Berg]

On Fr, 2015-08-28 at 09:46 +0100, Matthew Brett wrote:
 Hi,
 
 On Fri, Aug 28, 2015 at 5:59 AM, Jaime Fernández del Río
 jaime.f...@gmail.com wrote:
  On Thu, Aug 27, 2015 at 11:06 AM, Matthew Brett matthew.br...@gmail.com
  wrote:
 
  Hi,
 
  On Thu, Aug 27, 2015 at 6:23 PM,  josef.p...@gmail.com wrote:
  
  
   On Thu, Aug 27, 2015 at 12:22 PM, Matthew Brett
   matthew.br...@gmail.com
   wrote:
  
   Hi
  
   On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:
   
   
On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett
matthew.br...@gmail.com
wrote:
   
Hi,
   
On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
[snip]
 I don't really see a problem with codifying the status quo.
   
That's an excellent point.If we believe that the current
situation
is the best possible, both now and in the future, then codifying the
status quo is an excellent idea.
   
So, we should probably first start by asking ourselves:
   
* what numpy is doing well;
* what numpy could do better;
   
and then ask, is there some way we could make it more likely we will
improve over time.
   
[snip]
   
 As the current debate shows it's possible to have a public
 discussion
 about
 the direction of the project without having to delegate providing
 a
 vision
 to a president.
   
The idea of a president that I had in mind, was not someone who
makes
all decisions, but the person who holds themselves responsible for
the
performance of the project.  If the project has a coherent vision
already, the president has no need to provide one, but it's the
president's job to worry about whether we have vision or not, and do
what they need to, to make sure we don't lose track of that.   If
you
don't know it already, I highly recommend Jim Collins' work on
'level
5 leadership' [1]
   
   
Still doesn't sound like the need for a president to me
   
 the person who holds themselves responsible for the
performance of the project
   
sounds more like the role of the core group (adding plural to
persons)
to
me, and cannot be pushed of to an official president.
  
   Except that, in the past, having multiple people taking decisions has
   led to the situation where no-one feels themselves accountable for the
   result, hence this situation tends to lead to stagnation.
  
  
   Is there any evidence for this?
 
  Oh - dear - that's the key point, but I'm obviously not making it
  clearly enough.  Yes there is, and that was the evidence I was
  pointing to before.
 
  But anyway - Sebastian is right - this discussion isn't going anywhere
  useful.
 
  So - let's step back.
 
  In thinking about governance, we first need to ask what we want to
  achieve.  This includes considering the risks ahead for the project.
 
  So, in the spirit of fruitful discussion, can I ask what y'all
  consider to be the current problems with working on numpy (other than
  the technical ones).   What is numpy doing well, and what is it doing
  badly? What risks do we have to plan for in the future?
 
  joke
  Are you trying to prove the point that consensus doesn't work by making it
  impossible to reach a consensus on this? ;-)
  /joke
 
 Forgive me if I use this joke to see if I can get us any further.
 
 If this was code, I think this joke would not be funny, because we
 wouldn't expect to reach consensus without considering all the
 options, and discussing their pros and cons.
 
 Why would that not be useful in the case of forms of governance?
 

Oh, it is true. I think we (those in the room in Austin) just have
thought about it a bit already, so now we have to be a bit patient with
everyone who just saw the plans the first time. But I hope we can agree
that we should decide on some form of governance in the next few weeks,
even if it may not be perfect.

My personal problem with your ideas is not that I do not care for the
warnings, but having already spend some time trying to put together this
(and this is nothing weird, this is very common practice in open
source), I personally do not want to spend time inventing something
completely new.

We must discuss improvements to the document, and even whole different
approaches. But for me at least, I need something a little more
specific. Maybe I am daft, but I hear this is a bad idea without also
providing another approach (that seems doable).
And I do not buy that it is *that* bad, it is a very common governance
structure for open source. The presidency suggestions may be another
approach and certainly something we can pick up ideas from, but to me it
is so vague that I cannot even start comprehending what it would mean
for the actual governance structure specifically for numpy (considering
the size of the project, etc.).

But by all means, I like proposals/learning from your ideas (i.e. maybe
you can propose changes to the NEP sections), 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Jaime Fernández del Río]

On Fri, Aug 28, 2015 at 2:40 AM, Sebastian Berg sebast...@sipsolutions.net
wrote:

 On Fr, 2015-08-28 at 09:46 +0100, Matthew Brett wrote:
  Hi,
 
  On Fri, Aug 28, 2015 at 5:59 AM, Jaime Fernández del Río
  jaime.f...@gmail.com wrote:
   On Thu, Aug 27, 2015 at 11:06 AM, Matthew Brett 
 matthew.br...@gmail.com
   wrote:
  
   Hi,
  
   On Thu, Aug 27, 2015 at 6:23 PM,  josef.p...@gmail.com wrote:
   
   
On Thu, Aug 27, 2015 at 12:22 PM, Matthew Brett
matthew.br...@gmail.com
wrote:
   
Hi
   
On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:


 On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett
 matthew.br...@gmail.com
 wrote:

 Hi,

 On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
 [snip]
  I don't really see a problem with codifying the status quo.

 That's an excellent point.If we believe that the current
 situation
 is the best possible, both now and in the future, then
 codifying the
 status quo is an excellent idea.

 So, we should probably first start by asking ourselves:

 * what numpy is doing well;
 * what numpy could do better;

 and then ask, is there some way we could make it more likely we
 will
 improve over time.

 [snip]

  As the current debate shows it's possible to have a public
  discussion
  about
  the direction of the project without having to delegate
 providing
  a
  vision
  to a president.

 The idea of a president that I had in mind, was not someone who
 makes
 all decisions, but the person who holds themselves responsible
 for
 the
 performance of the project.  If the project has a coherent
 vision
 already, the president has no need to provide one, but it's the
 president's job to worry about whether we have vision or not,
 and do
 what they need to, to make sure we don't lose track of that.
  If
 you
 don't know it already, I highly recommend Jim Collins' work on
 'level
 5 leadership' [1]


 Still doesn't sound like the need for a president to me

  the person who holds themselves responsible for the
 performance of the project

 sounds more like the role of the core group (adding plural to
 persons)
 to
 me, and cannot be pushed of to an official president.
   
Except that, in the past, having multiple people taking decisions
 has
led to the situation where no-one feels themselves accountable for
 the
result, hence this situation tends to lead to stagnation.
   
   
Is there any evidence for this?
  
   Oh - dear - that's the key point, but I'm obviously not making it
   clearly enough.  Yes there is, and that was the evidence I was
   pointing to before.
  
   But anyway - Sebastian is right - this discussion isn't going anywhere
   useful.
  
   So - let's step back.
  
   In thinking about governance, we first need to ask what we want to
   achieve.  This includes considering the risks ahead for the project.
  
   So, in the spirit of fruitful discussion, can I ask what y'all
   consider to be the current problems with working on numpy (other than
   the technical ones).   What is numpy doing well, and what is it doing
   badly? What risks do we have to plan for in the future?
  
   joke
   Are you trying to prove the point that consensus doesn't work by
 making it
   impossible to reach a consensus on this? ;-)
   /joke
 
  Forgive me if I use this joke to see if I can get us any further.
 
  If this was code, I think this joke would not be funny, because we
  wouldn't expect to reach consensus without considering all the
  options, and discussing their pros and cons.
 
  Why would that not be useful in the case of forms of governance?
 

 Oh, it is true. I think we (those in the room in Austin) just have
 thought about it a bit already, so now we have to be a bit patient with
 everyone who just saw the plans the first time. But I hope we can agree
 that we should decide on some form of governance in the next few weeks,
 even if it may not be perfect.

 My personal problem with your ideas is not that I do not care for the
 warnings, but having already spend some time trying to put together this
 (and this is nothing weird, this is very common practice in open
 source), I personally do not want to spend time inventing something
 completely new.

 We must discuss improvements to the document, and even whole different
 approaches. But for me at least, I need something a little more
 specific. Maybe I am daft, but I hear this is a bad idea without also
 providing another approach (that seems doable).
 And I do not buy that it is *that* bad, it is a very common governance
 structure for open source. The presidency suggestions may be another
 approach and certainly something we can pick up ideas from, but to me it
 is so vague that I cannot even start comprehending what it would mean

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Charles R Harris]

On Fri, Aug 28, 2015 at 3:36 PM, Jaime Fernández del Río 
jaime.f...@gmail.com wrote:

 On Fri, Aug 28, 2015 at 2:40 AM, Sebastian Berg 
 sebast...@sipsolutions.net wrote:

 On Fr, 2015-08-28 at 09:46 +0100, Matthew Brett wrote:
  Hi,
 
  On Fri, Aug 28, 2015 at 5:59 AM, Jaime Fernández del Río
  jaime.f...@gmail.com wrote:
   On Thu, Aug 27, 2015 at 11:06 AM, Matthew Brett 
 matthew.br...@gmail.com
   wrote:
  
   Hi,
  
   On Thu, Aug 27, 2015 at 6:23 PM,  josef.p...@gmail.com wrote:
   
   
On Thu, Aug 27, 2015 at 12:22 PM, Matthew Brett
matthew.br...@gmail.com
wrote:
   
Hi
   
On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:


 On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett
 matthew.br...@gmail.com
 wrote:

 Hi,

 On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com
 wrote:
 [snip]
  I don't really see a problem with codifying the status quo.

 That's an excellent point.If we believe that the current
 situation
 is the best possible, both now and in the future, then
 codifying the
 status quo is an excellent idea.

 So, we should probably first start by asking ourselves:

 * what numpy is doing well;
 * what numpy could do better;

 and then ask, is there some way we could make it more likely
 we will
 improve over time.

 [snip]

  As the current debate shows it's possible to have a public
  discussion
  about
  the direction of the project without having to delegate
 providing
  a
  vision
  to a president.

 The idea of a president that I had in mind, was not someone who
 makes
 all decisions, but the person who holds themselves responsible
 for
 the
 performance of the project.  If the project has a coherent
 vision
 already, the president has no need to provide one, but it's the
 president's job to worry about whether we have vision or not,
 and do
 what they need to, to make sure we don't lose track of that.
  If
 you
 don't know it already, I highly recommend Jim Collins' work on
 'level
 5 leadership' [1]


 Still doesn't sound like the need for a president to me

  the person who holds themselves responsible for the
 performance of the project

 sounds more like the role of the core group (adding plural to
 persons)
 to
 me, and cannot be pushed of to an official president.
   
Except that, in the past, having multiple people taking decisions
 has
led to the situation where no-one feels themselves accountable
 for the
result, hence this situation tends to lead to stagnation.
   
   
Is there any evidence for this?
  
   Oh - dear - that's the key point, but I'm obviously not making it
   clearly enough.  Yes there is, and that was the evidence I was
   pointing to before.
  
   But anyway - Sebastian is right - this discussion isn't going
 anywhere
   useful.
  
   So - let's step back.
  
   In thinking about governance, we first need to ask what we want to
   achieve.  This includes considering the risks ahead for the project.
  
   So, in the spirit of fruitful discussion, can I ask what y'all
   consider to be the current problems with working on numpy (other than
   the technical ones).   What is numpy doing well, and what is it doing
   badly? What risks do we have to plan for in the future?
  
   joke
   Are you trying to prove the point that consensus doesn't work by
 making it
   impossible to reach a consensus on this? ;-)
   /joke
 
  Forgive me if I use this joke to see if I can get us any further.
 
  If this was code, I think this joke would not be funny, because we
  wouldn't expect to reach consensus without considering all the
  options, and discussing their pros and cons.
 
  Why would that not be useful in the case of forms of governance?
 

 Oh, it is true. I think we (those in the room in Austin) just have
 thought about it a bit already, so now we have to be a bit patient with
 everyone who just saw the plans the first time. But I hope we can agree
 that we should decide on some form of governance in the next few weeks,
 even if it may not be perfect.

 My personal problem with your ideas is not that I do not care for the
 warnings, but having already spend some time trying to put together this
 (and this is nothing weird, this is very common practice in open
 source), I personally do not want to spend time inventing something
 completely new.

 We must discuss improvements to the document, and even whole different
 approaches. But for me at least, I need something a little more
 specific. Maybe I am daft, but I hear this is a bad idea without also
 providing another approach (that seems doable).
 And I do not buy that it is *that* bad, it is a very common governance
 structure for open source. The presidency suggestions may be another
 approach and certainly something we can pick up 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[josef.pktd]

On Thu, Aug 27, 2015 at 8:57 AM, Matthew Brett matthew.br...@gmail.com
wrote:

 Hi,

 On Thu, Aug 27, 2015 at 12:11 PM, Sebastian Berg
 sebast...@sipsolutions.net wrote:
  On Do, 2015-08-27 at 10:45 +0100, Matthew Brett wrote:
  Hi,
 
  On Thu, Aug 27, 2015 at 10:35 AM, Bryan Van de Ven bry...@continuum.io
 wrote:
  
   On Aug 27, 2015, at 10:22 AM, Matthew Brett matthew.br...@gmail.com
 wrote:
  
   In the case of the 'core' model, we have some compelling testimony
   from someone with a great deal of experience:
  
   
   Much of this early structure (CVS, web site, cabal [core group],
   etc.) was copied verbatim by other open source (this term not being
 in
   wide use yet) projects -- even the form of the project name and the
   term core. This later became a kind of standard template for
   starting up an open source project. [...] I'm sorry to say that I
   helped create this problem, and that most of the projects which
   modeled themselves after NetBSD (probably due to its high popularity
   in 1993 and 1994) have suffered similar problems. FreeBSD and
 XFree86,
   for example, have both forked successor projects (Dragonfly and
 X.org)
   for very similar reasons.
   
  
   Who goes on to propose:
  
   7) The core group must be replaced with people who are actually
  competent and dedicated enough to review proposals, accept
 feedback,
  and make good decisions.  More to the point, though, the core
 group
  must only act when *needed* -- most technical decisions should be
  left to the community to hash out; it must not preempt the
 community
  from developing better solutions.  (This is how the core group
  worked during most of the project's growth period.)
 
  Sure.  I think it's reasonable to give high weight to Hannum's
  assessment of the failure of the core group, but less weight to his
  proposal for a replacement, because at the time, I don't believe he
  was in a good position to assess whether his (apparent) alternative
  would run into the same trouble.
 
  It's always tempting to blame the people rather than the system, but
  in this case, I strongly suspect that it was the system that was
  fundamentally flawed, therefore either promoting the wrong people or
  putting otherwise competent people into situations where they are no
  longer getting useful feedback.
 
  Maybe so. I do not know much at all about these models, but I am not
  sure how much applies here to numpy. Isn't at least FreeBSD a magnitude
  larger then numpy?

 It seems to me that numpy suffers from the same risks of poor
 accountability, stagnation and conservatism that larger projects do.
 Is there a reason that would not be the case?

  We do need to have some formality about how to give out commit rights,
  and do final decision when all else fails.

 Yes, sure, something formal is probably but not certainly better than
 nothing, depending on what the 'something formal' is.

  One thing I do not know is how a community vote could work at all,
  considering I do not even know how to count its members. Votes and
  presidents make sense to me for large projects with hundrets of
  developers on different corners (think of the gnome foundation, debian
  probably) [1].

 The 'president' idea is to get at the problem of lack of
 accountability, along with selection for leadership skill rather than
 coding ability.   It's trying to get at the advantages of the BDFL
 model in our situation where there is no obvious BDFL.For the me
 the problem is that, at the moment, if the formal or informal
 governing body makes a bad decision, then no member will feel
 responsible for that decision or its consequences.  That tends to lead
 to an atmosphere of - oh well, what could we do, X wouldn't agree to
 A and Y wouldn't agree to B so we're stuck.   It seems to me we need
 a system such that whoever is in charge feels so strongly that it is
 their job to make numpy as good as possible, that they will take
 whatever difficult or sensitive decisions are necessary to make that
 happen.  On the other hand the 'core' system seems to function on a
 model of mutual deference and personal loyalty that I believe is
 destructive of good management.



I don't really see a problem with codifying the status quo.

It might become necessary to have something like an administrative director
if numpy becomes a more formal organization with funding, but for the
development of the project I don't see any need for a president.
If there is no obvious BDFL, then I guess there is also no obvious
president. (I would vote for Ralf as president of everything, but I don't
think he's available.)

As the current debate shows it's possible to have a public discussion about
the direction of the project without having to delegate providing a vision
to a president.

Given the current pattern all critical issues end up in a public debate on
the mailing list. What numpy (and scipy) need is to have someone as a tie
breaker to make any 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Bryan Van de Ven]

 On Aug 27, 2015, at 1:57 PM, Matthew Brett matthew.br...@gmail.com wrote:
 
 The 'president' idea 

...seems to be predicated on a steady stream of people who: actually want job, 
don't mind campaigning, are willing to accept any and all blame, and have the 
technical experience to make final decisions. As others have pointed out the 
active developer community for NumPy is not measured in the hundreds (or even 
the tens, really). So: what is your proposed recourse if you hold an election 
and no-one shows up to run?

Bryan 
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Thu, Aug 27, 2015 at 12:11 PM, Sebastian Berg
sebast...@sipsolutions.net wrote:
 On Do, 2015-08-27 at 10:45 +0100, Matthew Brett wrote:
 Hi,

 On Thu, Aug 27, 2015 at 10:35 AM, Bryan Van de Ven bry...@continuum.io 
 wrote:
 
  On Aug 27, 2015, at 10:22 AM, Matthew Brett matthew.br...@gmail.com 
  wrote:
 
  In the case of the 'core' model, we have some compelling testimony
  from someone with a great deal of experience:
 
  
  Much of this early structure (CVS, web site, cabal [core group],
  etc.) was copied verbatim by other open source (this term not being in
  wide use yet) projects -- even the form of the project name and the
  term core. This later became a kind of standard template for
  starting up an open source project. [...] I'm sorry to say that I
  helped create this problem, and that most of the projects which
  modeled themselves after NetBSD (probably due to its high popularity
  in 1993 and 1994) have suffered similar problems. FreeBSD and XFree86,
  for example, have both forked successor projects (Dragonfly and X.org)
  for very similar reasons.
  
 
  Who goes on to propose:
 
  7) The core group must be replaced with people who are actually
 competent and dedicated enough to review proposals, accept feedback,
 and make good decisions.  More to the point, though, the core group
 must only act when *needed* -- most technical decisions should be
 left to the community to hash out; it must not preempt the community
 from developing better solutions.  (This is how the core group
 worked during most of the project's growth period.)

 Sure.  I think it's reasonable to give high weight to Hannum's
 assessment of the failure of the core group, but less weight to his
 proposal for a replacement, because at the time, I don't believe he
 was in a good position to assess whether his (apparent) alternative
 would run into the same trouble.

 It's always tempting to blame the people rather than the system, but
 in this case, I strongly suspect that it was the system that was
 fundamentally flawed, therefore either promoting the wrong people or
 putting otherwise competent people into situations where they are no
 longer getting useful feedback.

 Maybe so. I do not know much at all about these models, but I am not
 sure how much applies here to numpy. Isn't at least FreeBSD a magnitude
 larger then numpy?

It seems to me that numpy suffers from the same risks of poor
accountability, stagnation and conservatism that larger projects do.
Is there a reason that would not be the case?

 We do need to have some formality about how to give out commit rights,
 and do final decision when all else fails.

Yes, sure, something formal is probably but not certainly better than
nothing, depending on what the 'something formal' is.

 One thing I do not know is how a community vote could work at all,
 considering I do not even know how to count its members. Votes and
 presidents make sense to me for large projects with hundrets of
 developers on different corners (think of the gnome foundation, debian
 probably) [1].

The 'president' idea is to get at the problem of lack of
accountability, along with selection for leadership skill rather than
coding ability.   It's trying to get at the advantages of the BDFL
model in our situation where there is no obvious BDFL.For the me
the problem is that, at the moment, if the formal or informal
governing body makes a bad decision, then no member will feel
responsible for that decision or its consequences.  That tends to lead
to an atmosphere of - oh well, what could we do, X wouldn't agree to
A and Y wouldn't agree to B so we're stuck.   It seems to me we need
a system such that whoever is in charge feels so strongly that it is
their job to make numpy as good as possible, that they will take
whatever difficult or sensitive decisions are necessary to make that
happen.  On the other hand the 'core' system seems to function on a
model of mutual deference and personal loyalty that I believe is
destructive of good management.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Sebastian Berg]

On Do, 2015-08-27 at 17:22 +0100, Matthew Brett wrote:
 Hi
 
 On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:
 
 
  On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett matthew.br...@gmail.com
  wrote:
 
  Hi,
 
  On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
  [snip]
   I don't really see a problem with codifying the status quo.
 
  That's an excellent point.If we believe that the current situation
  is the best possible, both now and in the future, then codifying the
  status quo is an excellent idea.
 
  So, we should probably first start by asking ourselves:
 
  * what numpy is doing well;
  * what numpy could do better;
 
  and then ask, is there some way we could make it more likely we will
  improve over time.
 
  [snip]
 
   As the current debate shows it's possible to have a public discussion
   about
   the direction of the project without having to delegate providing a
   vision
   to a president.
 
  The idea of a president that I had in mind, was not someone who makes
  all decisions, but the person who holds themselves responsible for the
  performance of the project.  If the project has a coherent vision
  already, the president has no need to provide one, but it's the
  president's job to worry about whether we have vision or not, and do
  what they need to, to make sure we don't lose track of that.   If you
  don't know it already, I highly recommend Jim Collins' work on 'level
  5 leadership' [1]
 
 
  Still doesn't sound like the need for a president to me
 
   the person who holds themselves responsible for the
  performance of the project
 
  sounds more like the role of the core group (adding plural to persons) to
  me, and cannot be pushed of to an official president.
 
 Except that, in the past, having multiple people taking decisions has
 led to the situation where no-one feels themselves accountable for the
 result, hence this situation tends to lead to stagnation.

Frankly, I am failing to see the direction of these arguments. One thing
to remember, that a core group is much like a BDFL/president with
multiple personalities ;), and a core group is not a fixed Oligarchy.
Anyone able and willing should be in it and the governance document is
clear about that I think (of course nothing is perfect, but we can try).

There is the thing of how. I simply fail to see how the president can
even be defined considering the size of the numpy development team (say
10, most of whom are busy with other things most of the time).

Also, I fail to see how the president would be any more useful then the
agreement of some tasks being handled by some people who are
enthusiastic about them (note those do not even have to be in the core
group for starters, though they should become part of it quickly).

This is a community effort and I am starting to feel that the ideas you
are giving are from a different management/company context.
The goal of the governance is to show how and hopefully make it easy for
*anyone* to provide vision.
We do not need a manager who decides how to focus allocate resources,
instead we must tell everyone that we are happy about any help we can
get, and that anyone can pick up a topic they are enthusiastic about and
drive numpy ahead.

And considering accountability, that help may well amount in saying:
Do NOT do this.

A president willing to run for such an election, should have a
specific vision? Why should they be special to implement it?
Note this is also the case in BDFL organizations. If you have a vision
to improve python, it does not really matter if you happen to be Guido.

You write a PEP and, if people like it, it will be implemented.

At the same time we *must* have a well defined form of governance also
for organizational things. Right now we cannot even decide on putting
someone in charge of overseeing our NumFOCUS donations. NumPy could not
even spend its own money!

Sorry, getting way too long :(

- Sebastian


  Nathaniel to push and organize the discussion, Chuck for continuity, and
  several core developers for detailed ideas and implementation, and a large
  number of contributors. (stylized roles)
  and noisy mailing list for feedback and discussion.
 
  Given the size of the numpy development group, numpy needs individuals for
  the vision and to push things not a president, vice-presidents and assistant
  vice-presidents, IMO.
 
 Yes, if the roles were honorary and administrative, they would not be useful.
 
 Cheers,
 
 Matthew
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion
 



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[josef.pktd]

On Thu, Aug 27, 2015 at 12:22 PM, Matthew Brett matthew.br...@gmail.com
wrote:

 Hi

 On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:
 
 
  On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett matthew.br...@gmail.com
 
  wrote:
 
  Hi,
 
  On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
  [snip]
   I don't really see a problem with codifying the status quo.
 
  That's an excellent point.If we believe that the current situation
  is the best possible, both now and in the future, then codifying the
  status quo is an excellent idea.
 
  So, we should probably first start by asking ourselves:
 
  * what numpy is doing well;
  * what numpy could do better;
 
  and then ask, is there some way we could make it more likely we will
  improve over time.
 
  [snip]
 
   As the current debate shows it's possible to have a public discussion
   about
   the direction of the project without having to delegate providing a
   vision
   to a president.
 
  The idea of a president that I had in mind, was not someone who makes
  all decisions, but the person who holds themselves responsible for the
  performance of the project.  If the project has a coherent vision
  already, the president has no need to provide one, but it's the
  president's job to worry about whether we have vision or not, and do
  what they need to, to make sure we don't lose track of that.   If you
  don't know it already, I highly recommend Jim Collins' work on 'level
  5 leadership' [1]
 
 
  Still doesn't sound like the need for a president to me
 
   the person who holds themselves responsible for the
  performance of the project
 
  sounds more like the role of the core group (adding plural to persons)
 to
  me, and cannot be pushed of to an official president.

 Except that, in the past, having multiple people taking decisions has
 led to the situation where no-one feels themselves accountable for the
 result, hence this situation tends to lead to stagnation.


Is there any evidence for this?

First, it's several individuals taking joint decisions, jointly agree or
not object (LGTM) to merging PRs, it's still a joint decision and
accountability is not exclusive. The PR review process makes decisions much
more into a joint decision process than it was with isolated SVN commits.
(*)

Second, if there are separated decisions, then it could also lead to excess
change. All these enthusiastic new developers bringing in whatever they
(and the local chief) like, and nobody to stop them.

In either case, the developer, or local chief, has to deal with the
consequences. You merged this PR, now fix it. or Why are you holding up
this PR? I can merge it.

(*) Even though I'm not a scipy developer anymore, I still feel partially
responsible for it, I'm still reviewing some PRs and comment on them,
sometimes as cheerleader in favor of something or sometimes pointing out
problems, or just checking that it makes sense, and always with an eye on
what downstream impact it might have.

Another thought: Having an accountable president might actually reduce the
feeling of accountability and responsibility of individual developers, so
the neteffect is negative.
The president is responsible for this (even though he doesn't have enough
time), so I can skip part of this review.

Josef




  Nathaniel to push and organize the discussion, Chuck for continuity, and
  several core developers for detailed ideas and implementation, and a
 large
  number of contributors. (stylized roles)
  and noisy mailing list for feedback and discussion.
 
  Given the size of the numpy development group, numpy needs individuals
 for
  the vision and to push things not a president, vice-presidents and
 assistant
  vice-presidents, IMO.

 Yes, if the roles were honorary and administrative, they would not be
 useful.


I'm not sure what you mean here. Given that it's all volunteer work, any
president wouldn't have any hard tools.

Josef




 Cheers,

 Matthew
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi

On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:


 On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett matthew.br...@gmail.com
 wrote:

 Hi,

 On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
 [snip]
  I don't really see a problem with codifying the status quo.

 That's an excellent point.If we believe that the current situation
 is the best possible, both now and in the future, then codifying the
 status quo is an excellent idea.

 So, we should probably first start by asking ourselves:

 * what numpy is doing well;
 * what numpy could do better;

 and then ask, is there some way we could make it more likely we will
 improve over time.

 [snip]

  As the current debate shows it's possible to have a public discussion
  about
  the direction of the project without having to delegate providing a
  vision
  to a president.

 The idea of a president that I had in mind, was not someone who makes
 all decisions, but the person who holds themselves responsible for the
 performance of the project.  If the project has a coherent vision
 already, the president has no need to provide one, but it's the
 president's job to worry about whether we have vision or not, and do
 what they need to, to make sure we don't lose track of that.   If you
 don't know it already, I highly recommend Jim Collins' work on 'level
 5 leadership' [1]


 Still doesn't sound like the need for a president to me

  the person who holds themselves responsible for the
 performance of the project

 sounds more like the role of the core group (adding plural to persons) to
 me, and cannot be pushed of to an official president.

Except that, in the past, having multiple people taking decisions has
led to the situation where no-one feels themselves accountable for the
result, hence this situation tends to lead to stagnation.

 Nathaniel to push and organize the discussion, Chuck for continuity, and
 several core developers for detailed ideas and implementation, and a large
 number of contributors. (stylized roles)
 and noisy mailing list for feedback and discussion.

 Given the size of the numpy development group, numpy needs individuals for
 the vision and to push things not a president, vice-presidents and assistant
 vice-presidents, IMO.

Yes, if the roles were honorary and administrative, they would not be useful.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[josef.pktd]

On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett matthew.br...@gmail.com
wrote:

 Hi,

 On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
 [snip]
  I don't really see a problem with codifying the status quo.

 That's an excellent point.If we believe that the current situation
 is the best possible, both now and in the future, then codifying the
 status quo is an excellent idea.

 So, we should probably first start by asking ourselves:

 * what numpy is doing well;
 * what numpy could do better;

 and then ask, is there some way we could make it more likely we will
 improve over time.

 [snip]

  As the current debate shows it's possible to have a public discussion
 about
  the direction of the project without having to delegate providing a
 vision
  to a president.

 The idea of a president that I had in mind, was not someone who makes
 all decisions, but the person who holds themselves responsible for the
 performance of the project.  If the project has a coherent vision
 already, the president has no need to provide one, but it's the
 president's job to worry about whether we have vision or not, and do
 what they need to, to make sure we don't lose track of that.   If you
 don't know it already, I highly recommend Jim Collins' work on 'level
 5 leadership' [1]


Still doesn't sound like the need for a president to me

 the person who holds themselves responsible for the
performance of the project

sounds more like the role of the core group (adding plural to persons) to
me, and cannot be pushed of to an official president.

Nathaniel to push and organize the discussion, Chuck for continuity, and
several core developers for detailed ideas and implementation, and a large
number of contributors. (stylized roles)
and noisy mailing list for feedback and discussion.

Given the size of the numpy development group, numpy needs individuals for
the vision and to push things not a president, vice-presidents and
assistant vice-presidents, IMO.

(Given the importance of numpy itself, there should be enough remedies if
the core group ever gets `out of touch` with the very large user base.)

Josef



  Given the current pattern all critical issues end up in a public debate
 on
  the mailing list. What numpy (and scipy) need is to have someone as a tie
  breaker to make any final decisions if there is no clear consensus, if
 there
  is no BDFL for it, then having the core group making those decisions
 looks
  appropriate to me.
 
  On the other hand the 'core' system seems to function on a
  model of mutual deference and personal loyalty that I believe is
  destructive of good management.
 
  That sounds actually like a good basis for team work to me. Also that has
  mutual in it instead of just deferring and being loyal to a president.
 
  Since I know scipy development much better:
 
  scipy has made a huge progress in the last 5 or 6 years since I've been
  following it, both in terms of code, in terms of development workflow,
 and
  in the number of developers. (When I started, I was essentially alone in
  scipy.stats, now there are 3 to 5 core developers that at least
 partially
  work on it, everything goes through PRs with public discussion and with
  critical issues additionally raised on the mailing list.)
 
  Ralf and Pauli are the defacto BDFLs for scipy overall, but all
 decisions in
  recent years have been without a fight, but not without lots of
 arguments,
  and, given the size and breadth of scipy, there are field experts
 (although
  not enough of those) to help in the discussion.

 I agree entirely, I think scipy is a good example where stability and
 clarity of leadership has made a huge difference to the health of the
 project.

 Cheers,

 Matthew

 [1]
 https://hbr.org/2005/07/level-5-leadership-the-triumph-of-humility-and-fierce-resolve
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
[snip]
 I don't really see a problem with codifying the status quo.

That's an excellent point.If we believe that the current situation
is the best possible, both now and in the future, then codifying the
status quo is an excellent idea.

So, we should probably first start by asking ourselves:

* what numpy is doing well;
* what numpy could do better;

and then ask, is there some way we could make it more likely we will
improve over time.

[snip]

 As the current debate shows it's possible to have a public discussion about
 the direction of the project without having to delegate providing a vision
 to a president.

The idea of a president that I had in mind, was not someone who makes
all decisions, but the person who holds themselves responsible for the
performance of the project.  If the project has a coherent vision
already, the president has no need to provide one, but it's the
president's job to worry about whether we have vision or not, and do
what they need to, to make sure we don't lose track of that.   If you
don't know it already, I highly recommend Jim Collins' work on 'level
5 leadership' [1]

 Given the current pattern all critical issues end up in a public debate on
 the mailing list. What numpy (and scipy) need is to have someone as a tie
 breaker to make any final decisions if there is no clear consensus, if there
 is no BDFL for it, then having the core group making those decisions looks
 appropriate to me.

 On the other hand the 'core' system seems to function on a
 model of mutual deference and personal loyalty that I believe is
 destructive of good management.

 That sounds actually like a good basis for team work to me. Also that has
 mutual in it instead of just deferring and being loyal to a president.

 Since I know scipy development much better:

 scipy has made a huge progress in the last 5 or 6 years since I've been
 following it, both in terms of code, in terms of development workflow, and
 in the number of developers. (When I started, I was essentially alone in
 scipy.stats, now there are 3 to 5 core developers that at least partially
 work on it, everything goes through PRs with public discussion and with
 critical issues additionally raised on the mailing list.)

 Ralf and Pauli are the defacto BDFLs for scipy overall, but all decisions in
 recent years have been without a fight, but not without lots of arguments,
 and, given the size and breadth of scipy, there are field experts (although
 not enough of those) to help in the discussion.

I agree entirely, I think scipy is a good example where stability and
clarity of leadership has made a huge difference to the health of the
project.

Cheers,

Matthew

[1] 
https://hbr.org/2005/07/level-5-leadership-the-triumph-of-humility-and-fierce-resolve
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Thu, Aug 27, 2015 at 3:43 PM, Bryan Van de Ven bry...@continuum.io wrote:

 On Aug 27, 2015, at 1:57 PM, Matthew Brett matthew.br...@gmail.com wrote:

 The 'president' idea

 ...seems to be predicated on a steady stream of people who: actually want 
 job, don't mind campaigning, are willing to accept any and all blame, and 
 have the technical experience to make final decisions. As others have 
 pointed out the active developer community for NumPy is not measured in the 
 hundreds (or even the tens, really). So: what is your proposed recourse if 
 you hold an election and no-one shows up to run?


That seems to me a soluble problem, if there's agreement that the
president idea is a sensible one.  One very simple idea would be to
revert to a 'core' system for the term for which there were no
candidates.

On the other hand, I suspect that there are people who care enough
about numpy that they are prepared to step up and take the blame if
things go wrong.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[josef.pktd]

On Thu, Aug 27, 2015 at 2:06 PM, Matthew Brett matthew.br...@gmail.com
wrote:

 Hi,

 On Thu, Aug 27, 2015 at 6:23 PM,  josef.p...@gmail.com wrote:
 
 
  On Thu, Aug 27, 2015 at 12:22 PM, Matthew Brett matthew.br...@gmail.com
 
  wrote:
 
  Hi
 
  On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:
  
  
   On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett
   matthew.br...@gmail.com
   wrote:
  
   Hi,
  
   On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
   [snip]
I don't really see a problem with codifying the status quo.
  
   That's an excellent point.If we believe that the current
 situation
   is the best possible, both now and in the future, then codifying the
   status quo is an excellent idea.
  
   So, we should probably first start by asking ourselves:
  
   * what numpy is doing well;
   * what numpy could do better;
  
   and then ask, is there some way we could make it more likely we will
   improve over time.
  
   [snip]
  
As the current debate shows it's possible to have a public
 discussion
about
the direction of the project without having to delegate providing a
vision
to a president.
  
   The idea of a president that I had in mind, was not someone who makes
   all decisions, but the person who holds themselves responsible for
 the
   performance of the project.  If the project has a coherent vision
   already, the president has no need to provide one, but it's the
   president's job to worry about whether we have vision or not, and do
   what they need to, to make sure we don't lose track of that.   If you
   don't know it already, I highly recommend Jim Collins' work on 'level
   5 leadership' [1]
  
  
   Still doesn't sound like the need for a president to me
  
the person who holds themselves responsible for the
   performance of the project
  
   sounds more like the role of the core group (adding plural to
 persons)
   to
   me, and cannot be pushed of to an official president.
 
  Except that, in the past, having multiple people taking decisions has
  led to the situation where no-one feels themselves accountable for the
  result, hence this situation tends to lead to stagnation.
 
 
  Is there any evidence for this?

 Oh - dear - that's the key point, but I'm obviously not making it
 clearly enough.  Yes there is, and that was the evidence I was
 pointing to before.


If you mean the XFree and NetBSD cases, then I don't see any similarity to
the numpy or scipy development pattern. If I would draw any conclusion,
then maybe that NetBSD hat too much of formal governance structures and not
enough informal governance. It would be difficult to take over the
government if there is no government.

just one aside
*No desire to recruit new users  *

We are on a mission to take over the world (*). And forks of numpy like
pandas turn out to be mostly complementary and increase the userbase of
numpy.

(R and Python are in friendly, or sometimes unfriendly, competition, but,
AFAICS, we are both gaining users because of the others' presence. It's not
a zero sum game in this case.)

(*) But that's not in the mission statement.




 But anyway - Sebastian is right - this discussion isn't going anywhere
 useful.

 So - let's step back.

 In thinking about governance, we first need to ask what we want to
 achieve.  This includes considering the risks ahead for the project.

 So, in the spirit of fruitful discussion, can I ask what y'all
 consider to be the current problems with working on numpy (other than
 the technical ones).   What is numpy doing well, and what is it doing
 badly? What risks do we have to plan for in the future?


I thought that was implicit or explicit in the other thread.

Josef




 Cheers,

 Matthew
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Stefan van der Walt]

On 2015-08-27 11:06:10, Matthew Brett matthew.br...@gmail.com 
wrote:
 So, in the spirit of fruitful discussion, can I ask what y'all 
 consider to be the current problems with working on numpy (other 
 than the technical ones).   What is numpy doing well, and what 
 is it doing badly? What risks do we have to plan for in the 
 future?

It looks to me as though the team is doing an excellent job of 
maintaining NumPy.  The growth of the project has stagnated 
somewhat for numerous reasons---and a lack of ideas on the table 
is not one of them, rather whether / how to take them forward.

The question, and I think what you also highlighted in the earlier 
part of this discussion, is: how to decide on which vision to 
adopt, and who takes responsibility for making that happen?

Are the two models proposed thus far so different, or can they be 
merged in a way that makes sense?  E.g., can we work as a 
community to rally behind a vision as set out by one person, and 
then repeat that process to focus on another a year later?  Think 
of it as the iterative development equivalent of governance.

This may just be another way of phrasing a precedency, but with a 
strong emphasis on its temporary nature, as well as a focus on a 
group-decided outcome.  Alternatively, see it as a community 
governance model with a strong emphasis on responsibility.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Sebastian Berg]

On Do, 2015-08-27 at 12:34 -0700, Stefan van der Walt wrote:
 On 2015-08-27 11:06:10, Matthew Brett matthew.br...@gmail.com 
 wrote:
  So, in the spirit of fruitful discussion, can I ask what y'all 
  consider to be the current problems with working on numpy (other 
  than the technical ones).   What is numpy doing well, and what 
  is it doing badly? What risks do we have to plan for in the 
  future?
 
 It looks to me as though the team is doing an excellent job of 
 maintaining NumPy.  The growth of the project has stagnated 
 somewhat for numerous reasons---and a lack of ideas on the table 
 is not one of them, rather whether / how to take them forward.
 
 The question, and I think what you also highlighted in the earlier 
 part of this discussion, is: how to decide on which vision to 
 adopt, and who takes responsibility for making that happen?
 
 Are the two models proposed thus far so different, or can they be 
 merged in a way that makes sense?  E.g., can we work as a 
 community to rally behind a vision as set out by one person, and 
 then repeat that process to focus on another a year later?  Think 
 of it as the iterative development equivalent of governance.
 
 This may just be another way of phrasing a precedency, but with a 
 strong emphasis on its temporary nature, as well as a focus on a 
 group-decided outcome.  Alternatively, see it as a community 
 governance model with a strong emphasis on responsibility.
 

Agreed. Are not PEP's/NEP's just that (and could possibly be formalized
more, not sure how much they are in the current proposal) in some sense?
Since they have a sponsor/author who can be said to be assigned to
it/responsible once accepted.

I will add one more thing which I think is important:

The governance has to be create as little hassle as possible and it
should be simple/short enough to quickly understand.

- Sebastian


 Stéfan
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Jaime Fernández del Río]

On Thu, Aug 27, 2015 at 11:06 AM, Matthew Brett matthew.br...@gmail.com
wrote:

 Hi,

 On Thu, Aug 27, 2015 at 6:23 PM,  josef.p...@gmail.com wrote:
 
 
  On Thu, Aug 27, 2015 at 12:22 PM, Matthew Brett matthew.br...@gmail.com
 
  wrote:
 
  Hi
 
  On Thu, Aug 27, 2015 at 5:11 PM,  josef.p...@gmail.com wrote:
  
  
   On Thu, Aug 27, 2015 at 11:04 AM, Matthew Brett
   matthew.br...@gmail.com
   wrote:
  
   Hi,
  
   On Thu, Aug 27, 2015 at 3:34 PM,  josef.p...@gmail.com wrote:
   [snip]
I don't really see a problem with codifying the status quo.
  
   That's an excellent point.If we believe that the current
 situation
   is the best possible, both now and in the future, then codifying the
   status quo is an excellent idea.
  
   So, we should probably first start by asking ourselves:
  
   * what numpy is doing well;
   * what numpy could do better;
  
   and then ask, is there some way we could make it more likely we will
   improve over time.
  
   [snip]
  
As the current debate shows it's possible to have a public
 discussion
about
the direction of the project without having to delegate providing a
vision
to a president.
  
   The idea of a president that I had in mind, was not someone who makes
   all decisions, but the person who holds themselves responsible for
 the
   performance of the project.  If the project has a coherent vision
   already, the president has no need to provide one, but it's the
   president's job to worry about whether we have vision or not, and do
   what they need to, to make sure we don't lose track of that.   If you
   don't know it already, I highly recommend Jim Collins' work on 'level
   5 leadership' [1]
  
  
   Still doesn't sound like the need for a president to me
  
the person who holds themselves responsible for the
   performance of the project
  
   sounds more like the role of the core group (adding plural to
 persons)
   to
   me, and cannot be pushed of to an official president.
 
  Except that, in the past, having multiple people taking decisions has
  led to the situation where no-one feels themselves accountable for the
  result, hence this situation tends to lead to stagnation.
 
 
  Is there any evidence for this?

 Oh - dear - that's the key point, but I'm obviously not making it
 clearly enough.  Yes there is, and that was the evidence I was
 pointing to before.

 But anyway - Sebastian is right - this discussion isn't going anywhere
 useful.

 So - let's step back.

 In thinking about governance, we first need to ask what we want to
 achieve.  This includes considering the risks ahead for the project.

 So, in the spirit of fruitful discussion, can I ask what y'all
 consider to be the current problems with working on numpy (other than
 the technical ones).   What is numpy doing well, and what is it doing
 badly? What risks do we have to plan for in the future?


joke
Are you trying to prove the point that consensus doesn't work by making it
impossible to reach a consensus on this? ;-)
/joke

One thing we are doing very badly is leveraging resources outside of
contributions of work and time from individuals.  Getting sponsors to
finance work on what is the cornerstone of just about any Python package
that has to add two numbers together shouldn't be too hard, especially
seeing success stories like Jupyter's, who I believe has several paid
developers working full time.  That requires formalizing governance,
because apparently sponsors are a little wary of giving money to people on
the internet. ;-)  Fernando Pérez was extremely emphatic about the size of
the opportunity NumPy was letting slip by not formalizing *any* governance
model.  And it is a necessary first step so that e.g. we have the money to,
say a year from now, get the right people together for a couple of days to
figure out a better governance model.  I'd argue that money would be better
spent financing a talented developer to advance e.g. Nathaniel's new dtype
system to end all dtype systems, but that's a different story.

Largely because of the above, even if Nathaniel's document involved tossing
a coin to resolve disputes, I'd rather have that now than something much
better never. Because there really is no alternative to Nathaniel's
write-up of the status quo, other than the status quo without a write-up:
it has taken him two months to put this draft together, **after** we agreed
over several hours of face to face discussion on what the model should be.
And I'm sure he has hated every minute he has had to put into it.  So if we
keep going around this in circles, after a few days we will all grow tired
and go back to fighting over whether indexing should transpose subspaces or
not, and all that other cool stuff we really enjoy. And a year from now we
will be in the same place we are now, only a year older and deeper in
(technical) debt.

Jaime

-- 
(\__/)
( O.o)
(  ) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación 

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Thu, Aug 27, 2015 at 10:35 AM, Bryan Van de Ven bry...@continuum.io wrote:

 On Aug 27, 2015, at 10:22 AM, Matthew Brett matthew.br...@gmail.com wrote:

 In the case of the 'core' model, we have some compelling testimony
 from someone with a great deal of experience:

 
 Much of this early structure (CVS, web site, cabal [core group],
 etc.) was copied verbatim by other open source (this term not being in
 wide use yet) projects -- even the form of the project name and the
 term core. This later became a kind of standard template for
 starting up an open source project. [...] I'm sorry to say that I
 helped create this problem, and that most of the projects which
 modeled themselves after NetBSD (probably due to its high popularity
 in 1993 and 1994) have suffered similar problems. FreeBSD and XFree86,
 for example, have both forked successor projects (Dragonfly and X.org)
 for very similar reasons.
 

 Who goes on to propose:

 7) The core group must be replaced with people who are actually
competent and dedicated enough to review proposals, accept feedback,
and make good decisions.  More to the point, though, the core group
must only act when *needed* -- most technical decisions should be
left to the community to hash out; it must not preempt the community
from developing better solutions.  (This is how the core group
worked during most of the project's growth period.)

Sure.  I think it's reasonable to give high weight to Hannum's
assessment of the failure of the core group, but less weight to his
proposal for a replacement, because at the time, I don't believe he
was in a good position to assess whether his (apparent) alternative
would run into the same trouble.

It's always tempting to blame the people rather than the system, but
in this case, I strongly suspect that it was the system that was
fundamentally flawed, therefore either promoting the wrong people or
putting otherwise competent people into situations where they are no
longer getting useful feedback.

It would be great, and very convenient, if the only management we
needed was getting out of the way, but I doubt very much that that is
the case.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Nathaniel Smith]

On Thu, Aug 27, 2015 at 2:16 AM, Matthew Brett matthew.br...@gmail.com wrote:
 So, I speculate, that a good governance model would have:

 * one 'president' who has to take final responsibility for all decisions;
 * this president might well have a fixed term, maybe with limits on
 the number of terms they can serve.
 * the president would be chosen by community vote and explicitly on
 the basis that they were good managers as well as coders;
 * for the presidential election, the candidates should set out what
 their vision for the project is, and how they plan to achieve that
 vision;

We actually discussed some variants on this kind of idea at the
meeting, and I think the general sense of those present was we didn't
want to go there (for whatever that's worth).

At least personally, I have to admit that the idea of a governance
model involving elections fills me with creeping horror. The reason is
that whole point of having a governance model (IMHO) is to (a)
minimize the rise of interpersonal drama, (b) when some amount of
interpersonal drama does inevitably arise anyway, provide some
regulated channel for it, hopefully one that leads to a drama sink.
But elections are a huge massive drama source. No-one wants to spend
time campaigning or wondering how some technical proposal will effect
their re-election chances, we want to get this sorted out so that we
can stop thinking about it and go back to solving actually interesting
problems...

As for evidence... there are obviously projects that have had serious
problems with some variant of core team model, but there are also many
many successful projects that are also using variants of this model,
and the document I sent around attempts to incorporate the lessons
that have been learned in the process. OTOH after wracking my brain I
think the only project I'm familiar with that has elections at all
like this is Fedora, which elects... a core team (FESCo). Given that
we don't have the problem of trying to manage thousands of
contributors, I'm not sure their experience is really relevant. Or I
guess Debian's use of General Resolutions as a decision-making
procedure of last resort is kinda relevant, but... pretty different.
(They also elect the project leader, which is more similar to what
you're describing, but the project leader has no technical authority;
in Debian the final authority short of a GR is the CTTE, which is
explicitly designed as a classic beholden-to-nobody institution -- and
even overriding the CTTE requires a supermajority.)

I kinda feel like... as a rule of thumb, if your description of your
governance model starts with the words I speculate that..., then
NumPy is probably not a good project to use for your experiment?

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Thu, Aug 27, 2015 at 11:05 AM, Nathaniel Smith n...@pobox.com wrote:
 On Thu, Aug 27, 2015 at 2:16 AM, Matthew Brett matthew.br...@gmail.com 
 wrote:
 So, I speculate, that a good governance model would have:

 * one 'president' who has to take final responsibility for all decisions;
 * this president might well have a fixed term, maybe with limits on
 the number of terms they can serve.
 * the president would be chosen by community vote and explicitly on
 the basis that they were good managers as well as coders;
 * for the presidential election, the candidates should set out what
 their vision for the project is, and how they plan to achieve that
 vision;

 We actually discussed some variants on this kind of idea at the
 meeting, and I think the general sense of those present was we didn't
 want to go there (for whatever that's worth).

 At least personally, I have to admit that the idea of a governance
 model involving elections fills me with creeping horror. The reason is
 that whole point of having a governance model (IMHO) is to (a)
 minimize the rise of interpersonal drama

Right - I think this is key to the problem in this model.  It is
designed not to cause any trouble, and to keep things running as they
are without controversy.  It works OK on average as long as 'no
change' is the desired outcome.  In general the core group know each
other fairly well, and feel a sense of shared loyalty to the group.
This loyalty is exercised when some outside or inside force challenges
the direction of the project.  This was what made it so hard for the
XFree86 core group to pull back from the course they had set.

The question is - is avoiding the potential controversy important
enough to force us into a model that has (in my opinion) a high risk
of tending to conservatism and stagnation?

 As for evidence... there are obviously projects that have had serious
 problems with some variant of core team model, but there are also many
 many successful projects that are also using variants of this model,
 and the document I sent around attempts to incorporate the lessons
 that have been learned in the process. OTOH after wracking my brain I
 think the only project I'm familiar with that has elections at all
 like this is Fedora, which elects... a core team (FESCo). Given that
 we don't have the problem of trying to manage thousands of
 contributors, I'm not sure their experience is really relevant. Or I
 guess Debian's use of General Resolutions as a decision-making
 procedure of last resort is kinda relevant, but... pretty different.
 (They also elect the project leader, which is more similar to what
 you're describing, but the project leader has no technical authority;
 in Debian the final authority short of a GR is the CTTE, which is
 explicitly designed as a classic beholden-to-nobody institution -- and
 even overriding the CTTE requires a supermajority.)

 I kinda feel like... as a rule of thumb, if your description of your
 governance model starts with the words I speculate that..., then
 NumPy is probably not a good project to use for your experiment?

So my argument would be that our current data on success (lots of
projects use this model and many of them are OK) is much less useful
than the data from successful forks.

I suppose my question ends up being - do you agree that the core model
does have these risks?   Do they worry you?  What do you think we can
do to guard against them?

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Sebastian Berg]

On Do, 2015-08-27 at 10:45 +0100, Matthew Brett wrote:
 Hi,
 
 On Thu, Aug 27, 2015 at 10:35 AM, Bryan Van de Ven bry...@continuum.io 
 wrote:
 
  On Aug 27, 2015, at 10:22 AM, Matthew Brett matthew.br...@gmail.com 
  wrote:
 
  In the case of the 'core' model, we have some compelling testimony
  from someone with a great deal of experience:
 
  
  Much of this early structure (CVS, web site, cabal [core group],
  etc.) was copied verbatim by other open source (this term not being in
  wide use yet) projects -- even the form of the project name and the
  term core. This later became a kind of standard template for
  starting up an open source project. [...] I'm sorry to say that I
  helped create this problem, and that most of the projects which
  modeled themselves after NetBSD (probably due to its high popularity
  in 1993 and 1994) have suffered similar problems. FreeBSD and XFree86,
  for example, have both forked successor projects (Dragonfly and X.org)
  for very similar reasons.
  
 
  Who goes on to propose:
 
  7) The core group must be replaced with people who are actually
 competent and dedicated enough to review proposals, accept feedback,
 and make good decisions.  More to the point, though, the core group
 must only act when *needed* -- most technical decisions should be
 left to the community to hash out; it must not preempt the community
 from developing better solutions.  (This is how the core group
 worked during most of the project's growth period.)
 
 Sure.  I think it's reasonable to give high weight to Hannum's
 assessment of the failure of the core group, but less weight to his
 proposal for a replacement, because at the time, I don't believe he
 was in a good position to assess whether his (apparent) alternative
 would run into the same trouble.
 
 It's always tempting to blame the people rather than the system, but
 in this case, I strongly suspect that it was the system that was
 fundamentally flawed, therefore either promoting the wrong people or
 putting otherwise competent people into situations where they are no
 longer getting useful feedback.

Maybe so. I do not know much at all about these models, but I am not
sure how much applies here to numpy. Isn't at least FreeBSD a magnitude
larger then numpy?
We do need to have some formality about how to give out commit rights,
and do final decision when all else fails.

One thing I do not know is how a community vote could work at all,
considering I do not even know how to count its members. Votes and
presidents make sense to me for large projects with hundrets of
developers on different corners (think of the gnome foundation, debian
probably) [1].

One thing I could imagine adding is that the community should be
encouraged to ask for/propose new members for the core team.

Nobody is particularly in love with this model, but maybe out of our own
ignorance, we do not see many alternatives after ruling out a BDFL. Yes,
it is a lot fixing a status quo, but we have to fixate something. Any
alternative suggestions are welcome and would be even after deciding on
this. Though maybe that takes away some momentum.

- Sebastian


[1] If we were to have a central governance for the SciPy stack the
story would seem very different to me.


 
 It would be great, and very convenient, if the only management we
 needed was getting out of the way, but I doubt very much that that is
 the case.
 
 Cheers,
 
 Matthew
 ___
 NumPy-Discussion mailing list
 NumPy-Discussion@scipy.org
 http://mail.scipy.org/mailman/listinfo/numpy-discussion
 



signature.asc
Description: This is a digitally signed message part
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

On Thu, Aug 27, 2015 at 9:36 AM, Matthew Brett matthew.br...@gmail.com wrote:
 Hi,

 On Wed, Aug 26, 2015 at 11:46 PM, Stefan van der Walt
 stef...@berkeley.edu wrote:
 Hi Matthew

 On 2015-08-26 10:50:47, Matthew Brett matthew.br...@gmail.com
 wrote:
 In short, the core structure seems to be characteristically
 associated with a conservatism and lack of vision that causes
 the project to stagnate.

 Can you describe how a democratic governance structure would look?
 It's not clear from the discussions linked where successful
 examples are to be found.

 Ah yes - as I was writing at the top of the xfree86 summary, it's
 difficult to assess governance models, because you cannot tell if a
 project that has a particular governance model would have been more
 successful with another model.   For example, would clang be competing
 so successfully with gcc, if gcc had had a different governance model?
  Would Apache be further ahead of the many competitors in the
 web-server space with different management? Difficult to know.

 The advantage of studying forks is that they usually arise from
 disagreements about how a project is managed.  All other things being
 equal, we might expect a fork to fail, given the general aversion to
 forks and the considerable new work that has to be done to get one
 going. So, if a fork succeeds in the long term, that is probably an
 indication that the governance / management of the fork is indeed an
 improvement on the previous model.

 So, in answer to your question, it's difficult to know if a particular
 governance model is successful.   It isn't enough that a project has
 lasted, or is still active, because there are so many factors in play.
   On the other hand, I think it is possible to point to models that
 have a tendency to fail in particular ways, and the by-invitation
 meritocratic 'core' group (I think this is close to the 'steering
 committee' in our current draft) is the model that failed for NetBSD
 and XFree86, with a particular pattern of poor or absent
 accountability and lack of project vision.

Sorry to follow up on my own email, but:

I'm just speculating here, without data, but I suspect one the key
elements that led to the decline and fall of NetBSD and XFree86 was
the perception that there was no way for the community to depose the
government.   It seems these projects managed to combine aspects of
the dictatorship model, with lots of emphasis on personal loyalty and
expected gratitude, with a dysfunctional oligarchy, in which no-one
felt able or willing to change the project direction, when the project
was failing.

The other problem with the meritocracy / invitation model, is that
some people are terrible managers.  In the XFree86 project, for
example, I think David Dawes did a terrible job of guiding the project
when it ran into trouble.  He was in the position he was in because of
his huge commitment and contributions to the project, but I think he
was not the right person to manage the project.

The standard 'core' model, doesn't take that into account.  For
example, I suspect that, if we had a David Dawes, no matter how
terrible we thought they were at managing, we would feel obliged to
put them onto the steering committee.  It is much easier to count or
review commits than it is to assess someone for their qualities as a
leader or manager.   We most of us would hate to be the person to make
that assessment, and it's very tempting to negotiate ourselves into a
world-view where this assessment is not necessary.

So, I speculate, that a good governance model would have:

* one 'president' who has to take final responsibility for all decisions;
* this president might well have a fixed term, maybe with limits on
the number of terms they can serve.
* the president would be chosen by community vote and explicitly on
the basis that they were good managers as well as coders;
* for the presidential election, the candidates should set out what
their vision for the project is, and how they plan to achieve that
vision;

The point about these features is that we explicitly emphasize
accountability, vision and management ability.  Instead of a small
number of people being in the position of assessing their peers for
their ability to manage, the whole community (somehow defined) takes
responsibility for that assessment, therefore making it easier to
think about without distracting issues of personal loyalty or implied
obligation.

See you,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Bryan Van de Ven]

 On Aug 27, 2015, at 9:36 AM, Matthew Brett matthew.br...@gmail.com wrote:
 
 
 So, in answer to your question, it's difficult to know if a particular
 governance model is successful.   It isn't enough that a project has
 lasted, or is still active, because there are so many factors in play.
  On the other hand, I think it is possible to point to models that
 have a tendency to fail in particular ways, and the by-invitation
 meritocratic 'core' group (I think this is close to the 'steering
 committee' in our current draft) is the model that failed for NetBSD
 and XFree86, with a particular pattern of poor or absent
 accountability and lack of project vision.

Anecdotes about two projects is not compelling evidence of anything unless you 
can also point to a comparison of the corresponding success rate. Two failures 
out of three is suggestive. Two failures out of three hundred is significantly 
less interesting. More useful would be  actual details of an alternative 
proposal or pointers to examples of alternative arrangements that could be 
modeled. 

Bryan 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Thu, Aug 27, 2015 at 10:15 AM, Bryan Van de Ven bry...@continuum.io wrote:

 On Aug 27, 2015, at 9:36 AM, Matthew Brett matthew.br...@gmail.com wrote:


 So, in answer to your question, it's difficult to know if a particular
 governance model is successful.   It isn't enough that a project has
 lasted, or is still active, because there are so many factors in play.
  On the other hand, I think it is possible to point to models that
 have a tendency to fail in particular ways, and the by-invitation
 meritocratic 'core' group (I think this is close to the 'steering
 committee' in our current draft) is the model that failed for NetBSD
 and XFree86, with a particular pattern of poor or absent
 accountability and lack of project vision.

 Anecdotes about two projects is not compelling evidence of anything unless 
 you can also point to a comparison of the corresponding success rate. Two 
 failures out of three is suggestive. Two failures out of three hundred is 
 significantly less interesting. More useful would be  actual details of an 
 alternative proposal or pointers to examples of alternative arrangements that 
 could be modeled.


Unfortunately, I don't think we have much choice but to do our best in
sifting through the anecdotal evidence we have available, weak and
contradictory as it is.   Successful forks in large projects are
pretty rare, and as I was arguing before, they are particularly useful
as evidence about governance models.

In the case of the 'core' model, we have some compelling testimony
from someone with a great deal of experience:


Much of this early structure (CVS, web site, cabal [core group],
etc.) was copied verbatim by other open source (this term not being in
wide use yet) projects -- even the form of the project name and the
term core. This later became a kind of standard template for
starting up an open source project. [...] I'm sorry to say that I
helped create this problem, and that most of the projects which
modeled themselves after NetBSD (probably due to its high popularity
in 1993 and 1994) have suffered similar problems. FreeBSD and XFree86,
for example, have both forked successor projects (Dragonfly and X.org)
for very similar reasons.


http://mail-index.netbsd.org/netbsd-users/2006/08/30/0016.html

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Bryan Van de Ven]

 On Aug 27, 2015, at 10:22 AM, Matthew Brett matthew.br...@gmail.com wrote:
 
 In the case of the 'core' model, we have some compelling testimony
 from someone with a great deal of experience:
 
 
 Much of this early structure (CVS, web site, cabal [core group],
 etc.) was copied verbatim by other open source (this term not being in
 wide use yet) projects -- even the form of the project name and the
 term core. This later became a kind of standard template for
 starting up an open source project. [...] I'm sorry to say that I
 helped create this problem, and that most of the projects which
 modeled themselves after NetBSD (probably due to its high popularity
 in 1993 and 1994) have suffered similar problems. FreeBSD and XFree86,
 for example, have both forked successor projects (Dragonfly and X.org)
 for very similar reasons.
 

Who goes on to propose:

7) The core group must be replaced with people who are actually
   competent and dedicated enough to review proposals, accept feedback,
   and make good decisions.  More to the point, though, the core group
   must only act when *needed* -- most technical decisions should be
   left to the community to hash out; it must not preempt the community
   from developing better solutions.  (This is how the core group
   worked during most of the project's growth period.)


Which, AFAICT, is exactly in line with the Numpy proposal:


During the everyday project activities, council members participate in
all discussions, code review and other project activities as peers
with all other Contributors and the Community. In these everyday
activities, Council Members do not have any special power or privilege
through their membership on the Council.
...
However, the Council's primary responsibility is to facilitate the
ordinary community-based decision making procedure described above. If
we ever have to step in and formally override the community for the
health of the Project, then we will do so, but we will consider
reaching this point to indicate a failure in our leadership.


Bryan 
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

On Wed, Aug 26, 2015 at 11:46 PM, Stefan van der Walt
stef...@berkeley.edu wrote:
 Hi Matthew

 On 2015-08-26 10:50:47, Matthew Brett matthew.br...@gmail.com
 wrote:
 In short, the core structure seems to be characteristically
 associated with a conservatism and lack of vision that causes
 the project to stagnate.

 Can you describe how a democratic governance structure would look?
 It's not clear from the discussions linked where successful
 examples are to be found.

Ah yes - as I was writing at the top of the xfree86 summary, it's
difficult to assess governance models, because you cannot tell if a
project that has a particular governance model would have been more
successful with another model.   For example, would clang be competing
so successfully with gcc, if gcc had had a different governance model?
 Would Apache be further ahead of the many competitors in the
web-server space with different management? Difficult to know.

The advantage of studying forks is that they usually arise from
disagreements about how a project is managed.  All other things being
equal, we might expect a fork to fail, given the general aversion to
forks and the considerable new work that has to be done to get one
going. So, if a fork succeeds in the long term, that is probably an
indication that the governance / management of the fork is indeed an
improvement on the previous model.

So, in answer to your question, it's difficult to know if a particular
governance model is successful.   It isn't enough that a project has
lasted, or is still active, because there are so many factors in play.
  On the other hand, I think it is possible to point to models that
have a tendency to fail in particular ways, and the by-invitation
meritocratic 'core' group (I think this is close to the 'steering
committee' in our current draft) is the model that failed for NetBSD
and XFree86, with a particular pattern of poor or absent
accountability and lack of project vision.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Stefan van der Walt]

Hi Sebastian

On 2015-08-27 14:45:50, Sebastian Berg 
sebast...@sipsolutions.net wrote:
 Agreed. Are not PEP's/NEP's just that (and could possibly be 
 formalized more, not sure how much they are in the current 
 proposal) in some sense?  Since they have a sponsor/author who 
 can be said to be assigned to it/responsible once accepted.

I would consider a collection of NEPs for the following year to be 
such a thing.  When implementing a bigger plan, it does help to 
have one person who owns the entire vision at the helm, pushing it 
forward.  I think of it like a symphony orchestra: we agree to 
play the same music, and then let the director oversee the 
execution of the piece as a whole.

 The governance has to be create as little hassle as possible and 
 it should be simple/short enough to quickly understand.

I completely agree; and I think bikeshedding (everyone gets to 
argue their point of view, regardless of the stakes) can be the 
antithesis to productive focus.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Matthew Brett]

Hi,

Splitting this one off too because it's a rather different discussion,
although related.

On Tue, Aug 25, 2015 at 11:03 AM, Nathaniel Smith n...@pobox.com wrote:

[snip]

 Formalizing our governance/decision making
 ==

   This was a major focus of discussion. At a high level, the consensus
   was to steal IPython's governance document (IPEP 29) and modify it
   to remove its use of a BDFL as a backstop to normal community
   consensus-based decision, and replace it with a new backstop based
   on Apache-project-style consensus voting amongst the core team.

Here's a plea to avoid a 'core' structure if at all possible.
Historically it seems to have some severe risks, and experienced
people have blamed this structure for the decline of various projects
including NetBSD and Xfree86, summaries here:

http://asterisk.dynevor.org/melting-core.html
http://asterisk.dynevor.org/xfree-forked.html

In short, the core structure seems to be characteristically associated
with a conservatism and lack of vision that causes the project to
stagnate.

There's also evidence from the NetBSD / OpenBSD split [1] and the
XFree86 / X.org split [2] - that the core structure can lead to bad
decisions being taken in private that no or few members of the core
group are prepared to defend.  I guess what is happening is that
distributed responsibility leads to poor accountability, and therefore
poor decisions.

So, I hope very much we can avoid that trap in our own governance.

Best,

Matthew

[1] http://mail-index.netbsd.org/netbsd-users/1994/12/23/.html
[2] http://www.xfree86.org/pipermail/forum/2003-March/001997.html
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comments on governance proposal (was: Notes from the numpy dev meeting at scipy 2015)

[Stefan van der Walt]

Hi Matthew

On 2015-08-26 10:50:47, Matthew Brett matthew.br...@gmail.com 
wrote:
 In short, the core structure seems to be characteristically 
 associated with a conservatism and lack of vision that causes 
 the project to stagnate.

Can you describe how a democratic governance structure would look? 
It's not clear from the discussions linked where successful 
examples are to be found.

Thanks
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion