Re: [Numpy-discussion] Numpy governance update
On Feb 17, 2012, at 5:20 AM, John Hunter wrote: clip And he has proven his ability to lead when *almost everyone* was against him. At the height of the Numeric/numarray split, and I was deeply involved in this as the mpl author because we had a numerix compatibility layer to allow users to use one or the other, Travis proposed writing numpy to solve both camp's problems. I really can't remember a single individual who supported him. What I remember is the cacophony of voices who though this was a bad idea, because of the third fork problem. But Travis forged ahead, on his own, wrote numpy, and re-united the Numeric and numarray camps. clip Thanks John for including this piece of NumPy history in this context. My voice was part of the cacophony about the third fork problem back in 2005. I was pretty darn uncomfortable on the perspective of adding support for a third numerical package on PyTables. However, Travis started to work with all engines powered on, and in a few months he had built a thing that was overly better than Numeric and numarray together. The rest is history. But I remember this period (2005) as one of the most dramatic examples on how the capacity and dedication of a single individual can shape the world. -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi Ben, On Thu, Feb 16, 2012 at 9:54 PM, Benjamin Root ben.r...@ou.edu wrote: On Thursday, February 16, 2012, John Hunter wrote: On Thu, Feb 16, 2012 at 7:26 PM, Alan G Isaac alan.is...@gmail.com wrote: On 2/16/2012 7:22 PM, Matthew Brett wrote: This has not been an encouraging episode in striving for consensus. I disagree. Failure to reach consensus does not imply lack of striving. Hey Alan, thanks for your thoughtful and nuanced views. I agree with everything you've said, but have a few additional points. At the risk of wading into a thread that has grown far too long, and echoing Eric's comments that the idea of governance is murky at best when there is no provision for enforceability, I have a few comments. Full disclosure: Travis has asked me and I have agreed to to serve on a board for numfocus, the not-for-profit arm of his efforts to promote numpy and related tools. Although I have no special numpy developer chops, as the original author of matplotlib, which is one of the leading numpy clients, he asked me to join his organization as a community representative. I support his efforts, and so agreed to join the numfocus board. My first and most important point is that the subtext of many postings here about the fear of undue and inappropriate influence of Continuum under Travis' leadership is far overblown. Travis created numpy -- it is his baby. Undeniably, he created it by standing on the shoulders of giants: Jim Hugunin, Paul Dubois, Perry Greenfield and his team, and many others. But the idea that we need to guard against the possibility that his corporate interests will compromise his interests in what is best for numpy is academic at best. As someone who has created a significant project in the realm of scientific computing in Python, I can tell you that it is something I take quite a bit of pride in and it is very important to me that the project thrives as it was intended to: as a free, open-source, best-practice way of doing science. I know Travis well enough to know he feels the same way -- numpy doing well is *at least* important to him his company doing well. All of his recent actions to start a company and foundation which focuses resources on numpy and related tools reinforce that view. If he had a different temperament, he wouldn't have devoted five to ten years of is life to Numeric, scipy and numpy. He is a BDFL for a reason: he has earned our trust. And he has proven his ability to lead when *almost everyone* was against him. At the height of the Numeric/numarray split, and I was deeply involved in this as the mpl author because we had a numerix compatibility layer to allow users to use one or the other, Travis proposed writing numpy to solve both camp's problems. I really can't remember a single individual who supported him. What I remember is the cacophony of voices who though this was a bad idea, because of the third fork problem. But Travis forged ahead, on his own, wrote numpy, and re-united the Numeric and numarray camps. And all-the-while he maintained his friendship with the numarray developers (Perry Greenfield who led the numarray development effort has also been invited by Travis to the numfocus board, as has Fernando Perez and Jarrod Millman). Although MPL at the time agreed to support a third version in its numerix compatibility layer for numpy, I can thankfully say we have since dropped support for the compatibility layer entirely as we all use numpy now. This to me is the distilled essence of leadership, against the voices of the masses, and it bears remembering. I have two more points I want to make: one is on democracy, and one is on corporate control. On corporate control: there have been a number of posts in this thread about the worries and dangers that Continuum poses as the corporate sponser of numpy development, about how this may cause numpy to shift from a model of a few loosely connected, decentralized cadre of volunteers to a centrally controlled steering committee of programmers who are controlled by corporate headquarters and who make all their decisions around the water cooler unobserved by the community of users. I want to make a connection to something that happened in the history of matplotlib development, something that is not strictly analogous but I think close enough to be informative. Sometime around 2005, Perry Greenfield, who heads the development team of the Space Telescope Science Institute (STScI) that is charged with processing the Hubble image pipeline, emailed me that he was considering using matplotlib as their primary image visualization tool. I can't tell you how excited I was at the time. The idea of having institutional sponsorship from someone as prestigious and resourceful as STScI was hugely motivating. I worked feverishly for months to add stuff they needed: better rendering, better image support, mathtext and
Re: [Numpy-discussion] Numpy governance update
An example I really like is LibreOffice's get involved page. http://www.libreoffice.org/get-involved/ Producing something similar for NumPy will take some work, but I believe it's needed. Speaking as someone who has contributed to numpy in a microscopic fashion, I agree completely. I spent quite a few hours digging through the webpages, asking for help on the mailing list, reading the Trac, reading git tutorials etc. before I managed to do something remotely useful. In general, I think the webpage for numpy (and scipy, but let's not discuss that here) would benefit from some refurbishing, including the documentation pages. As an example, one of the links on the webpage is Numpy for MATLAB users. I never used matlab much, so this is completely irrelevant for me. I think there should be a discussion about what goes on the front page, and it should be as little as possible, but not less than that. Make it easy for people to 1) start using numpy 2) reading detailed documentation 3) reporting bugs 4) contributing to numpy because those are the fundamental things a user/developer wants from an open source project. Right now there's Trac, github, numpy.scipy.org, http://docs.scipy.org/doc/, the mailing list, and someone mentioned a google group discussing something or other. It took me years to figure out how things are patched together, and I'm still not sure exactly who reads the Trac discussion, github discussion, and mailing list discussions. tl;dr: Numpy is awesome (TM) but needs a more coherent online presence, and one that makes it easy to contribute back to the project. Thanks for making numpy awesome! Paul ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/15/12 6:27 PM, Dag Sverre Seljebotn wrote: But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork the project.) Interesting point. I hope I'm not pitching a log onto the fire here, but in numpy's case, there are very many capable developers on other projects who depend on numpy who could credibly threaten a fork if they felt numpy was drastically going wrong. Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Feb 15, 2012, at 6:18 PM, Joe Harrington wrote: Of course, balancing all of this (and our security blanket) is the possibility of someone splitting the code if they don't like how Continuum runs things. Perry, you've done that yourself to this code's predecessor, so you know the risks. You did that in response to one constituency's moving the code in a direction you didn't like (or not moving it in one you did, I don't remember exactly), as in your example #2. So, while progress might be made when that happens, last time it hurt astronomers enough that you rolled your own and had to put several FTE on the problem. That split held back adoption of numpy both in the astronomy community and outside it, for like 5 years. Perhaps some governance would have saved you the effort and cost and the community the grief of the numarray split. Of course, lots of good eventually came from the split. It wasn't quite like that (hindsight often obscures the perspective at the time). At that time, there was a quasi-consensus that Numeric needed some sort of rewrite. When we started numarray, it wasn't our intent to split the community. That did happen since numarray didn't satisfy enough of the community to get them to buy into it. (It's even more involved than that, but there is no need to rehash those details). I'm not sure what to make of the claim the split held back adoption of numpy. It only makes sense if you say it held back adoption of Numeric in the astronomy community. Numpy wasn't available, and when it was, it didn't take nearly that long to get adopted. I'd have to check, but I'm pretty sure we switched to using it as quickly as possible once it was ready to use. And I still maintain Numeric wasn't really suitable for our needs. Some overhaul was needed, and with that would have been some pain. Could it have all gone smoother somehow? In some ideal world, perhaps. But maybe numarray was a secret plot to get Travis to do numpy all along, and that was the only way to get where we needed to get ;-) Perry ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Feb 16, 2012, at 12:15 PM, Jason Grout wrote: On 2/15/12 6:27 PM, Dag Sverre Seljebotn wrote: But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork the project.) Interesting point. I hope I'm not pitching a log onto the fire here, but in numpy's case, there are very many capable developers on other projects who depend on numpy who could credibly threaten a fork if they felt numpy was drastically going wrong. Jason, that there capable developers out there that are able to fork NumPy (or any other project you can realize) is a given. The point Dag was signaling is that this threaten is more probable to happen *inside* the community. And you pointed out an important aspect too by saying if they felt numpy was drastically going wrong. It makes me the impression that some people is very frightened about something really bad would happen, well before it happens. While I agree that this is *possible*, I'd also advocate to give Travis the benefit of doubt. I'm convinced he (and Continuum as a whole) is making things happen that will benefit the entire NumPy community; but in case something gets really wrong and catastrophic, it is always a relief to know that things can be reverted in the pure open source tradition (by either doing a fork, creating a new foundation, or even better, proposing a new way to do things). What it does not sound reasonable to me is to allow fear to block Continuum efforts for making a better NumPy. I think it is better to relax a bit, see how things are going, and then judge by looking at the *results*. My two cents, Disclaimer: As my e-mail address makes clear, I'm a Continuum guy. -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/16/12 6:23 AM, Francesc Alted wrote: On Feb 16, 2012, at 12:15 PM, Jason Grout wrote: On 2/15/12 6:27 PM, Dag Sverre Seljebotn wrote: But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork the project.) Interesting point. I hope I'm not pitching a log onto the fire here, but in numpy's case, there are very many capable developers on other projects who depend on numpy who could credibly threaten a fork if they felt numpy was drastically going wrong. Jason, that there capable developers out there that are able to fork NumPy (or any other project you can realize) is a given. The point Dag was signaling is that this threaten is more probable to happen *inside* the community. Sure. Given numpy's status as a fundamental building block of many systems, though, if there was a perceived problem by downstream, it's more liable to be forked than most other projects that aren't so close to the headwaters. And you pointed out an important aspect too by saying if they felt numpy was drastically going wrong. It makes me the impression that some people is very frightened about something really bad would happen, well before it happens. While I agree that this is *possible*, I'd also advocate to give Travis the benefit of doubt. I'm convinced he (and Continuum as a whole) is making things happen that will benefit the entire NumPy community; but in case something gets really wrong and catastrophic, it is always a relief to know that things can be reverted in the pure open source tradition (by either doing a fork, creating a new foundation, or even better, proposing a new way to do things). What it does not sound reasonable to me is to allow fear to block Continuum efforts for making a better NumPy. I think it is better to relax a bit, see how things are going, and then judge by looking at the *results*. I'm really happy about Continuum. I agree with Mark that numpy certainly could use a few more core developers. I've not decided on how much structure I feel numpy governance needs (nor do I think it's particularly important for me to decide how I feel at this point on the subject). Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
If I can chime in as a newcomer on this list: I don't think a conflict of interest is at all likely, but I can see the point of those saying that it's worth thinking about this while everything is going well. If any tension does arise, it will be all but impossible to decide on a fair governance structure, because everyone will root for the system that looks likely to produce their favoured outcome. It strikes me that the effort everyone's put into this thread could have by now designed some way to resolve disputes. ;-) It could be as simple as 'so-and-so gets to make the final call', through to committees, voting systems, etc. So long as everything's going well, it shouldn't restrict anyone, and it would reassure anyone who does have concerns (justified or not) about conflicts of interest. Thanks, Thomas ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 16 February 2012 15:08, Thomas Kluyver tak...@gmail.com wrote: It strikes me that the effort everyone's put into this thread could have by now designed some way to resolve disputes. ;-) This is not intended to downplay the concerns raised in this thread, but I can't help myself. I propose the following (tongue-in-cheek) patch against the current numpy master branch. https://github.com/scottza/numpy/compare/constitution If this gets enough interest, I'll consider submitting a real pull request ;-) Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/16/12 8:06 AM, Scott Sinclair wrote: On 16 February 2012 15:08, Thomas Kluyvertak...@gmail.com wrote: It strikes me that the effort everyone's put into this thread could have by now designed some way to resolve disputes. ;-) This is not intended to downplay the concerns raised in this thread, but I can't help myself. I propose the following (tongue-in-cheek) patch against the current numpy master branch. https://github.com/scottza/numpy/compare/constitution If this gets enough interest, I'll consider submitting a real pull request ;-) Time to start submitting lots of 1-line commits and typo fixes to pad my karma :). Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Feb 16, 2012, at 12:08 AM, Matthew Brett wrote: The question is more about what can possibly be done about it. To really shift power, my hunch is that the only practical way would be to, like Mark said, make sure there are very active non-Continuum-employed developers. But perhaps I'm wrong. It's not obvious to me that there isn't a set of guidelines, procedures, structures that would help to keep things clear in this situation. Matthew, I think this is the crux of the issue. There are two kinds of disagreements which could polarize Numpy development: disagreements over vision/values, and disagreements over implementation. The latter can be (and has been) resolved in an ad-hoc fashion because we are all consenting adults here, and as long as there is a consensus about the shared values (i.e. long-term vision) of the project, we can usually work something out. Disagreements over values and long-term vision are the ones that actually do split developer communities, and which procedural guidelines are really quite poor at resolving. In the realm of open source software, value differences (most commonly, licensing disagreements) generally manifest as forks, regardless of what governance may be in place. At the end of the day, you cannot compel people to continue committing to a project that they feel is going the *wrong direction*, not merely the right direction in the wrong way. In the physical world, where we are forced to share geographic space with people who may have vastly different values, it is useful to have a framework for resolution of value differences, because a fork attempt usually means physical warfare. Hence, constitutions, checks balances, impeachment procedures, etc. are all there to avoid forking. But with software, forks are not so costly, and not always a bad thing. Numpy itself arose from merging Numeric and its fork, Numarray, and X.org and EGCS are examples of big forks of major projects which later became the mainline trunk. In short, even if you *could* put governance in place to prevent a fork, that's not always a Good Thing. Creative destruction is vital to the health of any organism or ecosystem, because that is how evolution frequently achieves its greatest results. Of course, this is not to say that I have any desire to see Numpy forked. What I *do* desire is a modular, extensible core of Numpy will allow the experimentation and creative destruction to occur, while minimizing the merge effort when people realize that someone cool has been developed. Lowering the barrier to entry for hacking on the core array code is not merely for Continuum's benefit, but rather will benefit the ecosystem as a whole. No matter how one feels about the potential conflicts of interest, I think we can all agree that the alternative of stagnation is far, far worse. The only way to avoid stagnation is to give the hackers and rebels plenty of room to play, while ensuring a stable base platform for end users and downstream projects to avoid code churn. Travis's and Mark's roadmap proposals for creating a modular core and an extensible C-level ABI are a key technical mechanism for achieving this. Ultimately, procedures and guidelines are only a means to an end, not an ends unto themselves. Curiously enough, I have not yet seen anyone articulate the desire for those *ends* themselves to be written down or manifest as a document. Now, if the Numpy developers want to produce a vision document or values statement for the project, I think that would help as a reference point for any potential disagreements over the direction of the project as commercial stakeholders become involved. But, of course, the request for such a document is itself an unfunded mandate, so it's perfectly possible we may get a one-liner like make Python scientific computing awesome. :-) -Peter Disclaimer: I work with Travis at Continuum. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 02/16/2012 08:06 AM, Scott Sinclair wrote: On 16 February 2012 15:08, Thomas Kluyvertak...@gmail.com wrote: It strikes me that the effort everyone's put into this thread could have by now designed some way to resolve disputes. ;-) This is not intended to downplay the concerns raised in this thread, but I can't help myself. I propose the following (tongue-in-cheek) patch against the current numpy master branch. https://github.com/scottza/numpy/compare/constitution If this gets enough interest, I'll consider submitting a real pull request ;-) Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Now that is totally disrespectful and just plain ignorant! Not to mention the inability to count people correctly. Yes, 'you pushed my button' so to speak. As I understand it, all the pre-git history just contains the information of the person who actually committed the change into the numpy trunk. It does not hold any information of Numeric and the history of numarray so I really question the accuracy of the counting. Also it misses many of the 'user' patches that lead to those changes (perhaps these user-patches are now in git). The second aspect is time frame as you do get a very different list if you just restrict it to 'current developers' eg adding '--since=1 year ago. It is disrespectful because many of the heated discussions are not about code per se but about the design and expected behavior. Counting commits or lines will never tell you any of those things. So I do agree with David's suggestion. Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 12:27 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. I'm not really worried about the Continuum having some nefarious corporate intent. But I am worried about how these plans will affect numpy, and I think there serious risks if we don't think about process. Money has a dramatic effect on FOSS development, and not always in a positive way, even when -- or *especially* when -- everyone has the best of intentions. I'm actually *more* worried about altruistic full-time developers doing work on behalf of the community than I am about developers who are working strictly in some company's interests. Finding a good design for software is like a nasty optimization problem -- it's easy to get stuck in local maxima, and any one person has only an imperfect, noisy estimate of the objective function. So you need lots of eyes to catch mistakes, filter out the noise, and explore multiple maxima in parallel. The classic FOSS model of volunteer developers who are in charge of project direction does a *great* job of solving this problem. (Linux beat all the classic Unixen on technical quality, and it did it using college students and volunteers -- it's not like Sun, IBM, HP etc. couldn't afford better engineers! But they still lost.) Volunteers are intimately familiar with the itch they're trying to scratch and the trade-offs involved in doing so, and they need to work together to produce anything major, so you get lots of different, high-quality perspectives to help you figure out which approach is best. Developers who are working for some corporate interest alter this balance, because in a do-ocracy, someone who can throw a few full-time developers at something suddenly is suddenly has effectively complete control over project direction. There's no moral problem here when the dictator is benevolent, but suddenly you have an informational bottleneck -- even benevolent dictators make mistakes, and they certainly aren't omniscient. Even this isn't *so* bad though, so long as the corporation is scratching their own itch -- at least you can be pretty sure that whatever they produce will at least make them happy, which implies a certain level of utility. The riskiest case is paying developers to scratch someone else's itch. IIUC, that's a major goal of Travis's here, to find a way to pay developers to make numpy better for everyone. But, now you need some way for the community to figure out what better means, because the developers themselves don't necessarily know. It's not their itch anymore. Running a poll or whatever might be a nice start, but we all know how tough it is to extract useful design information from users. You need a lot more than that if you want to keep the quality up. Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. There's a major win in terms of total effort, but you inevitably lose a lot of diversity of viewpoints. My feeling is it will only be a net win if the new employees put serious, bend-over-backwards effort into taking advantage of the volunteer community's wisdom. This is why the NA discussion seems so relevant to me here -- everyone involved absolutely had good intentions, excellent skills, etc., and yet the outcome is still a huge unresolved mess. It was supposed to make numpy more attractive for a certain set of applications, like statistical analysis, where R is currently preferred. Instead, there have been massive changes merged into numpy mainline, but most of the intended target market for these changes is indifferent to them; they don't solve the problem they're supposed to. And along the way we've not just spent a bunch of Enthought's money, but also wasted dozens of hours of volunteer time while seriously alienating some of numpy's most dedicated advocates in that target market. We could debate about blame, and I'm sure there's plenty to spread around, but I also think the fundamental problem isn't one of blame at all -- it's that Mark, Charles and Travis *aren't* scratching an itch; AFAICT the NA functionality is not something they actually need themselves. Which means they're fighting uphill when trying to find the best solutions, and haven't managed it yet. And were working on a deadline, to boot. It's obvious that one should try for consensus as long as possible, including listening to users. But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork
Re: [Numpy-discussion] Numpy governance update
On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. That's not what Travis, or anyone else, proposed. Travis V.___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 11:23 AM, Inati, Souheil (NIH/NIMH) [E] As great and trustworthy as Travis is, there is a very real potential for conflict of interest here. He is going to be leading an organization to raise and distribute funding and at the same time leading a commercial for profit enterprise that would apply to this foundation for funds, as well as being a major player in the direction of the open source project that his company is building on. This is not in and of itself a problem, but the boundaries have to be very clear and laid out in advance. I disagree here -- a business that contributes to an Open-Source project is really no different than an individual that contributes -- it (or he or she) contributes because it sees a benefit -- that could be financial, that could be just for fun, whatever. Sometime individuals get paid to contribute, sometimes companies do -- why is there a difference? To be personal about it -- Continuum writing a bunch of numpy code will be no different than when Travis personally on his own time wrote bunch of numpy code -- and I think we all agree that the project and the community is very grateful he did that when he did. If anyone (company or individual) goes off on their own and writes a bunch of code that community doesn't embrace, then we have a fork -- sometimes that for the better, but for the most part, I think everyone involved does not want to see that happen -- and I think there is a general consensus that a more formal governing stucture is a good idea, in part to prevent that. HOwever -- it's still an open source project -- no onde (or institution) can tell anyone else what to do or how to do it, the the project will be moved forward by those that actually do stuff: - write core code - write supporting code - document stuff - test stuff - package stuff - contribute tech support on the list - contribute to the conversion about development issues - ... and yes -- actually getting around to forming a foundation, or securing funding, or other institutional activities. So while it may seem like a small group of people kind of went off on their own to form a foundation -- that's the only way things ever get done on an open-source project! There may or may not be a lot of discussion about something first, but it only gets down when someone sits down and does it. So Bravo for moving the project forward! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett Personally, I would say that making the founder of a company, which is working to make money from Numpy, the only decision maker on numpy - is - scary. not to me: -- power always goes to those that actually write the code -- as far as I can recall, there has never been a large group of folks contributing to the core code so have a company being the primary contributor and decision maker is no different that what we've always had -- particularly when Travis pretty much single-handedly re-factored Numeric to give us numpy. Oh -- one difference -- having a company with more that one coder means more code! If continuum does indeed develop a bloated, ugly mess to meet their client's needs -- none of us have to use it. -Chris But maybe it's the best way. But, again, we're all high-functioning sensible people, I'm sure it's possible for us to formulate what the risks are, what the potential solutions are, and come up with the best - maybe short-term - solution, See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 9:56 AM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 12:27 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. I'm not really worried about the Continuum having some nefarious corporate intent. But I am worried about how these plans will affect numpy, and I think there serious risks if we don't think about process. Money has a dramatic effect on FOSS development, and not always in a positive way, even when -- or *especially* when -- everyone has the best of intentions. I'm actually *more* worried about altruistic full-time developers doing work on behalf of the community than I am about developers who are working strictly in some company's interests. Finding a good design for software is like a nasty optimization problem -- it's easy to get stuck in local maxima, and any one person has only an imperfect, noisy estimate of the objective function. So you need lots of eyes to catch mistakes, filter out the noise, and explore multiple maxima in parallel. The classic FOSS model of volunteer developers who are in charge of project direction does a *great* job of solving this problem. (Linux beat all the classic Unixen on technical quality, and it did it using college students and volunteers -- it's not like Sun, IBM, HP etc. couldn't afford better engineers! But they still lost.) Volunteers are intimately familiar with the itch they're trying to scratch and the trade-offs involved in doing so, and they need to work together to produce anything major, so you get lots of different, high-quality perspectives to help you figure out which approach is best. Linux is probably a bad choice as example here. Right up to about 2002 Linus was pretty much the only entry point into mainline as he applied all the patches by hand and reviewed all of them. This of course slowed Linux development considerably. I also had the opportunity to fix up some of the drivers for my own machine and can testify that the code quality of the patches was mixed. Now, of course, with 1 or more patches going in during the open period of each development cycle, Linus relies on lieutenants to handle the subsystems, but he can be damn scathing when he takes an interest in some code and doesn't like what he sees. And he *can* be scathing, not just because he started the whole thing, but because he is darn good and the other developers respect that. But my point here is that Linus pretty much shapes Linux. Developers who are working for some corporate interest alter this balance, because in a do-ocracy, someone who can throw a few full-time developers at something suddenly is suddenly has effectively complete control over project direction. There's no moral problem here when the dictator is benevolent, but suddenly you have an informational bottleneck -- even benevolent dictators make mistakes, and they certainly aren't omniscient. Even this isn't *so* bad though, so long as the corporation is scratching their own itch -- at least you can be pretty sure that whatever they produce will at least make them happy, which implies a certain level of utility. Linus deals with this by saying, fork, fork, fork. Of course the gpl makes that a more viable response. The riskiest case is paying developers to scratch someone else's itch. IIUC, that's a major goal of Travis's here, to find a way to pay developers to make numpy better for everyone. But, now you need some way for the community to figure out what better means, because the developers themselves don't necessarily know. It's not their itch anymore. Running a poll or whatever might be a nice start, but we all know how tough it is to extract useful design information from users. You need a lot more than that if you want to keep the quality up. Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. There's a major win in terms of total effort, but you inevitably lose a lot of diversity of viewpoints. My feeling is it will only be a net win if the new employees put serious, bend-over-backwards effort into taking advantage of the volunteer community's wisdom. This is why the NA discussion seems so relevant to me here -- everyone involved absolutely had good intentions, excellent skills, etc., and yet the outcome is still a huge unresolved mess. It was supposed to make numpy more attractive for a certain set of applications, like statistical analysis, where R is currently preferred. Instead, there have been massive changes merged into numpy mainline, but
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 12:53 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Feb 16, 2012 at 9:56 AM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 12:27 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. I'm not really worried about the Continuum having some nefarious corporate intent. But I am worried about how these plans will affect numpy, and I think there serious risks if we don't think about process. Money has a dramatic effect on FOSS development, and not always in a positive way, even when -- or *especially* when -- everyone has the best of intentions. I'm actually *more* worried about altruistic full-time developers doing work on behalf of the community than I am about developers who are working strictly in some company's interests. Finding a good design for software is like a nasty optimization problem -- it's easy to get stuck in local maxima, and any one person has only an imperfect, noisy estimate of the objective function. So you need lots of eyes to catch mistakes, filter out the noise, and explore multiple maxima in parallel. The classic FOSS model of volunteer developers who are in charge of project direction does a *great* job of solving this problem. (Linux beat all the classic Unixen on technical quality, and it did it using college students and volunteers -- it's not like Sun, IBM, HP etc. couldn't afford better engineers! But they still lost.) Volunteers are intimately familiar with the itch they're trying to scratch and the trade-offs involved in doing so, and they need to work together to produce anything major, so you get lots of different, high-quality perspectives to help you figure out which approach is best. Linux is probably a bad choice as example here. Right up to about 2002 Linus was pretty much the only entry point into mainline as he applied all the patches by hand and reviewed all of them. This of course slowed Linux development considerably. I also had the opportunity to fix up some of the drivers for my own machine and can testify that the code quality of the patches was mixed. Now, of course, with 1 or more patches going in during the open period of each development cycle, Linus relies on lieutenants to handle the subsystems, but he can be damn scathing when he takes an interest in some code and doesn't like what he sees. And he *can* be scathing, not just because he started the whole thing, but because he is darn good and the other developers respect that. But my point here is that Linus pretty much shapes Linux. Developers who are working for some corporate interest alter this balance, because in a do-ocracy, someone who can throw a few full-time developers at something suddenly is suddenly has effectively complete control over project direction. There's no moral problem here when the dictator is benevolent, but suddenly you have an informational bottleneck -- even benevolent dictators make mistakes, and they certainly aren't omniscient. Even this isn't *so* bad though, so long as the corporation is scratching their own itch -- at least you can be pretty sure that whatever they produce will at least make them happy, which implies a certain level of utility. Linus deals with this by saying, fork, fork, fork. Of course the gpl makes that a more viable response. The riskiest case is paying developers to scratch someone else's itch. IIUC, that's a major goal of Travis's here, to find a way to pay developers to make numpy better for everyone. But, now you need some way for the community to figure out what better means, because the developers themselves don't necessarily know. It's not their itch anymore. Running a poll or whatever might be a nice start, but we all know how tough it is to extract useful design information from users. You need a lot more than that if you want to keep the quality up. Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. There's a major win in terms of total effort, but you inevitably lose a lot of diversity of viewpoints. My feeling is it will only be a net win if the new employees put serious, bend-over-backwards effort into taking advantage of the volunteer community's wisdom. This is why the NA discussion seems so relevant to me here -- everyone involved absolutely had good intentions, excellent skills, etc., and yet the outcome is still a huge unresolved mess. It was supposed to make numpy more attractive for a certain set of applications, like statistical analysis,
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 11:09 AM, josef.p...@gmail.com wrote: On Thu, Feb 16, 2012 at 12:53 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Feb 16, 2012 at 9:56 AM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 12:27 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. I'm not really worried about the Continuum having some nefarious corporate intent. But I am worried about how these plans will affect numpy, and I think there serious risks if we don't think about process. Money has a dramatic effect on FOSS development, and not always in a positive way, even when -- or *especially* when -- everyone has the best of intentions. I'm actually *more* worried about altruistic full-time developers doing work on behalf of the community than I am about developers who are working strictly in some company's interests. Finding a good design for software is like a nasty optimization problem -- it's easy to get stuck in local maxima, and any one person has only an imperfect, noisy estimate of the objective function. So you need lots of eyes to catch mistakes, filter out the noise, and explore multiple maxima in parallel. The classic FOSS model of volunteer developers who are in charge of project direction does a *great* job of solving this problem. (Linux beat all the classic Unixen on technical quality, and it did it using college students and volunteers -- it's not like Sun, IBM, HP etc. couldn't afford better engineers! But they still lost.) Volunteers are intimately familiar with the itch they're trying to scratch and the trade-offs involved in doing so, and they need to work together to produce anything major, so you get lots of different, high-quality perspectives to help you figure out which approach is best. Linux is probably a bad choice as example here. Right up to about 2002 Linus was pretty much the only entry point into mainline as he applied all the patches by hand and reviewed all of them. This of course slowed Linux development considerably. I also had the opportunity to fix up some of the drivers for my own machine and can testify that the code quality of the patches was mixed. Now, of course, with 1 or more patches going in during the open period of each development cycle, Linus relies on lieutenants to handle the subsystems, but he can be damn scathing when he takes an interest in some code and doesn't like what he sees. And he *can* be scathing, not just because he started the whole thing, but because he is darn good and the other developers respect that. But my point here is that Linus pretty much shapes Linux. Developers who are working for some corporate interest alter this balance, because in a do-ocracy, someone who can throw a few full-time developers at something suddenly is suddenly has effectively complete control over project direction. There's no moral problem here when the dictator is benevolent, but suddenly you have an informational bottleneck -- even benevolent dictators make mistakes, and they certainly aren't omniscient. Even this isn't *so* bad though, so long as the corporation is scratching their own itch -- at least you can be pretty sure that whatever they produce will at least make them happy, which implies a certain level of utility. Linus deals with this by saying, fork, fork, fork. Of course the gpl makes that a more viable response. The riskiest case is paying developers to scratch someone else's itch. IIUC, that's a major goal of Travis's here, to find a way to pay developers to make numpy better for everyone. But, now you need some way for the community to figure out what better means, because the developers themselves don't necessarily know. It's not their itch anymore. Running a poll or whatever might be a nice start, but we all know how tough it is to extract useful design information from users. You need a lot more than that if you want to keep the quality up. Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. There's a major win in terms of total effort, but you inevitably lose a lot of diversity of viewpoints. My feeling is it will only be a net win if the new employees put serious, bend-over-backwards effort into taking advantage of the volunteer community's wisdom. This is why the NA discussion seems so relevant to me here -- everyone involved absolutely had good intentions, excellent skills, etc., and
Re: [Numpy-discussion] Numpy governance update
Hi, On Thu, Feb 16, 2012 at 4:23 AM, Francesc Alted franc...@continuum.io wrote: On Feb 16, 2012, at 12:15 PM, Jason Grout wrote: On 2/15/12 6:27 PM, Dag Sverre Seljebotn wrote: But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork the project.) Interesting point. I hope I'm not pitching a log onto the fire here, but in numpy's case, there are very many capable developers on other projects who depend on numpy who could credibly threaten a fork if they felt numpy was drastically going wrong. Jason, that there capable developers out there that are able to fork NumPy (or any other project you can realize) is a given. The point Dag was signaling is that this threaten is more probable to happen *inside* the community. And you pointed out an important aspect too by saying if they felt numpy was drastically going wrong. It makes me the impression that some people is very frightened about something really bad would happen, well before it happens. While I agree that this is *possible*, I'd also advocate to give Travis the benefit of doubt. I'm convinced he (and Continuum as a whole) is making things happen that will benefit the entire NumPy community; but in case something gets really wrong and catastrophic, it is always a relief to know that things can be reverted in the pure open source tradition (by either doing a fork, creating a new foundation, or even better, proposing a new way to do things). What it does not sound reasonable to me is to allow fear to block Continuum efforts for making a better NumPy. I think it is better to relax a bit, see how things are going, and then judge by looking at the *results*. I'm finding this conversation a bit frustrating. The question on the table as I understand it, is just the following: Is there any governance structure / procedure / set of guidelines that would help ensure the long-term health of the numpy project? The subtext of your response is that you regard *any structure at all* as damaging to the numpy effort and in particular, as damaging to the efforts of Continuum. It seems to me that is a very extreme point of view, and I think, honestly, it is not tenable. But surely - surely - the best thing to do here is to formulate something that might be acceptable, and for everyone to say what they think the problems would be. Do you agree? Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 8:03 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Thu, Feb 16, 2012 at 4:23 AM, Francesc Alted franc...@continuum.io wrote: On Feb 16, 2012, at 12:15 PM, Jason Grout wrote: On 2/15/12 6:27 PM, Dag Sverre Seljebotn wrote: But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork the project.) Interesting point. I hope I'm not pitching a log onto the fire here, but in numpy's case, there are very many capable developers on other projects who depend on numpy who could credibly threaten a fork if they felt numpy was drastically going wrong. Jason, that there capable developers out there that are able to fork NumPy (or any other project you can realize) is a given. The point Dag was signaling is that this threaten is more probable to happen *inside* the community. And you pointed out an important aspect too by saying if they felt numpy was drastically going wrong. It makes me the impression that some people is very frightened about something really bad would happen, well before it happens. While I agree that this is *possible*, I'd also advocate to give Travis the benefit of doubt. I'm convinced he (and Continuum as a whole) is making things happen that will benefit the entire NumPy community; but in case something gets really wrong and catastrophic, it is always a relief to know that things can be reverted in the pure open source tradition (by either doing a fork, creating a new foundation, or even better, proposing a new way to do things). What it does not sound reasonable to me is to allow fear to block Continuum efforts for making a better NumPy. I think it is better to relax a bit, see how things are going, and then judge by looking at the *results*. I'm finding this conversation a bit frustrating. The question on the table as I understand it, is just the following: Is there any governance structure / procedure / set of guidelines that would help ensure the long-term health of the numpy project? The subtext of your response is that you regard *any structure at all* as damaging to the numpy effort and in particular, as damaging to the efforts of Continuum. It seems to me that is a very extreme point of view, and I think, honestly, it is not tenable. That's not exactly how I'd interpret Peter's answer. But surely - surely - the best thing to do here is to formulate something that might be acceptable, and for everyone to say what they think the problems would be. Do you agree? David has made a concrete proposal for a procedure. It looks to me like that's an appropriate and adequate safeguard against Continuum pushing things into Numpy. Would that be enough for you? If not, would it at least be a good start? Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 11:03 AM, Matthew Brett matthew.br...@gmail.com wrote: But surely - surely - the best thing to do here is to formulate something that might be acceptable, and for everyone to say what they think the problems would be. Do you agree? Absolutely -- but just like anything else in open source -- nothing gets done because people think it should get done -- it gets done because someone sits down and does it. having a governance structure is not my itch -- I'm not going to scratch it -- is someone itchy enough to scratch? if so, do so -- while we all continue to talk about what color the bicycle shed behind the foundation offices should be ;-) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 7:46 PM, Benjamin Root ben.r...@ou.edu wrote: Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). BTW, this is still on my todo list -- sorry for dropping the ball here. Perhaps once I find a flat here in Edinburgh. The NA discussion is the perfect example where a governance structure would help resolve disputes. I think the important question is, in an ideal world, what would have been done to help resolve this dispute? My best idea was to try and organize a document articulating points of consensus -- I'm not sure what sort of governance structure would have helped with that. A committee with an odd number of members is good at voting on things, but would a vote have helped? I dunno, I'm not saying it wouldn't -- just that it's something we might want to think about before we start writing bylaws. -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 11:03 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Thu, Feb 16, 2012 at 4:23 AM, Francesc Alted franc...@continuum.io wrote: On Feb 16, 2012, at 12:15 PM, Jason Grout wrote: On 2/15/12 6:27 PM, Dag Sverre Seljebotn wrote: But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork the project.) Interesting point. I hope I'm not pitching a log onto the fire here, but in numpy's case, there are very many capable developers on other projects who depend on numpy who could credibly threaten a fork if they felt numpy was drastically going wrong. Jason, that there capable developers out there that are able to fork NumPy (or any other project you can realize) is a given. The point Dag was signaling is that this threaten is more probable to happen *inside* the community. And you pointed out an important aspect too by saying if they felt numpy was drastically going wrong. It makes me the impression that some people is very frightened about something really bad would happen, well before it happens. While I agree that this is *possible*, I'd also advocate to give Travis the benefit of doubt. I'm convinced he (and Continuum as a whole) is making things happen that will benefit the entire NumPy community; but in case something gets really wrong and catastrophic, it is always a relief to know that things can be reverted in the pure open source tradition (by either doing a fork, creating a new foundation, or even better, proposing a new way to do things). What it does not sound reasonable to me is to allow fear to block Continuum efforts for making a better NumPy. I think it is better to relax a bit, see how things are going, and then judge by looking at the *results*. I'm finding this conversation a bit frustrating. The question on the table as I understand it, is just the following: Is there any governance structure / procedure / set of guidelines that would help ensure the long-term health of the numpy project? The subtext of your response is that you regard *any structure at all* as damaging to the numpy effort and in particular, as damaging to the efforts of Continuum. It seems to me that is a very extreme point of view, and I think, honestly, it is not tenable. But surely - surely - the best thing to do here is to formulate something that might be acceptable, and for everyone to say what they think the problems would be. Do you agree? Perhaps I'm mistaken, but I think the subtext has more been that worrying about potential problems--which aren't yet actual problems--isn't terribly productive. Particularly when the people involved are smart, invested in the success of the broader numpy package, and very deserving of the benefit of the doubt. Also, as Ralf said, David made a concrete proposal. What are your comments on his proposal? -Chris JS Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. That's not what Travis, or anyone else, proposed. Maybe I was unclear -- all I mean here is that if we suddenly have a few people working full-time on numpy (as Travis proposed), then that will cause two things: -- a massive increase in the total number of person-hours going into numpy -- a smaller group of people will be responsible for a much larger proportion of those person-hours (and this is leaving aside the other ways that it can be difficult for full-time developers and volunteers to interact -- the volunteers aren't in the office, the full-timers may not have the patience to wait for a long email-paced conversation before making a decision, etc.) I think Travis' proposal is potentially a great thing, but it's not as simple as just saying hey we hired some people now our software will be better. Ask Fred Brooks ;-) -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 2:13 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. That's not what Travis, or anyone else, proposed. Maybe I was unclear -- all I mean here is that if we suddenly have a few people working full-time on numpy (as Travis proposed), then that will cause two things: -- a massive increase in the total number of person-hours going into numpy -- a smaller group of people will be responsible for a much larger proportion of those person-hours (and this is leaving aside the other ways that it can be difficult for full-time developers and volunteers to interact -- the volunteers aren't in the office, the full-timers may not have the patience to wait for a long email-paced conversation before making a decision, etc.) I think Travis' proposal is potentially a great thing, but it's not as simple as just saying hey we hired some people now our software will be better. Ask Fred Brooks ;-) -- Nathaniel Just a thought I had. From the perspective of any company, they do not want to devote developer resources to an open-source project if the features are going to get rejected (either by the core-devs or by community backlash). Maybe the governance structure could be more along the lines of an advise/consent process for NEPs. This way, a company puts together a plan of action for some features and submits it to the central body (however that is defined). Comments and revisions are done. Finally, if the plan is approved, the company can feel confident that their efforts and resources won't get rejected after committing to the changes. Small changes and bugfixes are not effected by this. Large changes need planning and commentary anyway. This allows some official representation of the community to have some sort of light-handed control over the vision of the project. Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 1:13 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. That's not what Travis, or anyone else, proposed. Maybe I was unclear -- all I mean here is that if we suddenly have a few people working full-time on numpy (as Travis proposed), then that will cause two things: -- a massive increase in the total number of person-hours going into numpy -- a smaller group of people will be responsible for a much larger proportion of those person-hours (and this is leaving aside the other ways that it can be difficult for full-time developers and volunteers to interact -- the volunteers aren't in the office, the full-timers may not have the patience to wait for a long email-paced conversation before making a decision, etc.) I think Travis' proposal is potentially a great thing, but it's not as simple as just saying hey we hired some people now our software will be better. Ask Fred Brooks ;-) What, you are invoking Fred Brooks for a team of, maybe, four? Numpy ain't OS/360. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 8:36 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Feb 16, 2012 at 1:13 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. That's not what Travis, or anyone else, proposed. Maybe I was unclear -- all I mean here is that if we suddenly have a few people working full-time on numpy (as Travis proposed), then that will cause two things: -- a massive increase in the total number of person-hours going into numpy -- a smaller group of people will be responsible for a much larger proportion of those person-hours (and this is leaving aside the other ways that it can be difficult for full-time developers and volunteers to interact -- the volunteers aren't in the office, the full-timers may not have the patience to wait for a long email-paced conversation before making a decision, etc.) I think Travis' proposal is potentially a great thing, but it's not as simple as just saying hey we hired some people now our software will be better. Ask Fred Brooks ;-) What, you are invoking Fred Brooks for a team of, maybe, four? Numpy ain't OS/360. For the general idea that you can't just translate person-hours of effort into results? Yes, though do note the winky emoticon, which is used to indicate that a statement is somewhat tongue in cheek ;-). Do you have any thoughts on the actual content of my concerns? Do you agree that there's a risk that in Travis's plan, you'll be losing out on valuable input from non-core-contributors who are nonetheless experts in particular areas? -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 12:45 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 8:36 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Feb 16, 2012 at 1:13 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. That's not what Travis, or anyone else, proposed. Maybe I was unclear -- all I mean here is that if we suddenly have a few people working full-time on numpy (as Travis proposed), then that will cause two things: -- a massive increase in the total number of person-hours going into numpy -- a smaller group of people will be responsible for a much larger proportion of those person-hours (and this is leaving aside the other ways that it can be difficult for full-time developers and volunteers to interact -- the volunteers aren't in the office, the full-timers may not have the patience to wait for a long email-paced conversation before making a decision, etc.) I think Travis' proposal is potentially a great thing, but it's not as simple as just saying hey we hired some people now our software will be better. Ask Fred Brooks ;-) What, you are invoking Fred Brooks for a team of, maybe, four? Numpy ain't OS/360. For the general idea that you can't just translate person-hours of effort into results? Yes, though do note the winky emoticon, which is used to indicate that a statement is somewhat tongue in cheek ;-). Do you have any thoughts on the actual content of my concerns? Do you agree that there's a risk that in Travis's plan, you'll be losing out on valuable input from non-core-contributors who are nonetheless experts in particular areas? I'm not really sure how. All the developers involved are attentive enough to make announcements of pull requests and requests for comments on proposed changes. So if there's expert opinion to be had easily, i.e. through the mailing list, then I can only imagine they'd go out and get it. This also jibes with Benjamin Root's comment. Major changes will be discussed anyways. So I'm not sure how this particular objection is relevant. -Chris -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 1:45 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 8:36 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Feb 16, 2012 at 1:13 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Feb 16, 2012 at 5:17 PM, Travis Vaught tra...@vaught.net wrote: On Feb 16, 2012, at 10:56 AM, Nathaniel Smith wrote: Travis's proposal is that we go from a large number of self-selecting people putting in little bits of time to a small number of designated people putting in lots of time. That's not what Travis, or anyone else, proposed. Maybe I was unclear -- all I mean here is that if we suddenly have a few people working full-time on numpy (as Travis proposed), then that will cause two things: -- a massive increase in the total number of person-hours going into numpy -- a smaller group of people will be responsible for a much larger proportion of those person-hours (and this is leaving aside the other ways that it can be difficult for full-time developers and volunteers to interact -- the volunteers aren't in the office, the full-timers may not have the patience to wait for a long email-paced conversation before making a decision, etc.) I think Travis' proposal is potentially a great thing, but it's not as simple as just saying hey we hired some people now our software will be better. Ask Fred Brooks ;-) What, you are invoking Fred Brooks for a team of, maybe, four? Numpy ain't OS/360. For the general idea that you can't just translate person-hours of effort into results? Yes, though do note the winky emoticon, which is used to indicate that a statement is somewhat tongue in cheek ;-). Do you have any thoughts on the actual content of my concerns? Do you agree that there's a risk that in Travis's plan, you'll be losing out on valuable input from non-core-contributors who are nonetheless experts in particular areas? I'd be more concerned if I saw more input from non-core-contributors. The sticky issues I see are more along the lines of 1) Trademarking Numpy (TM), which probably needs doing, but who holds the trademark? 2) Distribution of money, accounting, and maybe meeting minutes. If donations are targeted to specific uses, that probably isn't a problem. Advertizing income could be in issue, though. I don't know how much transparency is required by 501(c), probably not much judging by the organizations that have that status. I think Mark's proposal to revisit the issue if/when the number of core contributors reaches maybe 5 is a good one. But in order to attract that many developers long term requires making the code more attractive and laying out a direction. I hope that the initial work along that line is soon published on the list, the sooner the better IMHO. It's not difficult to become a core developer at this point, apart from the non-trivial task of understanding the code and wanting to scratch an itch, since we are pretty desperate for developers. That is to say, the barriers are technical, not social. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
This has been a clarifying discussion for some people. I'm glad people are speaking up. I believe in the value of consensus and the value of users opinions.I want to make sure that people who use NumPy and haven't yet learned how to contribute, feel like they have a voice. I have always been very open about adding people to the lists that I have influence over and giving people permissions to contribute even when they disagree with me. I recognize the value of multiple points of view. That is why in addition to creating the company (with a goal to allow at least some people to spend their day-job working on NumPy), I've pushed to organize a Foundation whose essential mission is to make sure that the core tools used for Python in Science stay open, maintained, and available. I will work very hard to do all I can to make these ventures successful. I had thought I would be able to spend more time on NumPy and SciPy over the past 4 years. This did not work out --- which is why I made a career change.All I can point to is my previous work and say thank you to all who have done so much for the communities I have been able to participate in. I believe in the power of community development, but I also believe in the power of directed development towards solving people's problems in an open market where people can choose to either interact with the provider or find another supplier.Having two organizations that I support helps me direct my energies towards both of those values. I resonate with Linus' s individual leanings. I'm not a big fan of design-by-committee as I haven't seen it be very successful in creating new technologies. It is pretty good at enforcing the status-quo. If I felt like that is what NumPy needed I would be fine with it. However, I feel that NumPy is going to be surpassed with other solutions if steps are not taken to improve the code-base *and* add new features. I'm very interested in discussions about how this work is to be accomplished.I'm with Mark that I believe this discussion will be more useful in 6 months when we have made it easier for more people to get involved with core code development. At the end of the day it is about people and what they spend there time doing. Whatever I do, inside or outside the community, people are free to accept or reject.I can only promise to do my best.It's all I ask of everyone I work with. It is gratifying to see that NumPy has become a well-used project and that there are significant numbers of stake-holders who want to see the project continue to succeed and be useful for them.My goal with the Foundation, with the Company and with my professional life is to see that the growth of Python in Science, Technology, and Data Analysis continues and even accelerates. My view right now is similar to Mark's in that we don't have enough core developers.Charles and Ralf and Pauli and David before them have done an amazing job at pruning, cleaning, and maintaining what is there. Obviously, Jim, Perry, Todd, Rick, Konrad, David A., and Paul Dubois have had significant impact before them.NumPy has always been a community project, but it needs some energy put into it. As someone who is intimately familiar with the code-base (having worked on Numeric as early as 1998 as well as been part of many discussions about Scientific Computing on Python), I'm trying to infuse that energy as best I can. NumPy has a chance to be far more than it is. There are people using inferior solutions because of missing features in NumPy and the lack of awareness of how to use NumPy. There are other important use-cases that NumPy is an almost-there solution for.As it solves these problems, even more users will come to our community, and there needs to be a way to hear their voice as well. Just for the record, I don't believe the NA discussion has been finalized. In fact, the NA discussion this summer was one of the factors that led to my decision to put myself back into NumPy development full time --- I just had to figure out how to do it in a way that my family could accept.I think I could have contributed to that discussion as someone who understands both the code and how it is and has been used. For the next 6-12 months, I am comfortable taking the benevolent dictator role. During that time, I hope we can find many more core developers and then re-visit the discussion. My view is that design decisions should be a consensus based on current contributors to the code base and major users. To continue to be relevant, NumPy has to serve it's customers. They are the ones who will have the final say. If others feel like they can do better, a fork is an option. I don't want that to happen, but it is the only effective and practical governance structure that exists in my mind outside of the
Re: [Numpy-discussion] Numpy governance update
Hi, Just for my own sake, can I clarify what you are saying here? On Thu, Feb 16, 2012 at 1:11 PM, Travis Oliphant tra...@continuum.io wrote: I'm not a big fan of design-by-committee as I haven't seen it be very successful in creating new technologies. It is pretty good at enforcing the status-quo. If I felt like that is what NumPy needed I would be fine with it. Was it your impression that what was being proposed, was design by committee? However, I feel that NumPy is going to be surpassed with other solutions if steps are not taken to improve the code-base *and* add new features. As far as you are concerned, is there any controversy about that? For the next 6-12 months, I am comfortable taking the benevolent dictator role. During that time, I hope we can find many more core developers and then re-visit the discussion. My view is that design decisions should be a consensus based on current contributors to the code base and major users. To continue to be relevant, NumPy has to serve it's customers. They are the ones who will have the final say. If others feel like they can do better, a fork is an option. I don't want that to happen, but it is the only effective and practical governance structure that exists in my mind outside of the self-governance of the people that participate. To confirm, you are saying that you can imagine no improvement in the current governance structure? No organizational structure can make up for the lack of great people putting their hearts and efforts into a great cause. But you agree that there might be an organizational structure that would make this harder or easier? Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Matthew, What you should take from my post is that I appreciate your concern for the future of the NumPy project, and am grateful that you have an eye to the sort of things that can go wrong --- it will help ensure they don't go wrong. But, I personally don't agree that it is necessary to put any more formal structure in place at this time, and we should wait for 6-12 months, and see where we are at while doing everything we can to get more people interested in contributing to the project. I'm comfortable playing the role of BDF12 with a cadre of developers/contributors who seeks to come to consensus.I believe there are sufficient checks on the process that will make it quite difficult for me to *abuse* that in the short term. Charles, Rolf, Mark, David, Robert, Josef, you, and many others are already quite adept at calling me out when I do things they don't like or think are problematic.I encourage them to continue this. I can't promise I'll do everything you want, but I can promise I will listen and take your opinions seriously --- just like I take the opinions of every contributor to the NumPy and SciPy lists seriously (though weighted by the work-effort they have put on the project). We can all only continue to do our best to help out wherever we can. Just so we are clear: Continuum's current major client is the larger NumPy/SciPy community itself and this will remain the case for at least several months.You have nothing to fear from other clients we are trying to please. Thus, we are incentivized to keep as many people happy as possible. In the second place, the Foundation's major client is the same community (and even broader) and the rest of the board is committed to the overall success of the ecosystem. There is a reason the board is comprised of a wide-representation of that eco-system. I am very hopeful that numfocus will evolve over time to have an active community of people who participate in it's processes and plans to support as many projects as it can given the bandwidth and funding available to it. So, if I don't participate in this discussion, anymore, it's because I am working on some open-source things I'd like to show at PyCon, and time is clicking down.If you really feel strongly about this, then I would suggest that you come up with a proposal for governance that you would like us all to review. At the SciPy conference in Austin this summer we can talk about it --- when many of us will be face-to-face. Best regards, -Travis On Feb 16, 2012, at 4:32 PM, Matthew Brett wrote: Hi, Just for my own sake, can I clarify what you are saying here? On Thu, Feb 16, 2012 at 1:11 PM, Travis Oliphant tra...@continuum.io wrote: I'm not a big fan of design-by-committee as I haven't seen it be very successful in creating new technologies. It is pretty good at enforcing the status-quo. If I felt like that is what NumPy needed I would be fine with it. Was it your impression that what was being proposed, was design by committee? However, I feel that NumPy is going to be surpassed with other solutions if steps are not taken to improve the code-base *and* add new features. As far as you are concerned, is there any controversy about that? For the next 6-12 months, I am comfortable taking the benevolent dictator role. During that time, I hope we can find many more core developers and then re-visit the discussion. My view is that design decisions should be a consensus based on current contributors to the code base and major users. To continue to be relevant, NumPy has to serve it's customers. They are the ones who will have the final say. If others feel like they can do better, a fork is an option. I don't want that to happen, but it is the only effective and practical governance structure that exists in my mind outside of the self-governance of the people that participate. To confirm, you are saying that you can imagine no improvement in the current governance structure? No organizational structure can make up for the lack of great people putting their hearts and efforts into a great cause. But you agree that there might be an organizational structure that would make this harder or easier? Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Thu, Feb 16, 2012 at 3:58 PM, Travis Oliphant tra...@continuum.io wrote: Matthew, What you should take from my post is that I appreciate your concern for the future of the NumPy project, and am grateful that you have an eye to the sort of things that can go wrong --- it will help ensure they don't go wrong. But, I personally don't agree that it is necessary to put any more formal structure in place at this time, and we should wait for 6-12 months, and see where we are at while doing everything we can to get more people interested in contributing to the project. I'm comfortable playing the role of BDF12 with a cadre of developers/contributors who seeks to come to consensus. I believe there are sufficient checks on the process that will make it quite difficult for me to *abuse* that in the short term. Charles, Rolf, Mark, David, Robert, Josef, you, and many others are already quite adept at calling me out when I do things they don't like or think are problematic. I encourage them to continue this. I can't promise I'll do everything you want, but I can promise I will listen and take your opinions seriously --- just like I take the opinions of every contributor to the NumPy and SciPy lists seriously (though weighted by the work-effort they have put on the project). We can all only continue to do our best to help out wherever we can. Just so we are clear: Continuum's current major client is the larger NumPy/SciPy community itself and this will remain the case for at least several months. You have nothing to fear from other clients we are trying to please. Thus, we are incentivized to keep as many people happy as possible. In the second place, the Foundation's major client is the same community (and even broader) and the rest of the board is committed to the overall success of the ecosystem. There is a reason the board is comprised of a wide-representation of that eco-system. I am very hopeful that numfocus will evolve over time to have an active community of people who participate in it's processes and plans to support as many projects as it can given the bandwidth and funding available to it. So, if I don't participate in this discussion, anymore, it's because I am working on some open-source things I'd like to show at PyCon, and time is clicking down. If you really feel strongly about this, then I would suggest that you come up with a proposal for governance that you would like us all to review. At the SciPy conference in Austin this summer we can talk about it --- when many of us will be face-to-face. This has not been an encouraging episode in striving for consensus. I see virtually no movement from your implied position at the beginning of this thread, other than the following 1) yes you are in charge 2) you'll consider other options in 6 to 12 months. I think you're saying here that you won't reply any more on this thread, and I suppose that reflects the importance you attach to this problem. I will not myself propose a governance model because I do not consider myself to have enough influence (on various metrics) to make it likely it would be supported. I wish that wasn't my perception of how things are done here. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2012-02-16, at 1:28 PM, Charles R Harris wrote: I think this is a good point, which is why the idea of a long term release is appealing. That release should be stodgy and safe, while the ongoing development can be much more radical in making changes. I sort of thought this *was* the state of affairs re: NumPy 2.0. And numpy really does need a fairly radical rewrite, just to clarify and simplify the base code easier if nothing else. New features I'm more leery about, at least until the code base is improved, which would be my short term priority. As someone who has now thrice ventured into the NumPy C code (twice to add features, once to fix a nasty bug I encountered) I simply could not agree more. While it's not a completely hopeless exercise for someone comfortable with C to get themselves acquainted with NumPy's C internals, the code base as is could be simpler. A refactoring and *documentation* effort would be a good way to get more people contributing to this side of NumPy. I believe the suggestion of seeing more of the C moved to Cython has also been floated before. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Thu, Feb 16, 2012 at 5:26 PM, Alan G Isaac alan.is...@gmail.com wrote: On 2/16/2012 7:22 PM, Matthew Brett wrote: This has not been an encouraging episode in striving for consensus. Striving for consensus does not mean that a minority automatically gets veto rights. 'Striving' for consensus does imply some attempt to get to grips with the arguments, and working on some compromise to accommodate both parties. It seems to me there was very great latitude for finding such a comprise here, but Travis has terminated the discussion and I see no sign of a compromise. Striving for consensus can't of course be regulated. The desire has to be there. It's probably true, as Nathaniel says, that there isn't much you can do to legislate on that. We can only try to persuade. I was trying to do that, I failed, I'll have to look back and see if there was something else I could have done that would have been more useful to the same end, Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thu, Feb 16, 2012 at 7:26 PM, Alan G Isaac alan.is...@gmail.com wrote: On 2/16/2012 7:22 PM, Matthew Brett wrote: This has not been an encouraging episode in striving for consensus. I disagree. Failure to reach consensus does not imply lack of striving. Hey Alan, thanks for your thoughtful and nuanced views. I agree with everything you've said, but have a few additional points. At the risk of wading into a thread that has grown far too long, and echoing Eric's comments that the idea of governance is murky at best when there is no provision for enforceability, I have a few comments. Full disclosure: Travis has asked me and I have agreed to to serve on a board for numfocus, the not-for-profit arm of his efforts to promote numpy and related tools. Although I have no special numpy developer chops, as the original author of matplotlib, which is one of the leading numpy clients, he asked me to join his organization as a community representative. I support his efforts, and so agreed to join the numfocus board. My first and most important point is that the subtext of many postings here about the fear of undue and inappropriate influence of Continuum under Travis' leadership is far overblown. Travis created numpy -- it is his baby. Undeniably, he created it by standing on the shoulders of giants: Jim Hugunin, Paul Dubois, Perry Greenfield and his team, and many others. But the idea that we need to guard against the possibility that his corporate interests will compromise his interests in what is best for numpy is academic at best. As someone who has created a significant project in the realm of scientific computing in Python, I can tell you that it is something I take quite a bit of pride in and it is very important to me that the project thrives as it was intended to: as a free, open-source, best-practice way of doing science. I know Travis well enough to know he feels the same way -- numpy doing well is *at least* important to him his company doing well. All of his recent actions to start a company and foundation which focuses resources on numpy and related tools reinforce that view. If he had a different temperament, he wouldn't have devoted five to ten years of is life to Numeric, scipy and numpy. He is a BDFL for a reason: he has earned our trust. And he has proven his ability to lead when *almost everyone* was against him. At the height of the Numeric/numarray split, and I was deeply involved in this as the mpl author because we had a numerix compatibility layer to allow users to use one or the other, Travis proposed writing numpy to solve both camp's problems. I really can't remember a single individual who supported him. What I remember is the cacophony of voices who though this was a bad idea, because of the third fork problem. But Travis forged ahead, on his own, wrote numpy, and re-united the Numeric and numarray camps. And all-the-while he maintained his friendship with the numarray developers (Perry Greenfield who led the numarray development effort has also been invited by Travis to the numfocus board, as has Fernando Perez and Jarrod Millman). Although MPL at the time agreed to support a third version in its numerix compatibility layer for numpy, I can thankfully say we have since dropped support for the compatibility layer entirely as we all use numpy now. This to me is the distilled essence of leadership, against the voices of the masses, and it bears remembering. I have two more points I want to make: one is on democracy, and one is on corporate control. On corporate control: there have been a number of posts in this thread about the worries and dangers that Continuum poses as the corporate sponser of numpy development, about how this may cause numpy to shift from a model of a few loosely connected, decentralized cadre of volunteers to a centrally controlled steering committee of programmers who are controlled by corporate headquarters and who make all their decisions around the water cooler unobserved by the community of users. I want to make a connection to something that happened in the history of matplotlib development, something that is not strictly analogous but I think close enough to be informative. Sometime around 2005, Perry Greenfield, who heads the development team of the Space Telescope Science Institute (STScI) that is charged with processing the Hubble image pipeline, emailed me that he was considering using matplotlib as their primary image visualization tool. I can't tell you how excited I was at the time. The idea of having institutional sponsorship from someone as prestigious and resourceful as STScI was hugely motivating. I worked feverishly for months to add stuff they needed: better rendering, better image support, mathtext and lots more. But more importantly, Perry was offering to bring institutional support to my project: well qualified full-time employees who dedicated a significant part of their time to matplotlib development. He
Re: [Numpy-discussion] Numpy governance update
Hi John, On Thu, Feb 16, 2012 at 8:20 PM, John Hunter jdh2...@gmail.com wrote: On Thu, Feb 16, 2012 at 7:26 PM, Alan G Isaac alan.is...@gmail.com wrote: On 2/16/2012 7:22 PM, Matthew Brett wrote: This has not been an encouraging episode in striving for consensus. I disagree. Failure to reach consensus does not imply lack of striving. Hey Alan, thanks for your thoughtful and nuanced views. I agree with everything you've said, but have a few additional points. I thought I'd looked deep in my heart and failed to find paranoia about corporate involvement in numpy. I am happy that Travis formed Continuum and look forward to the progress we can expect for numpy. I don't think the conversation was much about 'democracy'. As far as I was concerned, anything on the range of no change but at least being specific to full veto power from mailing list members was up for discussion and anything in between. I wish we had not had to deal with the various red herrings here, such as whether Continuum is good or bad, whether Travis has been given adequate credit, or whether companies are bad for software. But, we did. It's fine. Argument over now. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Thursday, February 16, 2012, John Hunter wrote:
On Thu, Feb 16, 2012 at 7:26 PM, Alan G Isaac
alan.is...@gmail.comjavascript:_e({}, 'cvml', 'alan.is...@gmail.com');
wrote:
On 2/16/2012 7:22 PM, Matthew Brett wrote:
This has not been an encouraging episode in striving for consensus.
I disagree.
Failure to reach consensus does not imply lack of striving.
Hey Alan, thanks for your thoughtful and nuanced views. I agree with
everything you've said, but have a few additional points.
At the risk of wading into a thread that has grown far too long, and
echoing Eric's comments that the idea of governance is murky at best
when there is no provision for enforceability, I have a few comments.
Full disclosure: Travis has asked me and I have agreed to to serve on
a board for numfocus, the not-for-profit arm of his efforts to
promote numpy and related tools. Although I have no special numpy
developer chops, as the original author of matplotlib, which is one of
the leading numpy clients, he asked me to join his organization as a
community representative. I support his efforts, and so agreed to
join the numfocus board.
My first and most important point is that the subtext of many postings here
about the fear of undue and inappropriate influence of Continuum under
Travis' leadership is far overblown. Travis created numpy -- it is
his baby. Undeniably, he created it by standing on the shoulders of
giants: Jim Hugunin, Paul Dubois, Perry Greenfield and his team, and
many others. But the idea that we need to guard against the
possibility that his corporate interests will compromise his interests
in what is best for numpy is academic at best.
As someone who has created a significant project in the realm of
scientific computing in Python, I can tell you that it is something
I take quite a bit of pride in and it is very important to me that the
project thrives as it was intended to: as a free, open-source,
best-practice way of doing science. I know Travis well enough to know
he feels the same way -- numpy doing well is *at least* important to
him his company doing well. All of his recent actions to start a
company and foundation which focuses resources on numpy and related
tools reinforce that view. If he had a different temperament, he
wouldn't have devoted five to ten years of is life to Numeric, scipy
and numpy. He is a BDFL for a reason: he has earned our trust.
And he has proven his ability to lead when *almost everyone* was
against him. At the height of the Numeric/numarray split, and I was
deeply involved in this as the mpl author because we had a numerix
compatibility layer to allow users to use one or the other, Travis
proposed writing numpy to solve both camp's problems. I really can't
remember a single individual who supported him. What I remember is
the cacophony of voices who though this was a bad idea, because of the
third fork problem. But Travis forged ahead, on his own, wrote
numpy, and re-united the Numeric and numarray camps. And
all-the-while he maintained his friendship with the numarray
developers (Perry Greenfield who led the numarray development effort
has also been invited by Travis to the numfocus board, as has Fernando
Perez and Jarrod Millman). Although MPL at the time agreed to support
a third version in its numerix compatibility layer for numpy, I can
thankfully say we have since dropped support for the compatibility
layer entirely as we all use numpy now. This to me is the distilled
essence of leadership, against the voices of the masses, and it bears
remembering.
I have two more points I want to make: one is on democracy, and one is
on corporate control. On corporate control: there have been a number
of posts in this thread about the worries and dangers that Continuum
poses as the corporate sponser of numpy development, about how this
may cause numpy to shift from a model of a few loosely connected,
decentralized cadre of volunteers to a centrally controlled steering
committee of programmers who are controlled by corporate headquarters
and who make all their decisions around the water cooler unobserved by
the community of users.
I want to make a connection to something that happened in the history
of matplotlib development, something that is not strictly analogous
but I think close enough to be informative. Sometime around 2005,
Perry Greenfield, who heads the development team of the Space
Telescope Science Institute (STScI) that is charged with processing
the Hubble image pipeline, emailed me that he was considering using
matplotlib as their primary image visualization tool. I can't tell
you how excited I was at the time. The idea of having institutional
sponsorship from someone as prestigious and resourceful as STScI was
hugely motivating. I worked feverishly for months to add stuff they
needed: better rendering, better image support, mathtext and lots
more. But more
Re: [Numpy-discussion] Numpy governance update
On Fri, Feb 17, 2012 at 12:54 AM, Benjamin Root ben.r...@ou.edu wrote: On Thursday, February 16, 2012, John Hunter wrote: On Thu, Feb 16, 2012 at 7:26 PM, Alan G Isaac alan.is...@gmail.com wrote: On 2/16/2012 7:22 PM, Matthew Brett wrote: This has not been an encouraging episode in striving for consensus. I disagree. Failure to reach consensus does not imply lack of striving. Hey Alan, thanks for your thoughtful and nuanced views. I agree with everything you've said, but have a few additional points. At the risk of wading into a thread that has grown far too long, and echoing Eric's comments that the idea of governance is murky at best when there is no provision for enforceability, I have a few comments. Full disclosure: Travis has asked me and I have agreed to to serve on a board for numfocus, the not-for-profit arm of his efforts to promote numpy and related tools. Although I have no special numpy developer chops, as the original author of matplotlib, which is one of the leading numpy clients, he asked me to join his organization as a community representative. I support his efforts, and so agreed to join the numfocus board. My first and most important point is that the subtext of many postings here about the fear of undue and inappropriate influence of Continuum under Travis' leadership is far overblown. Travis created numpy -- it is his baby. Undeniably, he created it by standing on the shoulders of giants: Jim Hugunin, Paul Dubois, Perry Greenfield and his team, and many others. But the idea that we need to guard against the possibility that his corporate interests will compromise his interests in what is best for numpy is academic at best. As someone who has created a significant project in the realm of scientific computing in Python, I can tell you that it is something I take quite a bit of pride in and it is very important to me that the project thrives as it was intended to: as a free, open-source, best-practice way of doing science. I know Travis well enough to know he feels the same way -- numpy doing well is *at least* important to him his company doing well. All of his recent actions to start a company and foundation which focuses resources on numpy and related tools reinforce that view. If he had a different temperament, he wouldn't have devoted five to ten years of is life to Numeric, scipy and numpy. He is a BDFL for a reason: he has earned our trust. And he has proven his ability to lead when *almost everyone* was against him. At the height of the Numeric/numarray split, and I was deeply involved in this as the mpl author because we had a numerix compatibility layer to allow users to use one or the other, Travis proposed writing numpy to solve both camp's problems. I really can't remember a single individual who supported him. What I remember is the cacophony of voices who though this was a bad idea, because of the third fork problem. But Travis forged ahead, on his own, wrote numpy, and re-united the Numeric and numarray camps. And all-the-while he maintained his friendship with the numarray developers (Perry Greenfield who led the numarray development effort has also been invited by Travis to the numfocus board, as has Fernando Perez and Jarrod Millman). Although MPL at the time agreed to support a third version in its numerix compatibility layer for numpy, I can thankfully say we have since dropped support for the compatibility layer entirely as we all use numpy now. This to me is the distilled essence of leadership, against the voices of the masses, and it bears remembering. I have two more points I want to make: one is on democracy, and one is on corporate control. On corporate control: there have been a number of posts in this thread about the worries and dangers that Continuum poses as the corporate sponser of numpy development, about how this may cause numpy to shift from a model of a few loosely connected, decentralized cadre of volunteers to a centrally controlled steering committee of programmers who are controlled by corporate headquarters and who make all their decisions around the water cooler unobserved by the community of users. I want to make a connection to something that happened in the history of matplotlib development, something that is not strictly analogous but I think close enough to be informative. Sometime around 2005, Perry Greenfield, who heads the development team of the Space Telescope Science Institute (STScI) that is charged with processing the Hubble image pipeline, emailed me that he was considering using matplotlib as their primary image visualization tool. I can't tell you how excited I was at the time. The idea of having institutional sponsorship from someone as prestigious and resourceful as STScI was hugely motivating. I worked feverishly for months to add stuff they needed: better rendering, better image support, mathtext and lots
Re: [Numpy-discussion] Numpy governance update
On 16 February 2012 17:31, Bruce Southey bsout...@gmail.com wrote: On 02/16/2012 08:06 AM, Scott Sinclair wrote: This is not intended to downplay the concerns raised in this thread, but I can't help myself. I propose the following (tongue-in-cheek) patch against the current numpy master branch. https://github.com/scottza/numpy/compare/constitution If this gets enough interest, I'll consider submitting a real pull request ;-) Now that is totally disrespectful and just plain ignorant! Not to mention the inability to count people correctly. I'm sorry that you feel that way and apologize if I've offended you. I didn't expect to and assure you that was not my intention. That said, I do hope that we can continue to make allowance for (very occasional) levity in the community. Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
Le 15/02/2012 04:07, Bruce Southey a écrit : The one thing that gets over looked here is that there is a huge diversity of users with very different skill levels. But very few people have an understanding of the core code. (In fact the other thread about type-casting suggests that it is extremely few people.) So in all of this, I do not yet see 'community'. But the only way you can change that perception is through actions. Hi Bruce, I agree with the skill issue you raised. My own experience being : 1) For some years, I've been a quite heavy user of numpy and various scipy modules. Zero knowledge of the numpy code 2) I recently (November 2011) subscribed to numpy scipy ML. Going through the various topics coming every day, I feel like I'm learning more faster. I'm now browsing numpy's GitHub from time to time. 3) Now I see regularly messages about topics like datetime(64) and NAs about which I feel I could share my ($.02 !) views as a user or as a potential user. But in the end I don't write anything because the issue is so complex that I feel both lost and silly. I have no solution to propose, so I try to keep on learning... -- Pierre ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/14/2012 10:07 PM, Bruce Southey wrote: The one thing that gets over looked here is that there is a huge diversity of users with very different skill levels. But very few people have an understanding of the core code. (In fact the other thread about type-casting suggests that it is extremely few people.) So in all of this, I do not yet see 'community'. As an active user and long-time list member who has never even looked at the core code, I perhaps presumptuously urge a moderation of rhetoric. I object to the idea that users like myself do not form part of the community. This list has 1400 subscribers, and the fact that most of us are quiet most of the time does not mean we are not interested or attentive to the discussions, including discussions of governance. It looks to me like this will be great for NumPy. People who would otherwise not be able to spend much time on NumPy will be spending a lot of time improving the code and adding features. In my view, this will help NumPy advance which will enlarge the user community, which will slowly but inevitably enlarge the contributor community. I'm pretty excited about Travis's bold efforts to find ways to allow him and others to spend more time on NumPy. I wish him the best of luck. Cheers, Alan Isaac ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 5:51 AM, Alan G Isaac alan.is...@gmail.com wrote: On 2/14/2012 10:07 PM, Bruce Southey wrote: The one thing that gets over looked here is that there is a huge diversity of users with very different skill levels. But very few people have an understanding of the core code. (In fact the other thread about type-casting suggests that it is extremely few people.) So in all of this, I do not yet see 'community'. As an active user and long-time list member who has never even looked at the core code, I perhaps presumptuously urge a moderation of rhetoric. I object to the idea that users like myself do not form part of the community. This list has 1400 subscribers, and the fact that most of us are quiet most of the time does not mean we are not interested or attentive to the discussions, including discussions of governance. It looks to me like this will be great for NumPy. People who would otherwise not be able to spend much time on NumPy will be spending a lot of time improving the code and adding features. In my view, this will help NumPy advance which will enlarge the user community, which will slowly but inevitably enlarge the contributor community. I'm pretty excited about Travis's bold efforts to find ways to allow him and others to spend more time on NumPy. I wish him the best of luck. I think it is important to stick to the thread topic here, which is 'Governance'. It's not about whether it is good or bad that Travis has re-engaged in Numpy and is funding development in Numpy through his company. I'm personally very glad to see Travis back on the list and engaged again, but that's really not what the thread is about. The thread is about whether we need explicit Numpy governance, especially in the situation where one new company will surely dominate numpy development in the short term at least. I would say - for the benefit of Continuum Analytics and for the Numpy community, there should be explicit governance, that takes this relationship into account. I believe that leaving the governance informal and underspecified at this stage would be a grave mistake, for everyone concerned. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hello, From: Matthew Brett [matthew.br...@gmail.com] Sent: Wednesday, February 15, 2012 1:50 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Numpy governance update Hi, On Wed, Feb 15, 2012 at 5:51 AM, Alan G Isaac alan.is...@gmail.com wrote: On 2/14/2012 10:07 PM, Bruce Southey wrote: The one thing that gets over looked here is that there is a huge diversity of users with very different skill levels. But very few people have an understanding of the core code. (In fact the other thread about type-casting suggests that it is extremely few people.) So in all of this, I do not yet see 'community'. As an active user and long-time list member who has never even looked at the core code, I perhaps presumptuously urge a moderation of rhetoric. I object to the idea that users like myself do not form part of the community. This list has 1400 subscribers, and the fact that most of us are quiet most of the time does not mean we are not interested or attentive to the discussions, including discussions of governance. It looks to me like this will be great for NumPy. People who would otherwise not be able to spend much time on NumPy will be spending a lot of time improving the code and adding features. In my view, this will help NumPy advance which will enlarge the user community, which will slowly but inevitably enlarge the contributor community. I'm pretty excited about Travis's bold efforts to find ways to allow him and others to spend more time on NumPy. I wish him the best of luck. I think it is important to stick to the thread topic here, which is 'Governance'. It's not about whether it is good or bad that Travis has re-engaged in Numpy and is funding development in Numpy through his company. I'm personally very glad to see Travis back on the list and engaged again, but that's really not what the thread is about. The thread is about whether we need explicit Numpy governance, especially in the situation where one new company will surely dominate numpy development in the short term at least. I would say - for the benefit of Continuum Analytics and for the Numpy community, there should be explicit governance, that takes this relationship into account. I believe that leaving the governance informal and underspecified at this stage would be a grave mistake, for everyone concerned. Best, Matthew ___ As another of the silent users of numpy, I agree with Matthew 100%. As great and trustworthy as Travis is, there is a very real potential for conflict of interest here. He is going to be leading an organization to raise and distribute funding and at the same time leading a commercial for profit enterprise that would apply to this foundation for funds, as well as being a major player in the direction of the open source project that his company is building on. This is not in and of itself a problem, but the boundaries have to be very clear and layed out in advance. Which hat is he wearing when he recommends one course of action over another? I understand the company is just getting off the ground and that the foundation is even less well formed, but numpy is a mature code with lots of users. It's governance structure should reflect this. Continued thanks for all of the hard work. -Souheil Inati ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/15/2012 1:50 PM, Matthew Brett wrote: I believe that leaving the governance informal and underspecified at this stage would be a grave mistake, for everyone concerned. To justify that concern, can you point to an analogous case, where things went awry by not formalizing the governance structure? Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? Still, honestly, I have trouble seeing how implementing this idea would currently have much affect on the substance or extent or direction of the conversations about NumPy. Thanks, Alan PS Just to jog the group memory, Travis announced more than four months ago *on this list* that he had been approached about the possibility of creating a foundation to support the development of SciPy and NumPy, and had become interested in creating a Foundation for the Advancement of Scientific, Technical, and Engineering Computing Using High Level Abstractions (FASTECUHLA), and had created an open discussion list for this at fastecu...@googlegroups.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 02/15/2012 08:50 AM, Matthew Brett wrote: Hi, On Wed, Feb 15, 2012 at 5:51 AM, Alan G Isaacalan.is...@gmail.com wrote: On 2/14/2012 10:07 PM, Bruce Southey wrote: The one thing that gets over looked here is that there is a huge diversity of users with very different skill levels. But very few people have an understanding of the core code. (In fact the other thread about type-casting suggests that it is extremely few people.) So in all of this, I do not yet see 'community'. As an active user and long-time list member who has never even looked at the core code, I perhaps presumptuously urge a moderation of rhetoric. I object to the idea that users like myself do not form part of the community. This list has 1400 subscribers, and the fact that most of us are quiet most of the time does not mean we are not interested or attentive to the discussions, including discussions of governance. It looks to me like this will be great for NumPy. People who would otherwise not be able to spend much time on NumPy will be spending a lot of time improving the code and adding features. In my view, this will help NumPy advance which will enlarge the user community, which will slowly but inevitably enlarge the contributor community. I'm pretty excited about Travis's bold efforts to find ways to allow him and others to spend more time on NumPy. I wish him the best of luck. I think it is important to stick to the thread topic here, which is 'Governance'. Do you have in mind a model of how this might work? (I suspect you have already answered a question like that in some earlier thread; sorry.) A comparable project that is doing it right? Governance implies enforcement power, doesn't it? Where, how, and by whom would the power be exercised? It's not about whether it is good or bad that Travis has re-engaged in Numpy and is funding development in Numpy through his company. I'm personally very glad to see Travis back on the list and engaged again, but that's really not what the thread is about. The thread is about whether we need explicit Numpy governance, especially in the situation where one new company will surely dominate numpy development in the short term at least. I would say - for the benefit of Continuum Analytics and for the Numpy community, there should be explicit governance, that takes this relationship into account. Please elaborate; are you saying that Continuum Analytics must develop numpy as decided by some outside body? Eric I believe that leaving the governance informal and underspecified at this stage would be a grave mistake, for everyone concerned. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/15/2012 2:46 PM, Benjamin Root wrote: I think it is only fair that the group occasionally pings this mailing-list for important progress reports. No offense intended, but that sounds like an unfunded mandate. More useful would be an offer to liaison between the two. Cheers, Alan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, Thanks for these interesting and specific questions. On Wed, Feb 15, 2012 at 11:33 AM, Eric Firing efir...@hawaii.edu wrote: On 02/15/2012 08:50 AM, Matthew Brett wrote: Hi, On Wed, Feb 15, 2012 at 5:51 AM, Alan G Isaacalan.is...@gmail.com wrote: On 2/14/2012 10:07 PM, Bruce Southey wrote: The one thing that gets over looked here is that there is a huge diversity of users with very different skill levels. But very few people have an understanding of the core code. (In fact the other thread about type-casting suggests that it is extremely few people.) So in all of this, I do not yet see 'community'. As an active user and long-time list member who has never even looked at the core code, I perhaps presumptuously urge a moderation of rhetoric. I object to the idea that users like myself do not form part of the community. This list has 1400 subscribers, and the fact that most of us are quiet most of the time does not mean we are not interested or attentive to the discussions, including discussions of governance. It looks to me like this will be great for NumPy. People who would otherwise not be able to spend much time on NumPy will be spending a lot of time improving the code and adding features. In my view, this will help NumPy advance which will enlarge the user community, which will slowly but inevitably enlarge the contributor community. I'm pretty excited about Travis's bold efforts to find ways to allow him and others to spend more time on NumPy. I wish him the best of luck. I think it is important to stick to the thread topic here, which is 'Governance'. Do you have in mind a model of how this might work? (I suspect you have already answered a question like that in some earlier thread; sorry.) A comparable project that is doing it right? The example that had come up previously was the book by Karl Fogel: http://producingoss.com/en/social-infrastructure.html http://producingoss.com/en/consensus-democracy.html In particular, the section When Consensus Cannot Be Reached, Vote in the second page. Here's an example of a voting policy: http://www.apache.org/foundation/voting.html Debian is a famous example: http://www.debian.org/devel/constitution Obviously some open-source projects do not have much of a formal governance structure, but I think in our case a) we have already run into problems with big decisions and b) we have now reached a situation where there is serious potential for actual or perceived problems with conflicts of interest. Governance implies enforcement power, doesn't it? Where, how, and by whom would the power be exercised? The governance that I had in mind is more to do with review and constraint of power. Thus, I believe we need a set of rules to govern how we deal with serious disputes, such as the masked array NA debate, or, previously the ABI breakage discussion at numpy 1.5.0. To go to a specific use-case. Let us imagine that Continuum think of an excellent feature they want in Numpy but that many others think would make the underlying array object too complicated. How would the desires of Continuum be weighed against the desires of other members of the community? It's not about whether it is good or bad that Travis has re-engaged in Numpy and is funding development in Numpy through his company. I'm personally very glad to see Travis back on the list and engaged again, but that's really not what the thread is about. The thread is about whether we need explicit Numpy governance, especially in the situation where one new company will surely dominate numpy development in the short term at least. I would say - for the benefit of Continuum Analytics and for the Numpy community, there should be explicit governance, that takes this relationship into account. Please elaborate; are you saying that Continuum Analytics must develop numpy as decided by some outside body? No - of course not. Here's the discussion from Karl Fogel's book: http://producingoss.com/en/contracting.html I'm proposing Governance not as some council that contracts work, but as a committee set up with formal rules that can resolve disputes and rule changes as they arise. This committee needs to be able to do this to make sure that the interests of the community (developers of numpy outside Continuum) are being represented. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/15/2012 2:46 PM, Benjamin Root wrote: The NA discussion is the perfect example where a governance structure would help resolve disputes. How? I'm not seeing it. Who would have behaved differently and why? Alan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. We should look at what other projects do. I think that this committee needs to be people who know numpy code; projects using numpy could advise, but people developing numpy should vote I think. There should be rules of engagement, a constitution, especially how to deal with disputes with Continuum or other contracting organizations. I would personally very much like to see a committment to consensus, where possible on these lines (as noted previously by Nathaniel): http://producingoss.com/en/consensus-democracy.html Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 2:03 PM, Alan G Isaac alan.is...@gmail.com wrote: On 2/15/2012 2:46 PM, Benjamin Root wrote: The NA discussion is the perfect example where a governance structure would help resolve disputes. How? I'm not seeing it. Who would have behaved differently and why? Alan I am pretty sure it was Matthew (but I could be wrong here) on numerous occasions would have stated that if there was some sort of process that was agreed upon beforehand, that he would have abided by the decision/outcome of that process. As much as I disagreed with Matthew and others on the design of NA (and the amount of review that went into the branch getting merged), I do agree that a formal process for handling grievances would have helped mitigate much of the problems in the discussions. Further, a more fleshed out review process for major changes would have aired out more of the design decisions (somewhat). Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
My analysis is fundamentally different than Matthew and Benjamin's for a few reasons. 1. The problem has been miscast. The economic interests of the developers *always* has had an apparent conflict with the economic interests of the users: users want developers to work more on the code, and developers need to make a living, which often involves spending their time on other things. On this score, nothing has really changed. 2. It seems pretty clear that Matthew wants some governance power to be held by individuals who are not actively developing NumPy. As Chuck Harris pointed out long ago, that dog ain't going to hunt. 3. Constitutions can be broken (and are, all the time). Designing a stable institution requires making it in the interests of the members to participate. Any formal governance structure that can be desirable for the NumPy community as a whole has to be desirable for the core developers. The right way to produce a governance structure is to make concrete proposals and show how these proposals are in the interest of the *developers* (as well as of the users). For example, Benjamin obliquely suggested that with an appropriate governance board, the NA discussion could have simply been shut down by having the developers vote (as part of their governance). This might be in the interest of the developers and of the community (I'm not sure), but I doubt it is what Matthew has in mind. In any case, until proposals are put on the table along with a clear effort to illustrate why it is in the interest of the *developers* to adopt the proposals, I really do not see this discussion moving forward. fwiw, Alan Isaac ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. If we need a formal structure, maybe a good approach is giving Travis the final say for now, until a trigger point occurs. That could be 6 months after the number of active developers hits 5, or something like that. At that point, we would reopen the discussion with a larger group of people who would directly play in that role, and any decision made then will probably be better than a decision we make now while the development team is so small. -Mark We should look at what other projects do. I think that this committee needs to be people who know numpy code; projects using numpy could advise, but people developing numpy should vote I think. There should be rules of engagement, a constitution, especially how to deal with disputes with Continuum or other contracting organizations. I would personally very much like to see a committment to consensus, where possible on these lines (as noted previously by Nathaniel): http://producingoss.com/en/consensus-democracy.html Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Feb 15, 2012, at 3:01 PM, Matthew Brett wrote: [...] My 2 cents. I think you put too much faith in formal systems. There are plenty of examples of formal governance that fail miserably. In the end it depends on the people and their willingness to continue cooperating. Formal governance won't protect people from misbehaving or abusing the formal system if they are so inclined. So my thought on this is why not see how it works out. Even if Travis's company has a conflict of interest (and that is certainly a possibility) it isn't always a bad thing. Look at two scenarios: 1) a project requires that all work is done by altruistic people with no conflicts of interest. But it languishes due to a lack of sufficient resources. 2) a big, bad, evil, self-interested company infuses lots of resources and talent, and they bend the project their way to meet their interests. The resulting project has lots of new capability, but isn't quite as pure as the altruistic people would have had it. (Mind you, it's still open source software!) Neither is ideal. But sometimes it's 2) that has led to progress. If the distortion of the self interested companies it too big, then it's a net negative. But even the self-interested company has a large stake in seeing the community not split. And you see this in the open source community all the time, even from the altruistic. Those that do the work generally get the most say in how it is done. Finally, I think you should cut Travis some slack here. No one has come close to the personal investment in numpy that he has (and you probably aren't aware of all if it). If anyone deserves the benefit of the doubt, it's Travis. Why not base criticism on actual problems rather than anticipated ones? Perry (full disclosure: one of those selected board members) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 1:55 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. You flatter me, but thanks ;) Over the past 15 months or so, it's been pretty much all Mark. snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the numpy C expertise moving essentially into one company. Then we can formulate some governance that would help lower those barriers and reduce those risks. If we need a formal structure, maybe a good approach is giving Travis the final say for now, until a trigger point occurs. That could be 6 months after the number of active developers hits 5, or something like that. At that point, we would reopen the discussion with a larger group of people who would directly play in that role, and any decision made then will probably be better than a decision we make now while the development team is so small. Honestly - as I was saying to Alan and indirectly to Ben - any formal model - at all - is preferable to the current situation. Personally, I would say that making the founder of a company, which is working to make money from Numpy, the only decision maker on numpy - is - scary. But maybe it's the best way. But, again, we're all high-functioning sensible people, I'm sure it's possible for us to formulate what the risks are, what the potential solutions are, and come up with the best - maybe short-term - solution, See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 12:45 PM, Alan G Isaac alan.is...@gmail.com wrote: My analysis is fundamentally different than Matthew and Benjamin's for a few reasons. 1. The problem has been miscast. The economic interests of the developers *always* has had an apparent conflict with the economic interests of the users: users want developers to work more on the code, and developers need to make a living, which often involves spending their time on other things. On this score, nothing has really changed. 2. It seems pretty clear that Matthew wants some governance power to be held by individuals who are not actively developing NumPy. As Chuck Harris pointed out long ago, that dog ain't going to hunt. 3. Constitutions can be broken (and are, all the time). Designing a stable institution requires making it in the interests of the members to participate. Any formal governance structure that can be desirable for the NumPy community as a whole has to be desirable for the core developers. The right way to produce a governance structure is to make concrete proposals and show how these proposals are in the interest of the *developers* (as well as of the users). For example, Benjamin obliquely suggested that with an appropriate governance board, the NA discussion could have simply been shut down by having the developers vote (as part of their governance). This might be in the interest of the developers and of the community (I'm not sure), but I doubt it is what Matthew has in mind. In any case, until proposals are put on the table along with a clear effort to illustrate why it is in the interest of the *developers* to adopt the proposals, I really do not see this discussion moving forward. That's helpful, it would be good to discuss concrete proposals. Would you care to flesh out your proposal in more detail or is it as you quoted it before? Where do you stand on the desirability of consensus? Do you have any suggestions on how to ensure that the non-Continuum community has sufficient weight in decision making? Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 4:43 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:45 PM, Alan G Isaac alan.is...@gmail.com wrote: My analysis is fundamentally different than Matthew and Benjamin's for a few reasons. 1. The problem has been miscast. The economic interests of the developers *always* has had an apparent conflict with the economic interests of the users: users want developers to work more on the code, and developers need to make a living, which often involves spending their time on other things. On this score, nothing has really changed. 2. It seems pretty clear that Matthew wants some governance power to be held by individuals who are not actively developing NumPy. As Chuck Harris pointed out long ago, that dog ain't going to hunt. 3. Constitutions can be broken (and are, all the time). Designing a stable institution requires making it in the interests of the members to participate. Any formal governance structure that can be desirable for the NumPy community as a whole has to be desirable for the core developers. The right way to produce a governance structure is to make concrete proposals and show how these proposals are in the interest of the *developers* (as well as of the users). For example, Benjamin obliquely suggested that with an appropriate governance board, the NA discussion could have simply been shut down by having the developers vote (as part of their governance). This might be in the interest of the developers and of the community (I'm not sure), but I doubt it is what Matthew has in mind. In any case, until proposals are put on the table along with a clear effort to illustrate why it is in the interest of the *developers* to adopt the proposals, I really do not see this discussion moving forward. That's helpful, it would be good to discuss concrete proposals. Would you care to flesh out your proposal in more detail or is it as you quoted it before? Where do you stand on the desirability of consensus? Do you have any suggestions on how to ensure that the non-Continuum community has sufficient weight in decision making? I'm going to miss Ralf as release manager, since in terms of governance he had the last control over what's actually in the released versions of numpy (and scipy). (I think the ABI breakage in 1.4 not 1.5 was pretty painful for a long time.) Josef Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 12:45 PM, Alan G Isaac alan.is...@gmail.com wrote: for the core developers. The right way to produce a governance structure is to make concrete proposals and show how these proposals are in the interest of the *developers* (as well as of the users). At this point, it seems to me that Matthew is simply trying to make the case that ==some== governance structure should be (or should be more clearly) set up. Even this fairly modest goal seems to be receiving blowback from some. Perhaps a specific proposal would be more convincing to those in opposition, but I'd like to think that the merits for a governance structure could be appreciated without having specific the details of what such a structure would look like. Matthew's links certainly make a good case, IMO. He has also described a number of scenarios where a governance structure would be helpful (even if we think the likelihood of such scenarios is small). ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 4:08 PM, T J tjhn...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:45 PM, Alan G Isaac alan.is...@gmail.comwrote: for the core developers. The right way to produce a governance structure is to make concrete proposals and show how these proposals are in the interest of the *developers* (as well as of the users). At this point, it seems to me that Matthew is simply trying to make the case that ==some== governance structure should be (or should be more clearly) set up. Even this fairly modest goal seems to be receiving blowback from some. Perhaps a specific proposal would be more convincing to those in opposition, but I'd like to think that the merits for a governance structure could be appreciated without having specific the details of what such a structure would look like. Matthew's links certainly make a good case, IMO. He has also described a number of scenarios where a governance structure would be helpful (even if we think the likelihood of such scenarios is small). Agreed. During the NA discussion, I remember at one point there was an attempt to create a governance structure to resolve the dispute. I pushed back on that idea at the time because we would be trying to form a governance while in heated arguments. I would like to see a very basic structure established and agreed upon now, while heads are cool and the pressure isn't on (relatively speaking). The point of these structures are for the unanticipated situations. Following the Boy Scouts motto: Be Prepared! Cheers! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. There hasn't really been any discussion about recruiting developers to compete with the governance topic, now we can let the topics compete. :) Some of the mechanisms which will help are already being set in motion through the discussion about better infrastructure support like bug trackers and continuous integration. The forthcoming roadmap discussion Travis alluded to, where we will propose a roadmap for review by the numpy user community, will include many more such points. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the numpy C expertise moving essentially into one company. Then we can formulate some governance that would help lower those barriers and reduce those risks. There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. The only way to reasonably mitigate the risk of development expertise being in one company is to recruit more developers from elsewhere. I think those new developers will appreciate having a say about how governance works, which is why I suggested to postpone the meat of this discussion with an interim solution, and move on to the recruitment topic. If we need a formal structure, maybe a good approach is giving Travis the final say for now, until a trigger point occurs. That could be 6 months after the number of active developers hits 5, or something like that. At that point, we would reopen the discussion with a larger group of people who would directly play in that role,
Re: [Numpy-discussion] Numpy governance update
On Feb 15, 2012, at 3:36 PM, Matthew Brett wrote: Honestly - as I was saying to Alan and indirectly to Ben - any formal model - at all - is preferable to the current situation. Personally, I would say that making the founder of a company, which is working to make money from Numpy, the only decision maker on numpy - is - scary. How is this different from the situation of the last 4 years? Travis was President at Enthought, which makes money from not only Numpy but SciPy as well. In addition to employing Travis, Enthought also employees many other key contributors to Numpy and Scipy, like Robert and David. Furthermore, the Scipy and Numpy mailing lists and repos and web pages were all hosted at Enthought. If they didn't like how a particular discussion was going, they could have memory-holed the entire conversation from the archives, or worse yet, revoked commit access and reverted changes. But such things never transpired, and of course most of us know that such things would never happen. I don't see why the current situation is any different from the previous situation, other than the fact that Travis actually plans on actively developing Numpy again, and that hardly seems scary. -Peter ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 2:30 PM, Peter Wang pw...@streamitive.com wrote: On Feb 15, 2012, at 3:36 PM, Matthew Brett wrote: Honestly - as I was saying to Alan and indirectly to Ben - any formal model - at all - is preferable to the current situation. Personally, I would say that making the founder of a company, which is working to make money from Numpy, the only decision maker on numpy - is - scary. How is this different from the situation of the last 4 years? Travis was President at Enthought, which makes money from not only Numpy but SciPy as well. In addition to employing Travis, Enthought also employees many other key contributors to Numpy and Scipy, like Robert and David. The difference is fairly obvious to me, but stop me if I'm wrong. First - although Enthought was in a position to influence numpy development, it didn't very much, partly, I suppose because Travis did not have time to contribute to numpy. The exception is of course the masked array stuff by Mark that caused a lot of controversy. Furthermore, the Scipy and Numpy mailing lists and repos and web pages were all hosted at Enthought. If they didn't like how a particular discussion was going, they could have memory-holed the entire conversation from the archives, or worse yet, revoked commit access and reverted changes. Obviously we should be realistic about the risks. Situations like that are very unlikely. But such things never transpired, and of course most of us know that such things would never happen. Right. I don't see why the current situation is any different from the previous situation, other than the fact that Travis actually plans on actively developing Numpy again, and that hardly seems scary. It would be silly to be worried about Travis contributing to numpy, in general. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 2/15/12 3:25 PM, Matthew Brett wrote: 4) It is possible for Continuum to want features that are good for Continuum, but bad for the code-base in general. For example, Continuum may have some product that requires a particular arcane feature in numpy. Through these mechanisms, Numpy can lose developers and commitment (please note) *relative to the situation where there is formal governance*. Obviously, at worst, this can lead to a split. We can avoid that *risk* with a sensible governance model that is satisfactory for all parties. I'm sure that's achievable. Hi All, I'm one of the Continuum devs tasked with contributing to numpy core. I have experience as a numpy user in the past but the core C code is new to me, and getting familiar with it has been an enlightening experience, to say the least. One of our primary long term goals is to make the core codebase much cleaner and more modular. The outcome we expect and hope for as a result of this effort are: 1) Encourage more core developers to join numpy because the codebase is more approachable (I hope we are all everyone agreed that it is very desirable to attract more core devs) 2) Allow the development of new types and features to have some relative insulation from one another and from numpy core Increased modularity mitigates the risk of any conflicts between Continuum and the numpy community (if any should ever actually arise), and reduces the chance of a split. Having more core devs spreads around the responsibility for making decisions while still vesting that responsibility largely among the folks actually contributing their time and effort. Perhaps then the most important question is how to get to a cleaned up, modular numpy core? The details of that roadmap should definitely be hashed out here on the list. But if we can get to that state, I think everyone can pursue both their shared and individual interests comfortably, regardless of what type of formal or informal governance might be adopted in the future. BTW I'd also just like to take this chance to say Hello to the list, I am very excited to help improve numpy. Bryan Van de Ven ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 1:00 PM, Perry Greenfield pe...@stsci.edu wrote: On Feb 15, 2012, at 3:01 PM, Matthew Brett wrote: [...] My 2 cents. [...] I am both elated and concerned. Since it's obvious what there is to be elated about, this post has a concerned tone. But overall, I think this is a great move with some obvious problems to solve before it moves forward. I think the effort spent now looking for a solution will be much less than the pain of trying to undo a mess involving contracts and clients. That way lies court and code splits. So, let's do the hard work of figuring out governance now. In principle, I agree with Matt. We're moving from pretty altruistic code development to a model in which the most active development will be paid for and therefore controlled by influences outside the user community. This can have a lot of unintended side effects, including those Matt pointed out. We might also feel some level of discontent among developers who are not paid vs. those who are. This might make it hard to recruit developers who are not Continuum employees. There are tons of examples of financial interests creeping in and mucking up community computer projects. Symbolics vs. Lisp Machines, Inc. Early shenanigans on internet technical committees by engineers working at companies that were behind the curve on product development. Etc. As for being responsive to the community, Continuum is already promising the world whole new directions in numpy (see continuum.io). Were those plans even mentioned on a mailing list? Is that the direction we want to go in? Are there negative consequeces to those plans? What are the plans, exactly? Having those who do the work make the decisions only works when those working consider the needs of all. The community believes that's true largely when it's included in the discussion and not taken by surprise, as seems to have happened here. Of course, balancing all of this (and our security blanket) is the possibility of someone splitting the code if they don't like how Continuum runs things. Perry, you've done that yourself to this code's predecessor, so you know the risks. You did that in response to one constituency's moving the code in a direction you didn't like (or not moving it in one you did, I don't remember exactly), as in your example #2. So, while progress might be made when that happens, last time it hurt astronomers enough that you rolled your own and had to put several FTE on the problem. That split held back adoption of numpy both in the astronomy community and outside it, for like 5 years. Perhaps some governance would have saved you the effort and cost and the community the grief of the numarray split. Of course, lots of good eventually came from the split. I'd like to see at least some serious thought on how to protect the interests of the community under this very different development model. Trying it out and deciding we don't like it later will be a *much* harder thing to sort out. At the same time, the idea of multiplying the number of people actually working, and of having continuous builds and good issue tracking and all the rest, including the enhancements listed on Travis's web site, are very exciting! Let's just make sure we retain our community orientation with this new model. --jh-- ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On 02/15/2012 02:24 PM, Mark Wiebe wrote: On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com mailto:mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu mailto:ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com mailto:alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. There hasn't really been any discussion about recruiting developers to compete with the governance topic, now we can let the topics compete. :) Some of the mechanisms which will help are already being set in motion through the discussion about better infrastructure support like bug trackers and continuous integration. The forthcoming roadmap discussion Travis alluded to, where we will propose a roadmap for review by the numpy user community, will include many more such points. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the numpy C expertise moving essentially into one company. Then we can formulate some governance that would help lower those barriers and reduce those risks. There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 7:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com mailto:mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu mailto:ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com mailto:alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. There hasn't really been any discussion about recruiting developers to compete with the governance topic, now we can let the topics compete. :) Some of the mechanisms which will help are already being set in motion through the discussion about better infrastructure support like bug trackers and continuous integration. The forthcoming roadmap discussion Travis alluded to, where we will propose a roadmap for review by the numpy user community, will include many more such points. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the numpy C expertise moving essentially into one company. Then we can formulate some governance that would help lower those barriers and reduce those risks. There certainly is governance now, it's just informal. It's a combination of how the design discussions are
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com mailto:mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu mailto:ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com mailto:alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. There hasn't really been any discussion about recruiting developers to compete with the governance topic, now we can let the topics compete. :) Some of the mechanisms which will help are already being set in motion through the discussion about better infrastructure support like bug trackers and continuous integration. The forthcoming roadmap discussion Travis alluded to, where we will propose a roadmap for review by the numpy user community, will include many more such points. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the numpy C expertise moving essentially into one company. Then we can formulate some governance that would help lower those barriers and reduce those risks. There certainly is governance now, it's just informal. It's a combination of how the design discussions
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 4:57 PM, josef.p...@gmail.com wrote: On Wed, Feb 15, 2012 at 7:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com mailto:mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu mailto:ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com mailto:alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. There hasn't really been any discussion about recruiting developers to compete with the governance topic, now we can let the topics compete. :) Some of the mechanisms which will help are already being set in motion through the discussion about better infrastructure support like bug trackers and continuous integration. The forthcoming roadmap discussion Travis alluded to, where we will propose a roadmap for review by the numpy user community, will include many more such points. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the numpy C expertise moving essentially into one company. Then we can formulate some
Re: [Numpy-discussion] Numpy governance update
On Wednesday, February 15, 2012, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 4:57 PM, josef.p...@gmail.com wrote: On Wed, Feb 15, 2012 at 7:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com mailto:mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu mailto:ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com mailto:alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) This is very true, at the moment the number of people doing feature-work within numpy purely in Python is similarly small and sporadic. Here's a current example: https://github.com/numpy/numpy/pull/198 Having such a small development core is one of the reasons it often takes a while for such pull requests to get reviewed by someone, and a situation Continuum and anyone else with resources to contribute can help improve. One thing that's clear to me is that the current documentation on how to contribute code, documentation, and other help to NumPy is lacking, and this is something that needs improvement. An example I really like is LibreOffice's get involved page. http://www.libreoffice.org/get-involved/ Producing something similar for NumPy will take some work, but I believe it's needed. Cheers, Mark +1000. Each time I have submitted a pull request, I always had to ask where the appropriate tests go. I still haven't gotten my head around the layout of the source tree, and my only saving grace has been 'git grep'. It is the little things that can keep someone from contributing. Anything to make this easier would be great. Maybe a protege system might be nice? Chuck ain't getting younger, ya'll! Ben Root P.S. - who knows? Maybe I will be one of those protoges depending on how my new job unfolds. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. In the spirit (as I read) of Dag's post, maybe we should accept that this thread is not going anywhere much, and summarize: The current situation is the following: Travis is de-facto BDFL for Numpy Disputes get resolved by convening an ad-hoc group of interested and / or active developers to resolve or vote, maybe off-list. How this happens is for Travis to call. I think that's reasonable? As far as I can make out, in favor of the current status quo with no significant modification are: Travis (is that right)? Mark Peter Bryan vdv Perry Dag In favor of some sort of formalization of governance to be decided are: Me Ben R (did I get that right?) Bruce Southey Souheil Inati TJ Joe H I am not quite sure which side of that fence are: Josef Alan Chuck If I missed someone who gave an opinion - sorry - please do speak up. I think it's clear that if - you, Travis, don't want to go this direction, there isn't much chance of anything happening, and I think those of us who think something needs doing will have to keep quiet, as Dag suggests. I would only suggest that you (Travis) specify that you will take the BDFL role so that we can be clear about the informal governance at least. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 8:49 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. In the spirit (as I read) of Dag's post, maybe we should accept that this thread is not going anywhere much, and summarize: The current situation is the following: Travis is de-facto BDFL for Numpy Disputes get resolved by convening an ad-hoc group of interested and / or active developers to resolve or vote, maybe off-list. How this happens is for Travis to call. I think that's reasonable? As far as I can make out, in favor of the current status quo with no significant modification are: Travis (is that right)? Mark Peter Bryan vdv Perry Dag In favor of some sort of formalization of governance to be decided are: Me Ben R (did I get that right?) Bruce Southey Souheil Inati TJ Joe H I am not quite sure which side of that fence are: Josef Actually in the sense of separation of powers, I would vote for Chuck as president, Travis as prime minister and an independent release manager as supreme court, and the noisy mailing list community as parliament. (I don't see a constitution yet.) Josef Alan Chuck If I missed someone who gave an opinion - sorry - please do speak up. I think it's clear that if - you, Travis, don't want to go this direction, there isn't much chance of anything happening, and I think those of us who think something needs doing will have to keep quiet, as Dag suggests. I would only suggest that you (Travis) specify that you will take the BDFL role so that we can be clear about the informal governance at least. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 6:07 PM, josef.p...@gmail.com wrote: On Wed, Feb 15, 2012 at 8:49 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. In the spirit (as I read) of Dag's post, maybe we should accept that this thread is not going anywhere much, and summarize: The current situation is the following: Travis is de-facto BDFL for Numpy Disputes get resolved by convening an ad-hoc group of interested and / or active developers to resolve or vote, maybe off-list. How this happens is for Travis to call. I think that's reasonable? As far as I can make out, in favor of the current status quo with no significant modification are: Travis (is that right)? Mark Peter Bryan vdv Perry Dag In favor of some sort of formalization of governance to be decided are: Me Ben R (did I get that right?) Bruce Southey Souheil Inati TJ Joe H I am not quite sure which side of that fence are: Josef Actually in the sense of separation of powers, I would vote for Chuck as president, Travis as prime minister and an independent release manager as supreme court, and the noisy mailing list community as parliament. That sounds dangerously Canadian ... But actually - I was hoping for an answer to whether you felt there was a need for a more formal governance structure, or not. (I don't see a constitution yet.) My feeling is there is not enough appetite for any change for that to be worth thinking about, but I might be wrong. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wednesday, February 15, 2012, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. In the spirit (as I read) of Dag's post, maybe we should accept that this thread is not going anywhere much, and summarize: The current situation is the following: Travis is de-facto BDFL for Numpy Disputes get resolved by convening an ad-hoc group of interested and / or active developers to resolve or vote, maybe off-list. How this happens is for Travis to call. I think that's reasonable? As far as I can make out, in favor of the current status quo with no significant modification are: Travis (is that right)? Mark Peter Bryan vdv Perry Dag In favor of some sort of formalization of governance to be decided are: Me Ben R (did I get that right?) Bruce Southey Souheil Inati TJ Joe H I am not quite sure which side of that fence are: Josef Alan Chuck If I missed someone who gave an opinion - sorry - please do speak up. Yes, you got my opinion right (don't know how it was ambiguous. Do I really equivocate that much?). I will note that I am fine with a very light-handed form of governance. The most important thing is that it is agreed upon. That means that when the time comes to solidify the details, we start a new thread and invite members to contribute. Then, when that is finalized, we start a *new* thread and ask users to vote for or against. More complicated governance structures can come later, building off of the existing system -- if desired. Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 9:12 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 6:07 PM, josef.p...@gmail.com wrote: On Wed, Feb 15, 2012 at 8:49 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. In the spirit (as I read) of Dag's post, maybe we should accept that this thread is not going anywhere much, and summarize: The current situation is the following: Travis is de-facto BDFL for Numpy Disputes get resolved by convening an ad-hoc group of interested and / or active developers to resolve or vote, maybe off-list. How this happens is for Travis to call. I think that's reasonable? As far as I can make out, in favor of the current status quo with no significant modification are: Travis (is that right)? Mark Peter Bryan vdv Perry Dag In favor of some sort of formalization of governance to be decided are: Me Ben R (did I get that right?) Bruce Southey Souheil Inati TJ Joe H I am not quite sure which side of that fence are: Josef Actually in the sense of separation of powers, I would vote for Chuck as president, Travis as prime minister and an independent release manager as supreme court, and the noisy mailing list community as parliament. That sounds dangerously Canadian ... Or Austrian or German But actually - I was hoping for an answer to whether you felt there was a need for a more formal governance structure, or not. I thought a president, a prime minister and a parliament makes for a formal government structure. :) maybe more personalized in the American tradition. I'm in favor of a more formal governance structure, however the only real enforcement I see is in the reputation and goodwill, if all keys are in one hand. I think spelling out both governance and guidelines for development and testing make it easier to make it clear what we can expect and so that we know when we should be upset (a bit of repeated game enforcement since I'm an economist). I have no idea how formal governance structures work in open source. Actually, I liked the recent situation with a visionary 2.0 sometimes in the future, while Chuck and Ralf kept putting out 1.x releases with careful control of going forward and not breaking anything (I'm not sure how to phrase this), with, of course, Mark doing large parts of the heavy work. Josef (I don't see a constitution yet.) My feeling is there is not enough appetite for any change for that to be worth thinking about, but I might be wrong. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 6:57 PM, josef.p...@gmail.com wrote: On Wed, Feb 15, 2012 at 7:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com mailto:mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu mailto:ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com mailto:alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. There hasn't really been any discussion about recruiting developers to compete with the governance topic, now we can let the topics compete. :) Some of the mechanisms which will help are already being set in motion through the discussion about better infrastructure support like bug trackers and continuous integration. The forthcoming roadmap discussion Travis alluded to, where we will propose a roadmap for review by the numpy user community, will include many more such points. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the numpy C expertise moving essentially into one company. Then we can formulate some governance that would help lower those barriers and reduce those risks. There certainly is governance now, it's just
Re: [Numpy-discussion] Numpy governance update
On 02/15/2012 05:02 PM, Matthew Brett wrote: Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. Ouch. Is that me, one of the non-contributing users? Was I suggesting that we set up a system to select non-developers to a board? I must say, now you mention it, I do feel a bit ridiculous. In retrospect I was unfair and my email way too harsh. Anyway, I'm really happy with your follow-up in turning this into something more constructive. And I do appreciate that you brought the topic up in the first place. It's obvious that one should try for consensus as long as possible, including listening to users. But in the very end, when agreement can't be reached by other means, the developers are the one making the calls. (This is simply a consequence that they are the only ones who can credibly threaten to fork the project.) I think the following are not in question: 1) Consensus is desirable 2) Developers need to have the final say. But, I think it is clear in the both the ABI numpy 1.5.0 dispute, and the mask / NA dispute, that we could have gone further in negotiating to consensus. The question we're considering here is whether there is any way of setting up a set of guidelines or procedures that would help us work at and reach consensus. Or if we don't reach consensus, finding a way to decide that is clear and fair. I don't think working on that seems as silly and / or rude to me as it does to you. Sure, structures that includes users in the process could be useful... but, if the devs are fine with the current situation (and I don't see Mark or Charles complaining), then I honestly think it is quite rude to not let the matter drop after the first ten posts or so. I have clearly overestimated my own importance and wisdom, and have made myself appear foolish in the eyes of my peers. Making things the way one wants it and scratching *ones own* itch is THE engine of open source development (whether one is putting in spare time or monetary funding). Trying to work against that with artificial structures doesn't sound wise for a project with as few devs as NumPy... You believe, I suppose, that there are no significant risks in nearly all the numpy core development being done by a new company, or at least, that there can little benefit to a governance discussion in that situation. I think you are wrong, but of course it's a tenable point of view, The question is more about what can possibly be done about it. To really shift power, my hunch is that the only practical way would be to, like Mark said, make sure there are very active non-Continuum-employed developers. But perhaps I'm wrong. Sometimes it is worth taking some risks because it means one can go forward faster. Possibly *a lot* faster, if one shifts things from email to personal communication. It is not like the current versions of NumPy disappear. If things do go wrong and NumPy is developed in some crazy direction, it's easy to go for the stagnated option simply by taking the current release and maintain bugfixes on it. Dag ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
Hi, On Wed, Feb 15, 2012 at 9:47 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 05:02 PM, Matthew Brett wrote: Hi, On Wed, Feb 15, 2012 at 4:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: There certainly is governance now, it's just informal. It's a combination of how the design discussions are carried out, how pull requests occur, and who has commit rights. +1 If non-contributing users came along on the Cython list demanding that we set up a system to select non-developers along on a board that would have discussions in order to veto pull requests, I don't know whether we'd ignore it or ridicule it or try to show some patience, but we certainly wouldn't take it seriously. Ouch. Is that me, one of the non-contributing users? Was I suggesting that we set up a system to select non-developers to a board? I must say, now you mention it, I do feel a bit ridiculous. In retrospect I was unfair and my email way too harsh. Anyway, I'm really happy with your follow-up in turning this into something more constructive. Don't worry - thanks for this reply. You believe, I suppose, that there are no significant risks in nearly all the numpy core development being done by a new company, or at least, that there can little benefit to a governance discussion in that situation. I think you are wrong, but of course it's a tenable point of view, The question is more about what can possibly be done about it. To really shift power, my hunch is that the only practical way would be to, like Mark said, make sure there are very active non-Continuum-employed developers. But perhaps I'm wrong. It's not obvious to me that there isn't a set of guidelines, procedures, structures that would help to keep things clear in this situation. Obviously it would be good to have more non-Continuum developers, but also obviously, there is a risk that that won't happen. Sometimes it is worth taking some risks because it means one can go forward faster. Possibly *a lot* faster, if one shifts things from email to personal communication. Yes, obviously it's in no-one's interest to slow down the Continuum developers. I wonder though whether there is a way of organizing things, that does not slow down the Continuum developers, but does keep the sense of community involvement and ownership. It is not like the current versions of NumPy disappear. If things do go wrong and NumPy is developed in some crazy direction, it's easy to go for the stagnated option simply by taking the current release and maintain bugfixes on it. But we all want to avoid a fork, which is what that could easily become. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
On Tue, Feb 14, 2012 at 5:17 PM, Travis Oliphant tra...@continuum.io wrote: Your points are well taken. However, my point is that this has been discussed on an open mailing list. Things weren't *as* open as they could have been, perhaps, in terms of board selection. But, there was opportunity for people to provide input. I am on the numpy, scipy, matplotlib, ipython and cython mailing lists. Jarrod and Fernando are friends of mine. I've been obviously concerned about numpy governance for some time. I didn't know about this mailing list, had only a vague idea that some sort of foundation was being proposed and I had no idea at all that you'd selected a board. Would you say that was closer to 'open' or closer to 'closed'? I see it a different way. First, the Foundation is not a NumPy-governance thing. Certainly it could grow in that direction over time, but it isn't there now, nor is that its goal. Second, the Foundation is just getting started. It's only come together over the past couple of weeks. The fact that we are talking about it now, seems to me to indicate that it is quite open --- certainly closer to 'open' then you seem to imply. Also, the fact that there was a public mailing list for its discussion certainly sounds open to me (poorly advertised I will grant you). I tried to include as many people as I thought were interested by the responses to the initial emails on the list. I reached out to people that contacted me expressing their interest, and included them on the mailing list. I can accept that I made mistakes. I can guarantee that I will make more. Your feedback is appreciated and noted. The fact is that the Foundation is really a service organization that will require a lot of work to run and administer. It's effectiveness at fulfilling its mission will depend on how well it serves the group on this list, as well as the other groups that are working on Python for Science. I'm all for getting as many volunteers as we can get for the Foundation. I've just been trying to get things organized. Sometimes this works best by phone calls and direct contact, rather than mailing lists. For those interested. The Foundation mission is to: * Promote Open Source Software for Science * Fund Open Source Projects in Science (currently NumPy, SciPy, IPython, and Matplotlib are first-tier with a whole host of second-tier projects that could received funding) * through grants * through code bounties * through graduate-student scholarships * Sponsor sprints * Sponsor conferences * Sponsor student travel * etc., etc. Whether or not it can do any of those things depends on whether or not it can raise money from people and organizations that benefit from the Scientific Python Stack. All of this will be advertised more as the year progresses. This sounds really exciting. I'm looking forward to seeing what you, Mark, et al release over the next year. -Chris Jordan-Squire Best regards, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 5:26 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 4:57 PM, josef.p...@gmail.com wrote: On Wed, Feb 15, 2012 at 7:27 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 02/15/2012 02:24 PM, Mark Wiebe wrote: On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe mwwi...@gmail.com mailto:mwwi...@gmail.com wrote: On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root ben.r...@ou.edu mailto:ben.r...@ou.edu wrote: On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac alan.is...@gmail.com mailto:alan.is...@gmail.com wrote: Can you provide an example where a more formal governance structure for NumPy would have meant more or better code development? (Please do not suggest the NA discussion!) Why not the NA discussion? Would we really want to have that happen again? Note that it still isn't fully resolved and progress still needs to be made (I think the last thread did an excellent job of fleshing out the ideas, but it became too much to digest. We may need to have someone go through the information, reduce it down and make one last push to bring it to a conclusion). The NA discussion is the perfect example where a governance structure would help resolve disputes. Yes, that was the most obvious example. I don't know about you, but I can't see any sign of that one being resolved. The other obvious example was the dispute about ABI breakage for numpy 1.5.0 where I believe Travis did invoke some sort of committee to vote, but (Travis can correct me if I'm wrong), the committee was named ad-hoc and contacted off-list. Can you provide an example of what you might envision as a more formal governance structure? (I assume that any such structure will not put people who are not core contributors to NumPy in a position to tell core contributors what to spend their time on.) Early last December, Chuck Harris estimated that three people were active NumPy developers. I liked the idea of creating a board of these 3 and a rule that says any active developer can request to join the board, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority. I also suggested inviting to this board a developer or two from important projects that are very NumPy dependent (e.g., Matplotlib). I still like this idea. Would it fully satisfy you? I actually like that idea. Matthew, is this along the lines of what you were thinking? Honestly it would make me very happy if the discussion moved to what form the governance should take. I would have thought that 3 was too small a number. One thing to note about this point is that during the NA discussion, the only people doing active C-level development were Charles and me. I suspect a discussion about how to recruit more people into that group might be more important than governance at this point in time. Mark - a) thanks for replying, it's good to hear your voice and b) I don't think there's any competition between the discussion about governance and the need to recruit more people into the group who understand the C code. There hasn't really been any discussion about recruiting developers to compete with the governance topic, now we can let the topics compete. :) Some of the mechanisms which will help are already being set in motion through the discussion about better infrastructure support like bug trackers and continuous integration. The forthcoming roadmap discussion Travis alluded to, where we will propose a roadmap for review by the numpy user community, will include many more such points. Remember we are deciding here between governance - of a form to be decided - and no governance - which I think is the current situation. I know your desire is to see more people contributing to the C code. It would help a lot if you could say what you think the barriers are, how they could be lowered, and the risks that you see as a result of the
Re: [Numpy-discussion] Numpy governance update
On Wed, Feb 15, 2012 at 10:30 PM, Peter Wang pw...@streamitive.com wrote: On Feb 15, 2012, at 3:36 PM, Matthew Brett wrote: Honestly - as I was saying to Alan and indirectly to Ben - any formal model - at all - is preferable to the current situation. Personally, I would say that making the founder of a company, which is working to make money from Numpy, the only decision maker on numpy - is - scary. How is this different from the situation of the last 4 years? Travis was President at Enthought, which makes money from not only Numpy but SciPy as well. In addition to employing Travis, Enthought also employees many other key contributors to Numpy and Scipy, like Robert and David. Furthermore, the Scipy and Numpy mailing lists and repos and web pages were all hosted at Enthought. If they didn't like how a particular discussion was going, they could have memory-holed the entire conversation from the archives, or worse yet, revoked commit access and reverted changes. I actually think it is somehow different. For once, while Travis was at Enthought, he contributed much less to the discussions (by his own account), so the risk of conflict of interest was not very high. My own contributions to numpy since I have joined Enthought are close to nil as well :) There have been cases of disagreements on NumPy: for any case where the decision taken by people from one company would prevail, you will not be able to prevent people from thinking the interests of the company prevailed. In numpy, where people make a suggestion and there was not enough review, the feature generally went in. This is fundamentally different from most open source projects I am aware of, and could go bad when considered with my previous point. As far as I am concerned, the following would be enough to resolve any issues: - having one (or more) persons outside any company interest (e.g. Chuck, Pauli) with a veto. - no significant feature goes in without a review from people outside the organization it is coming from. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
There is a mailing list for numfocus that you can sign up for if you would like to be part of those discussions. Let me know if you would like more information about that.John Hunter, Fernando Perez, me, Perry Greenfield, and Jarrod Millman are the initial board of the Foundation. But, I expect the Foundation directors to evolve over time. I should say that I have no knowledge of the events above other than from the mailing list (I say that only because some of you may know that I'm a friend and colleague of Jarrod and Fernando). Thanks for speaking up, Matthew. I knew that this was my first announcement of the Foundation to this list. Things are still just starting around that organization, and so there is plenty of time for input. This sort of thing has actually been under-way for a long time --- it just has not received much impetus until now for one reason or another. To be clear, there were several email posts about a Foundation to this list last fall and we took the discussion of the Foundation that has really been in the works for a couple of years (thanks to Jarrod), to a Google Group (very poorly) called Fastechula.There were 33 people who signed up for that list and discussions continued sporadically on that list away from this one. When we selected the name NumFOCUS just a few weeks ago, we created the list for numfocus and then I signed everyone up for that list who was on the other one. I apologize if anyone felt left out. That is not my intention. But, I also did not want to consume this mailing list with something that might be considered off-topic. I repeat that there is still plenty time for input. Obviously, the board has been selected. But that must be done by someone. I took the liberty to invite the first board members who graciously accepted the assignment. I consider the Foundation a service opportunity. I'm grateful that representatives from the major projects are willing to serve. I expect that to be a tradition, but it is one that needs to be discussed and developed. Yes, I have started a new company with Peter Wang. However, most of the people on this list will probably be most interested in the NumFOCUS work. The goal of the Foundation is to promote the entire Scientific Computing with Python ecosystem. It will not be taking over any of the public mailing lists where there is already a great deal of opportunity to express opinions and desires. The Foundation will have it's own public mailing list where mostly financial and funding matters that are common to all of the projects can be sent and discussed. Go here and sign up for the public mailing list if you are interested in the Foundation: http://groups.google.com/group/numfocus?hl=en We will be discussing the Foundation at PyCon as well. This may also mean different business models and licensing around some of the NumPy-related code that the company writes. Obviously your company will need to make enough money to cover your salaries and more. There is huge potential here for clashes of interest, and for perceived clashes of interest. The perceived clashes are just as damaging as the actual clashes. Perceptions can be damaging. This is one of the big reasons for the organization of the Foundation -- to be a place separate from any commercial venture which can direct resources to a vision whose goal is more democratically determined.I trust that people will observe results and come to expect good things that will naturally emerge by having more talented people involved in the process who are being directed by the community needs. I still don't think we've got a Numpy steering group. The combination of the huge concentration of numpy resources in your company, and a lack of explicit community governance, seems to me to be something that needs to be fixed urgently. Do you agree? I'm sensitive to the perception that some might have that Continuum might hi-jack NumPy. That is the central reason I am very supportive of and pushing the organization of NumFOCUS. I want corporate dollars that flow to NumPy to have some buffering between the money that is being spent and what is promoted. This can be a delicate situation, but I think it can also work well. RedHat, IBM, and Google all cooperate to make Linux better through the Linux Foundation. The same needs to be the case with NumPy. This will depend, of course, on everybody on this list and they way they receive new input and the way they communicate with each other. I think we do have a NumPy steering group if you want to call it that. It is currently me, Mark Wiebe, and Charles Harris.Rolf Gommers, Pauli Virtanen, David Cournapeau and Robert Kern also have opinions that carry significant weight.Are there other people that should be on this list? There are other people who also speak up on this
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
Hi, On Tue, Feb 14, 2012 at 1:54 PM, Travis Oliphant tra...@continuum.io wrote: There is a mailing list for numfocus that you can sign up for if you would like to be part of those discussions. Let me know if you would like more information about that. John Hunter, Fernando Perez, me, Perry Greenfield, and Jarrod Millman are the initial board of the Foundation. But, I expect the Foundation directors to evolve over time. I should say that I have no knowledge of the events above other than from the mailing list (I say that only because some of you may know that I'm a friend and colleague of Jarrod and Fernando). Thanks for speaking up, Matthew. I knew that this was my first announcement of the Foundation to this list. Things are still just starting around that organization, and so there is plenty of time for input. This sort of thing has actually been under-way for a long time --- it just has not received much impetus until now for one reason or another. To be clear, there were several email posts about a Foundation to this list last fall and we took the discussion of the Foundation that has really been in the works for a couple of years (thanks to Jarrod), to a Google Group (very poorly) called Fastechula. There were 33 people who signed up for that list and discussions continued sporadically on that list away from this one. When we selected the name NumFOCUS just a few weeks ago, we created the list for numfocus and then I signed everyone up for that list who was on the other one. I apologize if anyone felt left out. That is not my intention. My point is that there are two ways go to about this process, one is open and the other is closed. In the open version, someone proposes such a group to the mailing lists. They ask for expressions of interest. The discussion might then move to another mailing list that is publicly known and widely advertised. Members of the board are proposed in public. There might be some sort of formal or informal voting process. The reason to prefer this to the more informal private negotiations is that a) the community feels a greater ownership and control of the process and b) it is much harder to weaken or subvert an organization that explicitly does all its business in public. The counter-argument usually goes 'members X, Y and Z are of impeccable integrity and would only do what is best for the public good'. And usually, members X, Y and Z are indeed of impeccable integrity. Nevertheless I'm sure I don't have to unpack the evidence that this approach frequently fails and can fail in a catastrophic way. Perceptions can be damaging. This is one of the big reasons for the organization of the Foundation -- to be a place separate from any commercial venture which can direct resources to a vision whose goal is more democratically determined. Are you proposing that the Foundation oversee Numpy governance and direction? From your chosen members I'm guessing that the idea is for the foundation to think about broad strategy rather than - say - whether missing values should be encoded with masked arrays? I think we do have a NumPy steering group if you want to call it that. It is currently me, Mark Wiebe, and Charles Harris. Rolf Gommers, Pauli Virtanen, David Cournapeau and Robert Kern also have opinions that carry significant weight. Are there other people that should be on this list? There are other people who also speak up on this list whose opinions will be listened to and heard. In fact, I hope that many more people will come to the list and speak out as development increases. The point I was making was that the concentration of numpy development hours and talent in your company makes it urgent that the numpy governance is set out formally, that the interests of the company are made clear, and that the steering group can be assured of explicit and public independence from the interests of the company, if and when that becomes necessary. In the past, the numpy steering group has seemed a virtual organization, formed ad-hoc when needed, and with no formal governance. I'm saying that I firmly believe that has to change, to avoid the actual or perceived loss of community ownership. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
When we selected the name NumFOCUS just a few weeks ago, we created the list for numfocus and then I signed everyone up for that list who was on the other one. I apologize if anyone felt left out. That is not my intention. My point is that there are two ways go to about this process, one is open and the other is closed. In the open version, someone proposes such a group to the mailing lists. They ask for expressions of interest. The discussion might then move to another mailing list that is publicly known and widely advertised. Members of the board are proposed in public. There might be some sort of formal or informal voting process. The reason to prefer this to the more informal private negotiations is that a) the community feels a greater ownership and control of the process and b) it is much harder to weaken or subvert an organization that explicitly does all its business in public. Your points are well taken. However, my point is that this has been discussed on an open mailing list. Things weren't *as* open as they could have been, perhaps, in terms of board selection. But, there was opportunity for people to provide input. Perceptions can be damaging. This is one of the big reasons for the organization of the Foundation -- to be a place separate from any commercial venture which can direct resources to a vision whose goal is more democratically determined. Are you proposing that the Foundation oversee Numpy governance and direction? From your chosen members I'm guessing that the idea is for the foundation to think about broad strategy rather than - say - whether missing values should be encoded with masked arrays? No, I am not proposing that.The Foundation will be focused on higher-level broad strategy sorts of things: mostly around how to raise money and how to direct that money to projects that have their own development cycles. I would think the Foundation would be interested in paying for things like issue trackers and continuous integration servers as well. It will leave NumPy management to this list and the people who have gathered around this watering hole.Obviously, there will be points of connection, but exactly how this will play-out depends on who shows up to both organizations. I think we do have a NumPy steering group if you want to call it that. It is currently me, Mark Wiebe, and Charles Harris.Rolf Gommers, Pauli Virtanen, David Cournapeau and Robert Kern also have opinions that carry significant weight.Are there other people that should be on this list? There are other people who also speak up on this list whose opinions will be listened to and heard. In fact, I hope that many more people will come to the list and speak out as development increases. The point I was making was that the concentration of numpy development hours and talent in your company makes it urgent that the numpy governance is set out formally, that the interests of the company are made clear, and that the steering group can be assured of explicit and public independence from the interests of the company, if and when that becomes necessary. In the past, the numpy steering group has seemed a virtual organization, formed ad-hoc when needed, and with no formal governance. I'm saying that I firmly believe that has to change, to avoid the actual or perceived loss of community ownership. I hear your point.Thank you for sharing it.Fortunately, we are having this discussion, and plan to continue to have it as any concerns arise.I think the situation is actually less concentrated than it used to be when the SciPy steering committee was discussed. On that note, I think the SciPy steering committee needs serious revision as well.But, we've all just been getting along pretty well without too much formalism, so far, so perhaps that is enough for now. Thanks, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
On Tuesday, February 14, 2012, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 14, 2012 at 1:54 PM, Travis Oliphant tra...@continuum.io wrote: There is a mailing list for numfocus that you can sign up for if you would like to be part of those discussions. Let me know if you would like more information about that.John Hunter, Fernando Perez, me, Perry Greenfield, and Jarrod Millman are the initial board of the Foundation. But, I expect the Foundation directors to evolve over time. I should say that I have no knowledge of the events above other than from the mailing list (I say that only because some of you may know that I'm a friend and colleague of Jarrod and Fernando). Thanks for speaking up, Matthew. I knew that this was my first announcement of the Foundation to this list. Things are still just starting around that organization, and so there is plenty of time for input. This sort of thing has actually been under-way for a long time --- it just has not received much impetus until now for one reason or another. To be clear, there were several email posts about a Foundation to this list last fall and we took the discussion of the Foundation that has really been in the works for a couple of years (thanks to Jarrod), to a Google Group (very poorly) called Fastechula.There were 33 people who signed up for that list and discussions continued sporadically on that list away from this one. When we selected the name NumFOCUS just a few weeks ago, we created the list for numfocus and then I signed everyone up for that list who was on the other one. I apologize if anyone felt left out. That is not my intention. My point is that there are two ways go to about this process, one is open and the other is closed. In the open version, someone proposes such a group to the mailing lists. They ask for expressions of interest. The discussion might then move to another mailing list that is publicly known and widely advertised. Members of the board are proposed in public. There might be some sort of formal or informal voting process. The reason to prefer this to the more informal private negotiations is that a) the community feels a greater ownership and control of the process and b) it is much harder to weaken or subvert an organization that explicitly does all its business in public. The counter-argument usually goes 'members X, Y and Z are of impeccable integrity and would only do what is best for the public good'. And usually, members X, Y and Z are indeed of impeccable integrity. Nevertheless I'm sure I don't have to unpack the evidence that this approach frequently fails and can fail in a catastrophic way. Perceptions can be damaging. This is one of the big reasons for the organization of the Foundation -- to be a place separate from any commercial venture which can direct resources to a vision whose goal is more democratically determined. Are you proposing that the Foundation oversee Numpy governance and direction? From your chosen members I'm guessing that the idea is for the foundation to think about broad strategy rather than - say - whether missing values should be encoded with masked arrays? I think we do have a NumPy steering group if you want to call it that. It is currently me, Mark Wiebe, and Charles Harris.Rolf Gommers, Pauli Virtanen, David Cournapeau and Robert Kern also have opinions that carry significant weight.Are there other people that should be on this list? There are other people who also speak up on this list whose opinions will be listened to and heard. In fact, I hope that many more people will come to the list and speak out as development increases. The point I was making was that the concentration of numpy development hours and talent in your company makes it urgent that the numpy governance is set out formally, that the interests of the company are made clear, and that the steering group can be assured of explicit and public independence from the interests of the company, if and when that becomes necessary. In the past, the numpy steering group has seemed a virtual organization, formed ad-hoc when needed, and with no formal governance. I'm saying that I firmly believe that has to change, to avoid the actual or perceived loss of community ownership. Best, Matthew I have to agree with Mathew here, to a point. There has been discussions of these groups before, but I don't recall any announcement of this group. Of course, now that it has been announced, maybe a link to it should be prominent on the numpy/scipy pages(maybe others?). It should also be in the list of mailing lists. A funding org much like the Linux Foundation would be great, and I am all for it. A separate governing committee is also important, and I think we had some very good ideas in previous discussions. I also have to agree with Matthew's concerns about the concentration of
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
Hi, On Tue, Feb 14, 2012 at 3:58 PM, Travis Oliphant tra...@continuum.io wrote: When we selected the name NumFOCUS just a few weeks ago, we created the list for numfocus and then I signed everyone up for that list who was on the other one. I apologize if anyone felt left out. That is not my intention. My point is that there are two ways go to about this process, one is open and the other is closed. In the open version, someone proposes such a group to the mailing lists. They ask for expressions of interest. The discussion might then move to another mailing list that is publicly known and widely advertised. Members of the board are proposed in public. There might be some sort of formal or informal voting process. The reason to prefer this to the more informal private negotiations is that a) the community feels a greater ownership and control of the process and b) it is much harder to weaken or subvert an organization that explicitly does all its business in public. Your points are well taken. However, my point is that this has been discussed on an open mailing list. Things weren't *as* open as they could have been, perhaps, in terms of board selection. But, there was opportunity for people to provide input. I am on the numpy, scipy, matplotlib, ipython and cython mailing lists. Jarrod and Fernando are friends of mine. I've been obviously concerned about numpy governance for some time. I didn't know about this mailing list, had only a vague idea that some sort of foundation was being proposed and I had no idea at all that you'd selected a board. Would you say that was closer to 'open' or closer to 'closed'? Perceptions can be damaging. This is one of the big reasons for the organization of the Foundation -- to be a place separate from any commercial venture which can direct resources to a vision whose goal is more democratically determined. Are you proposing that the Foundation oversee Numpy governance and direction? From your chosen members I'm guessing that the idea is for the foundation to think about broad strategy rather than - say - whether missing values should be encoded with masked arrays? No, I am not proposing that. The Foundation will be focused on higher-level broad strategy sorts of things: mostly around how to raise money and how to direct that money to projects that have their own development cycles. I would think the Foundation would be interested in paying for things like issue trackers and continuous integration servers as well. It will leave NumPy management to this list and the people who have gathered around this watering hole. Obviously, there will be points of connection, but exactly how this will play-out depends on who shows up to both organizations. I think we do have a NumPy steering group if you want to call it that. It is currently me, Mark Wiebe, and Charles Harris. Rolf Gommers, Pauli Virtanen, David Cournapeau and Robert Kern also have opinions that carry significant weight. Are there other people that should be on this list? There are other people who also speak up on this list whose opinions will be listened to and heard. In fact, I hope that many more people will come to the list and speak out as development increases. The point I was making was that the concentration of numpy development hours and talent in your company makes it urgent that the numpy governance is set out formally, that the interests of the company are made clear, and that the steering group can be assured of explicit and public independence from the interests of the company, if and when that becomes necessary. In the past, the numpy steering group has seemed a virtual organization, formed ad-hoc when needed, and with no formal governance. I'm saying that I firmly believe that has to change, to avoid the actual or perceived loss of community ownership. I hear your point. Thank you for sharing it. Fortunately, we are having this discussion, and plan to continue to have it as any concerns arise. I think the situation is actually less concentrated than it used to be when the SciPy steering committee was discussed. On that note, I think the SciPy steering committee needs serious revision as well. But, we've all just been getting along pretty well without too much formalism, so far, so perhaps that is enough for now. But a) there have already been serious unresolved disagreements on this list (I note no resolution of the masks / NA debate) and b) the whole point is to set up structures that can deal with the problems before or as they arise. After the problem arises, it is too late. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
I have to agree with Mathew here, to a point. There has been discussions of these groups before, but I don't recall any announcement of this group. Of course, now that it has been announced, maybe a link to it should be prominent on the numpy/scipy pages(maybe others?). It should also be in the list of mailing lists. I'm happy for all these discussions to be in the open. A funding org much like the Linux Foundation would be great, and I am all for it. A separate governing committee is also important, and I think we had some very good ideas in previous discussions. I also have to agree with Matthew's concerns about the concentration of developer resources at Continuum. I think that establishing a community-driven governance committee would be crucial in making sure that Continuum's (and Enthought's??) efforts go to serve both the community and the company's customers. I can try and re-assure you that all will be well, but I know that time is the only thing that will prove that out as each one will decide for themselves whether or not their input is valued and acted upon. To provide some perspective, for the next 5 months at least, Continuum will be providing 3.5 people at least 50% to the NumPy project plus dev ops help to get issue tracking and continuous build integration set up.After that we will have at least 1.5 people devoted full-time to the open-source NumPy project (more if possible).I would like this support to actually go through the Foundation (which already has a community governance and non-profit mission statement), but this takes some leg-work in getting the Foundation setup and organizing those contracts. But, that is my intent and what I am working to get in place eventually. Obviously, the fact that I am deeply involved in NumPy complicates the question of community governance for some people, but I hope you will trust that we are just trying to improve NumPy as we understand it. I remain interested in others views of what improving NumPy means. But, I do have a long list of ideas that I am anxious to get started on. Travis, in about a month, I will be starting up work at a company that has been users of the SciPy stack, but has not been active members of the community. I wish to change that. Will this Funding committee serve as a face for numpy for private companies? Absolutely. The Foundation web-site is getting set up right now. It will be an evolving thing, and your feedback about how the Foundation can help you get your company involved will be very helpful.I would like multiple companies to interact through the Foundation, and that is how I would ultimately like Continuum to interact with the community as well. The fact that Continuum employs people who work on NumPy should be no more concerning than the fact that Google employs people that work on Python, or that Enthought employs people who work on SciPy and NumPy. I recognize that my role in NumPy, the Foundation, and my company may be concerning for some. I firmly believe that NumPy is successful because of everybody who has participated. I am not interested in somehow changing that. I just want to do what I can to accelerate it. I look forward to working with you and your company in the Foundation. Best regards, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
Hi, On Tue, Feb 14, 2012 at 4:43 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 14, 2012 at 3:58 PM, Travis Oliphant tra...@continuum.io wrote: When we selected the name NumFOCUS just a few weeks ago, we created the list for numfocus and then I signed everyone up for that list who was on the other one. I apologize if anyone felt left out. That is not my intention. My point is that there are two ways go to about this process, one is open and the other is closed. In the open version, someone proposes such a group to the mailing lists. They ask for expressions of interest. The discussion might then move to another mailing list that is publicly known and widely advertised. Members of the board are proposed in public. There might be some sort of formal or informal voting process. The reason to prefer this to the more informal private negotiations is that a) the community feels a greater ownership and control of the process and b) it is much harder to weaken or subvert an organization that explicitly does all its business in public. Your points are well taken. However, my point is that this has been discussed on an open mailing list. Things weren't *as* open as they could have been, perhaps, in terms of board selection. But, there was opportunity for people to provide input. I am on the numpy, scipy, matplotlib, ipython and cython mailing lists. Jarrod and Fernando are friends of mine. I've been obviously concerned about numpy governance for some time. I didn't know about this mailing list, had only a vague idea that some sort of foundation was being proposed and I had no idea at all that you'd selected a board. Would you say that was closer to 'open' or closer to 'closed'? By the way - I want to be clear - I am not suggesting that I should have been one of the people involved in these discussions. If you were choosing a small number of people to discuss this with, one of them should not be me. I am saying that, if I didn't know, it's reasonable to assume that very few people knew, who weren't being explicitly told, and that this means that the process was, effectively, closed. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
Your points are well taken. However, my point is that this has been discussed on an open mailing list. Things weren't *as* open as they could have been, perhaps, in terms of board selection. But, there was opportunity for people to provide input. I am on the numpy, scipy, matplotlib, ipython and cython mailing lists. Jarrod and Fernando are friends of mine. I've been obviously concerned about numpy governance for some time. I didn't know about this mailing list, had only a vague idea that some sort of foundation was being proposed and I had no idea at all that you'd selected a board. Would you say that was closer to 'open' or closer to 'closed'? I see it a different way.First, the Foundation is not a NumPy-governance thing. Certainly it could grow in that direction over time, but it isn't there now, nor is that its goal. Second, the Foundation is just getting started.It's only come together over the past couple of weeks.The fact that we are talking about it now, seems to me to indicate that it is quite open --- certainly closer to 'open' then you seem to imply. Also, the fact that there was a public mailing list for its discussion certainly sounds open to me (poorly advertised I will grant you). I tried to include as many people as I thought were interested by the responses to the initial emails on the list.I reached out to people that contacted me expressing their interest, and included them on the mailing list. I can accept that I made mistakes. I can guarantee that I will make more. Your feedback is appreciated and noted. The fact is that the Foundation is really a service organization that will require a lot of work to run and administer.It's effectiveness at fulfilling its mission will depend on how well it serves the group on this list, as well as the other groups that are working on Python for Science. I'm all for getting as many volunteers as we can get for the Foundation. I've just been trying to get things organized. Sometimes this works best by phone calls and direct contact, rather than mailing lists. For those interested. The Foundation mission is to: * Promote Open Source Software for Science * Fund Open Source Projects in Science (currently NumPy, SciPy, IPython, and Matplotlib are first-tier with a whole host of second-tier projects that could received funding) * through grants * through code bounties * through graduate-student scholarships * Sponsor sprints * Sponsor conferences * Sponsor student travel * etc., etc. Whether or not it can do any of those things depends on whether or not it can raise money from people and organizations that benefit from the Scientific Python Stack.All of this will be advertised more as the year progresses. Best regards, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
On 2/14/12 7:17 PM, Travis Oliphant wrote: * Fund Open Source Projects in Science (currently NumPy, SciPy, IPython, and Matplotlib are first-tier with a whole host of second-tier projects that could received funding) * through grants So, for example, would the Foundation apply to mentor Google Summer of Code projects? Jason ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy governance update - was: Updated differences between 1.5.1 to 1.6.1
On Tue, Feb 14, 2012 at 6:43 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Tue, Feb 14, 2012 at 3:58 PM, Travis Oliphant tra...@continuum.io wrote: When we selected the name NumFOCUS just a few weeks ago, we created the list for numfocus and then I signed everyone up for that list who was on the other one. I apologize if anyone felt left out. That is not my intention. My point is that there are two ways go to about this process, one is open and the other is closed. In the open version, someone proposes such a group to the mailing lists. They ask for expressions of interest. The discussion might then move to another mailing list that is publicly known and widely advertised. Members of the board are proposed in public. There might be some sort of formal or informal voting process. The reason to prefer this to the more informal private negotiations is that a) the community feels a greater ownership and control of the process and b) it is much harder to weaken or subvert an organization that explicitly does all its business in public. Your points are well taken. However, my point is that this has been discussed on an open mailing list. Things weren't *as* open as they could have been, perhaps, in terms of board selection. But, there was opportunity for people to provide input. I am on the numpy, scipy, matplotlib, ipython and cython mailing lists. Jarrod and Fernando are friends of mine. I've been obviously concerned about numpy governance for some time. I didn't know about this mailing list, had only a vague idea that some sort of foundation was being proposed and I had no idea at all that you'd selected a board. Would you say that was closer to 'open' or closer to 'closed'? Perceptions can be damaging. This is one of the big reasons for the organization of the Foundation -- to be a place separate from any commercial venture which can direct resources to a vision whose goal is more democratically determined. Are you proposing that the Foundation oversee Numpy governance and direction? From your chosen members I'm guessing that the idea is for the foundation to think about broad strategy rather than - say - whether missing values should be encoded with masked arrays? No, I am not proposing that. The Foundation will be focused on higher-level broad strategy sorts of things: mostly around how to raise money and how to direct that money to projects that have their own development cycles. I would think the Foundation would be interested in paying for things like issue trackers and continuous integration servers as well. It will leave NumPy management to this list and the people who have gathered around this watering hole. Obviously, there will be points of connection, but exactly how this will play-out depends on who shows up to both organizations. I think we do have a NumPy steering group if you want to call it that. It is currently me, Mark Wiebe, and Charles Harris. Rolf Gommers, Pauli Virtanen, David Cournapeau and Robert Kern also have opinions that carry significant weight. Are there other people that should be on this list? There are other people who also speak up on this list whose opinions will be listened to and heard. In fact, I hope that many more people will come to the list and speak out as development increases. The point I was making was that the concentration of numpy development hours and talent in your company makes it urgent that the numpy governance is set out formally, that the interests of the company are made clear, and that the steering group can be assured of explicit and public independence from the interests of the company, if and when that becomes necessary. In the past, the numpy steering group has seemed a virtual organization, formed ad-hoc when needed, and with no formal governance. I'm saying that I firmly believe that has to change, to avoid the actual or perceived loss of community ownership. I hear your point. Thank you for sharing it. Fortunately, we are having this discussion, and plan to continue to have it as any concerns arise. I think the situation is actually less concentrated than it used to be when the SciPy steering committee was discussed. On that note, I think the SciPy steering committee needs serious revision as well. But, we've all just been getting along pretty well without too much formalism, so far, so perhaps that is enough for now. But a) there have already been serious unresolved disagreements on this list (I note no resolution of the masks / NA debate) and b) the whole point is to set up structures that can deal with the problems before or as they arise. After the problem arises, it is too late. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org
