Re: [Numpy-discussion] A 1.6.2 release?
On Sun, Apr 22, 2012 at 3:44 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Apr 22, 2012 at 5:25 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sat, Apr 21, 2012 at 5:16 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sat, Apr 21, 2012 at 2:46 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Fri, Apr 20, 2012 at 8:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, Given the amount of new stuff coming in 1.7 and the slip in it's schedule, I wonder if it would be worth putting out a 1.6.2 release with fixes for einsum, ticket 1578, perhaps some others. My reasoning is that the fall releases of Fedora, Ubuntu are likely to still use 1.6 and they might as well use a somewhat fixed up version. The downside is located and backporting fixes is likely to be a fair amount of work. A 1.7 release would be preferable, but I'm not sure when we can make that happen. Travis still sounded hopeful of being able to resolve the 1.7 issues relatively soon. On the other hand, even if that's done in one month we'll still miss Debian stable and a 1.6.2 release won't be *that* much work. Let's go for it I would say. Aiming for a RC on May 2nd and final release on May 16th would work for me. I count 280 BUG commits since 1.6.1, so we are going to need to thin those out. Indeed. We can discard all commits related to NA and datetime, and then we should find some balance between how important the fixes are and how much risk there is that they break something. I agree with the couple of backports you've done so far, but I propose to do the rest via PRs. Charles, did you have some practical way in mind to select these commits? We could split it up by time range or by submodules for example. I'd prefer the latter. You would be able to do a better job of the commits touching numpy/core than I. How about you do that one and the polynomial module, and I do the rest? Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A 1.6.2 release?
On Mon, Apr 23, 2012 at 12:22 AM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Sun, Apr 22, 2012 at 3:44 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Apr 22, 2012 at 5:25 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sat, Apr 21, 2012 at 5:16 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sat, Apr 21, 2012 at 2:46 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Fri, Apr 20, 2012 at 8:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, Given the amount of new stuff coming in 1.7 and the slip in it's schedule, I wonder if it would be worth putting out a 1.6.2 release with fixes for einsum, ticket 1578, perhaps some others. My reasoning is that the fall releases of Fedora, Ubuntu are likely to still use 1.6 and they might as well use a somewhat fixed up version. The downside is located and backporting fixes is likely to be a fair amount of work. A 1.7 release would be preferable, but I'm not sure when we can make that happen. Travis still sounded hopeful of being able to resolve the 1.7 issues relatively soon. On the other hand, even if that's done in one month we'll still miss Debian stable and a 1.6.2 release won't be *that* much work. Let's go for it I would say. Aiming for a RC on May 2nd and final release on May 16th would work for me. I count 280 BUG commits since 1.6.1, so we are going to need to thin those out. Indeed. We can discard all commits related to NA and datetime, and then we should find some balance between how important the fixes are and how much risk there is that they break something. I agree with the couple of backports you've done so far, but I propose to do the rest via PRs. Charles, did you have some practical way in mind to select these commits? We could split it up by time range or by submodules for example. I'd prefer the latter. You would be able to do a better job of the commits touching numpy/core than I. How about you do that one and the polynomial module, and I do the rest? I'll give it a shot. I thought the first thing I would try is a search on tickets. We'll also need to track things and I haven't thought of a good way to do that apart from making a list and checking things off. I don't think there was too much polynomial fixing, mostly new stuff, but I'd like to use the current documentation. I don't know how you manage that for releases. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Command-line options for (Windows) NumPy Installer?
Thanks Ralf! I'm interested in unattended/silent installations. My best, Dave --- Date: Sat, 21 Apr 2012 10:48:36 +0200 From: Ralf Gommers ralf.gomm...@googlemail.com Subject: Re: [Numpy-discussion] Command-line options for (Windows) NumPy Installer? To: Discussion of Numerical Python numpy-discussion@scipy.org Message-ID: CABL7CQjHRun1BOndM0+adgS8msNkS4UR1KHUddc7vVYwTt=p...@mail.gmail.com Content-Type: text/plain; charset=windows-1252 On Fri, Apr 20, 2012 at 8:05 PM, Dave Fugate dfug...@microsoft.com wrote: Hi, is there any documentation available on exactly which command line options are available from NumPy?s ?superpack? installers on Windows? E.g., http://docs.scipy.org/doc/numpy/user/install.html mentions an ?/arch? flag, but I?m not seeing anything else called out. Other than arch selection I think it's a fairly standard NSIS installer. No idea what else you can do with it though from the command line. Are you looking to accomplish some specific task? Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A 1.6.2 release?
On Mon, Apr 23, 2012 at 8:47 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Apr 23, 2012 at 12:22 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sun, Apr 22, 2012 at 3:44 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Apr 22, 2012 at 5:25 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: Aiming for a RC on May 2nd and final release on May 16th would work for me. I count 280 BUG commits since 1.6.1, so we are going to need to thin those out. Indeed. We can discard all commits related to NA and datetime, and then we should find some balance between how important the fixes are and how much risk there is that they break something. I agree with the couple of backports you've done so far, but I propose to do the rest via PRs. Charles, did you have some practical way in mind to select these commits? We could split it up by time range or by submodules for example. I'd prefer the latter. You would be able to do a better job of the commits touching numpy/core than I. How about you do that one and the polynomial module, and I do the rest? I'll give it a shot. I thought the first thing I would try is a search on tickets. We'll also need to track things and I haven't thought of a good way to do that apart from making a list and checking things off. I don't think there was too much polynomial fixing, mostly new stuff, but I'd like to use the current documentation. I don't know how you manage that for releases. Nothing too fancy - I use the open tickets for the milestone at http://projects.scipy.org/numpy/report/3, plus the checklist at https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt and perhaps a small todo list in my inbox. Normally we only do bugfix releases for specific reasons, so besides those I just scan through the list of commits and pick only some relevant ones of which I'm sure that they won't give any problems. The fixed items under http://projects.scipy.org/numpy/query?status=closedgroup=resolutionmilestone=1.7.0 http://projects.scipy.org/numpy/query?status=closedgroup=resolutionmilestone=2.0.0 probably give the best overview. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NEP mask code and the 1.7 release
On Mon, Apr 23, 2012 at 12:15 AM, Nathaniel Smith n...@pobox.com wrote: We need to decide what to do with the NA masking code currently in master, vis-a-vis the 1.7 release. While this code is great at what it is, we don't actually have consensus yet that it's the best way to give our users what they want/need -- or even an appropriate way. So we need to figure out how to release 1.7 without committing ourselves to supporting this design in the future. Background: what does the code currently in master do? It adds 3 pointers at the end of the PyArrayObject struct (which is better known as the numpy.ndarray object). These new struct members, and some accessors for them, are exposed as part of the public API. There are also a few additions to the Python-level API (mask= argument to np.array, skipna= argument to ufuncs, etc.) What does this mean for compatibility? The change in the ndarray struct is not as problematic as it might seem, compatibility-wise, since Python objects are almost always referred to by pointers. Since the initial part of the struct will continue to have the same memory layout, existing source and binary code that works with PyArrayObject *pointers* will continue to work unchanged. One place where the actual struct size matters is for any C-level ndarray subclasses, which will have their memory layout change, and thus will need to be recompiled. (Python-level ndarray subclasses will have their memory layout change as well -- e.g., they will have different __dictoffset__ values -- but it's unlikely that any existing Python code depends on such details.) What if we want to change our minds later? --- For the same reasons as given above, any new code which avoids referencing the new struct fields referring to masks, or using the new masking APIs, will continue to work even if the masking is later removed. Any new code which *does* refer to the new masking APIs, or references the fields directly, will break if masking is later removed. Specifically, source will fail to compile, and existing binaries will silently access memory that is past the end of the PyArrayObject struct, which will have unpredictable consequences. (Most likely segfaults, but no guarantees.) This applies even to code which simply tries to check whether a mask is present. So I think the preconditions for leaving this code as-is for 1.7 are that we must agree: * We are willing to require a recompile of any C-level ndarray subclasses (do any exist?) As long as it's only subclasses I think this may be OK. Not 100% sure on this one though. * We are willing to make absolutely no guarantees about future compatibility for code which uses APIs marked experimental That is what I understand experimental to mean. Could stay, could change - no guarantees. * We are willing for this breakage to occur in the form of random segfaults This is not OK of course. But it shouldn't apply to the Python API, which I think is the most important one here. * We are okay with the extra 3 pointers worth of memory overhead on each ndarray Personally I can live with all of these if everyone else can, but I'm nervous about reducing our compatibility guarantees like that, and we'd probably need, at a minimum, a flashier EXPERIMENTAL sign than we currently have. (Maybe we should resurrect the weasels ;-) [1]) [1] http://mail.scipy.org/pipermail/numpy-discussion/2012-March/061204.html snip I'm personally willing to implement either of these changes. Thank you Nathaniel, that is a very important and helpful statement. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Command-line options for (Windows) NumPy Installer?
On Mon, Apr 23, 2012 at 5:55 PM, Dave Fugate dfug...@microsoft.com wrote: Thanks Ralf! I'm interested in unattended/silent installations. I'm afraid that that doesn't work. NSIS installers provide the /S option for silent installs, but it requires some changes to the install script that we apparently didn't make. I opened http://projects.scipy.org/numpy/ticket/2112 for this. Ralf --- Date: Sat, 21 Apr 2012 10:48:36 +0200 From: Ralf Gommers ralf.gomm...@googlemail.com Subject: Re: [Numpy-discussion] Command-line options for (Windows) NumPy Installer? To: Discussion of Numerical Python numpy-discussion@scipy.org Message-ID: CABL7CQjHRun1BOndM0+adgS8msNkS4UR1KHUddc7vVYwTt=p...@mail.gmail.com Content-Type: text/plain; charset=windows-1252 On Fri, Apr 20, 2012 at 8:05 PM, Dave Fugate dfug...@microsoft.com wrote: Hi, is there any documentation available on exactly which command line options are available from NumPy?s ?superpack? installers on Windows? E.g., http://docs.scipy.org/doc/numpy/user/install.html mentions an ?/arch? flag, but I?m not seeing anything else called out. Other than arch selection I think it's a fairly standard NSIS installer. No idea what else you can do with it though from the command line. Are you looking to accomplish some specific task? Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
Hi, On Sun, Apr 22, 2012 at 3:15 PM, Nathaniel Smith n...@pobox.com wrote: If you hang around big FOSS projects, you'll see the word consensus come up a lot. For example, the glibc steering committee recently dissolved itself in favor of governance directly by the consensus of the people active in glibc development[1]. It's the governing rule of the IETF, which defines many of the most important internet standards[2]. It is the primary way decisions are made on Wikipedia[3]. It's one of the fundamental aspects of accomplishing things within the Apache framework[4]. [1] https://lwn.net/Articles/488778/ [2] https://www.ietf.org/tao.html#getting.things.done [3] https://en.wikipedia.org/wiki/Wikipedia:Consensus [4] https://www.apache.org/foundation/voting.html I think the big problem here is that Chuck (I hope I'm not misrepresenting him) is not interested in discussion of process, and the last time we had a specific thread on governance, Travis strongly implied he was not very interested either, at least at the time. In that situation, there's rather a high threshold to pass before getting involved in the discussion, and I think you're seeing some evidence for that. So, as before, and as we discussed on gchat :) - whether this discussion can go anywhere depends on Travis. Travis - what do you think? See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] documentation bug: Matrix library page not populated
On Thu, Apr 19, 2012 at 3:12 AM, josef.p...@gmail.com wrote: On Wed, Apr 18, 2012 at 4:14 PM, Pauli Virtanen p...@iki.fi wrote: Hi, 18.04.2012 19:57, Alan G Isaac kirjoitti: http://docs.scipy.org/doc/numpy/reference/routines.matlib.html#module-numpy.matlib promises a list of functions that does not appear (at the moment, anyway). This doesn't seem to be due to a technical reason, but rather than because nobody has written a list of the functions in the docstring of the module. Is it a good idea to use this? Mixing namespaces would completely confuse me. for f in dir(numpy.matlib): ... try: ... if getattr(numpy.matlib, f).__module__ in ['numpy.matlib', 'numpy.matrixlib.defmatrix']: print f ... except: pass ... asmatrix bmat empty eye identity mat matrix ones rand randn repmat zeros Looks good to me. Did you plan to put this somewhere (PR, doc wiki)? Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] documentation bug: Matrix library page not populated
On Mon, Apr 23, 2012 at 2:05 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Apr 19, 2012 at 3:12 AM, josef.p...@gmail.com wrote: On Wed, Apr 18, 2012 at 4:14 PM, Pauli Virtanen p...@iki.fi wrote: Hi, 18.04.2012 19:57, Alan G Isaac kirjoitti: http://docs.scipy.org/doc/numpy/reference/routines.matlib.html#module-numpy.matlib promises a list of functions that does not appear (at the moment, anyway). This doesn't seem to be due to a technical reason, but rather than because nobody has written a list of the functions in the docstring of the module. Is it a good idea to use this? Mixing namespaces would completely confuse me. for f in dir(numpy.matlib): ... try: ... if getattr(numpy.matlib, f).__module__ in ['numpy.matlib', 'numpy.matrixlib.defmatrix']: print f ... except: pass ... asmatrix bmat empty eye identity mat matrix ones rand randn repmat zeros Looks good to me. Did you plan to put this somewhere (PR, doc wiki)? I was hoping it isn't me that struggles with rst http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.matlib.rst/ (Since we are not voting based on number of PRs, I prefer the doc wiki. Instant feedback. :) Josef Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
On Mon, Apr 23, 2012 at 1:04 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Apr 22, 2012 at 4:15 PM, Nathaniel Smith n...@pobox.com wrote: If you hang around big FOSS projects, you'll see the word consensus come up a lot. For example, the glibc steering committee recently dissolved itself in favor of governance directly by the consensus of the people active in glibc development[1]. It's the governing rule of the IETF, which defines many of the most important internet standards[2]. It is the primary way decisions are made on Wikipedia[3]. It's one of the fundamental aspects of accomplishing things within the Apache framework[4]. [1] https://lwn.net/Articles/488778/ [2] https://www.ietf.org/tao.html#getting.things.done [3] https://en.wikipedia.org/wiki/Wikipedia:Consensus [4] https://www.apache.org/foundation/voting.html But it turns out that this consensus thing is actually somewhat mysterious, and one that most programmers immersed in this culture pick it up by osmosis. And numpy in particular has a lot of developers who are not coming from a classic FOSS programmer background! So this is my personal attempt to articulate what it is, and why requiring consensus is probably the best possible approach to project decision making. So what is consensus? Like, voting or something? - This is surprisingly subtle and specific. Consensus means something like, everyone who cares is satisfied with the result. It does *not* mean * Every opinion counts equally * We vote on anything * Every solution must be perfect and flawless * Every solution must leave everyone overjoyed * Everyone must sign off on every solution. It *does* mean * We invite people to speak up * We generally trust individuals to decide how important their opinion is * We generally trust individuals to decide whether or not they can live with some outcome * If they can't, then we take the time to find something better. One simple way of stating this is, everyone has a veto. In practice, such vetoes are almost never used, so this rule is not particularly illuminating on its own. Hence, the rest of this document. What a waste of time! That all sounds very pretty on paper, but we have stuff to get done. --- First, I'll note that this seemingly utopian scheme has a track record of producing such impractical systems as TCP/IP, SMTP, DNS, Apache, GCC, Linux, Samba, Python, ... Linux is Linus' private tree. Everything that goes in is his decision, everything that stays out is his decision. Of course, he delegates much of the work to people he trusts, but it doesn't even reach the level of a BDFL, it's DFL. As for consensus, it basically comes down to convincing the gatekeepers one level below Linus that your code might be useful. So bad example. Same with TCP/IP, which was basically Kahn and Cerf consulting with a few others and working by request of DARPA. GCC was Richard Stallman (I got one of the first tapes for a $30 donation), Python was Guido. Some of the projects later developed some form of governance but Guido, for instance, can veto anything he dislikes even if he is disinclined to do so. I'm not saying you're wrong about open source, I'm just saying that that each project differs and it is wrong to imply that they follow some common form of governance under the rubric FOSS and that they all seek consensus. And they certainly don't *start* that way. And there are also plenty of projects that fail when the prime mover loses interest or folks get tired of the politics. So a few points here: Consensus-based decision-making is an ideal and a guide, not an algorithm. There's nothing at all inconsistent between having a BDFL and using consensus as the primary guide for decision making -- it just means that the BDFL chooses to exercise their power in that way, and is generally trusted to make judgement calls about specific cases. See Fernando's reply down-thread for an example of this. And I'm not saying that all FOSS projects follow some common form of governance. But I am saying that there's a substantial amount of shared development culture across most successful FOSS projects, and a ton of experience on how to run a project successfully. Project management is a difficult and arcane skill set, and one that's hard to learn except through apprenticeship and osmosis. And it's definitely not included in most courses on programming for scientists! So it'd be nice if numpy could avoid having to re-make some of these mistakes... But the other effect of this being cultural values rather than something explicit and articulated is that sometimes you can't see it from the outside. For example: Linux: Technically, everything you say is true. In practice, good luck convincing Linus or a subsystem maintainer to accept your patch when other
Re: [Numpy-discussion] documentation bug: Matrix library page not populated
On Mon, Apr 23, 2012 at 8:42 PM, josef.p...@gmail.com wrote: On Mon, Apr 23, 2012 at 2:05 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Apr 19, 2012 at 3:12 AM, josef.p...@gmail.com wrote: On Wed, Apr 18, 2012 at 4:14 PM, Pauli Virtanen p...@iki.fi wrote: Hi, 18.04.2012 19:57, Alan G Isaac kirjoitti: http://docs.scipy.org/doc/numpy/reference/routines.matlib.html#module-numpy.matlib promises a list of functions that does not appear (at the moment, anyway). This doesn't seem to be due to a technical reason, but rather than because nobody has written a list of the functions in the docstring of the module. Is it a good idea to use this? Mixing namespaces would completely confuse me. for f in dir(numpy.matlib): ... try: ... if getattr(numpy.matlib, f).__module__ in ['numpy.matlib', 'numpy.matrixlib.defmatrix']: print f ... except: pass ... asmatrix bmat empty eye identity mat matrix ones rand randn repmat zeros Looks good to me. Did you plan to put this somewhere (PR, doc wiki)? I was hoping it isn't me that struggles with rst http://docs.scipy.org/numpy/docs/numpy-docs/reference/routines.matlib.rst/ (Since we are not voting based on number of PRs, I prefer the doc wiki. Instant feedback. :) Great, thanks. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
Hi, On Mon, Apr 23, 2012 at 12:33 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Apr 23, 2012 at 1:04 AM, Charles R Harris charlesr.har...@gmail.com wrote: Linux is Linus' private tree. Everything that goes in is his decision, everything that stays out is his decision. Of course, he delegates much of the work to people he trusts, but it doesn't even reach the level of a BDFL, it's DFL. As for consensus, it basically comes down to convincing the gatekeepers one level below Linus that your code might be useful. So bad example. Same with TCP/IP, which was basically Kahn and Cerf consulting with a few others and working by request of DARPA. GCC was Richard Stallman (I got one of the first tapes for a $30 donation), Python was Guido. Some of the projects later developed some form of governance but Guido, for instance, can veto anything he dislikes even if he is disinclined to do so. I'm not saying you're wrong about open source, I'm just saying that that each project differs and it is wrong to imply that they follow some common form of governance under the rubric FOSS and that they all seek consensus. And they certainly don't *start* that way. And there are also plenty of projects that fail when the prime mover loses interest or folks get tired of the politics. [snip] Linux: Technically, everything you say is true. In practice, good luck convincing Linus or a subsystem maintainer to accept your patch when other people are raising substantive complaints. Here's an email I googled up in a few moments, in which Linus yells at people for trying to submit a patch to him without making sure that all interested parties have agreed: https://lkml.org/lkml/2009/9/14/481 Stuff regularly sits outside the kernel tree in limbo for *years* while people debate different approaches back and forth. To which I'd add: In fact, for [Linus'] decisions to be received as legitimate, they have to be consistent with the consensus of the opinions of participating developers as manifest on Linux mailing lists. It is not unusual for him to back down from a decision under the pressure of criticism from other developers. His position is based on the recognition of his fitness by the community of Linux developers and this type of authority is, therefore, constantly subject to withdrawal. His role is not that of a boss or a manager in the usual sense. In the final analysis, the direction of the project springs from the cumulative synthesis of modifications contributed by individual developers. http://shareable.net/blog/governance-of-open-source-george-dafermos-interview See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NEP mask code and the 1.7 release
On Mon, Apr 23, 2012 at 6:18 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Mon, Apr 23, 2012 at 12:15 AM, Nathaniel Smith n...@pobox.com wrote: We need to decide what to do with the NA masking code currently in master, vis-a-vis the 1.7 release. While this code is great at what it is, we don't actually have consensus yet that it's the best way to give our users what they want/need -- or even an appropriate way. So we need to figure out how to release 1.7 without committing ourselves to supporting this design in the future. Background: what does the code currently in master do? It adds 3 pointers at the end of the PyArrayObject struct (which is better known as the numpy.ndarray object). These new struct members, and some accessors for them, are exposed as part of the public API. There are also a few additions to the Python-level API (mask= argument to np.array, skipna= argument to ufuncs, etc.) What does this mean for compatibility? The change in the ndarray struct is not as problematic as it might seem, compatibility-wise, since Python objects are almost always referred to by pointers. Since the initial part of the struct will continue to have the same memory layout, existing source and binary code that works with PyArrayObject *pointers* will continue to work unchanged. One place where the actual struct size matters is for any C-level ndarray subclasses, which will have their memory layout change, and thus will need to be recompiled. (Python-level ndarray subclasses will have their memory layout change as well -- e.g., they will have different __dictoffset__ values -- but it's unlikely that any existing Python code depends on such details.) What if we want to change our minds later? --- For the same reasons as given above, any new code which avoids referencing the new struct fields referring to masks, or using the new masking APIs, will continue to work even if the masking is later removed. Any new code which *does* refer to the new masking APIs, or references the fields directly, will break if masking is later removed. Specifically, source will fail to compile, and existing binaries will silently access memory that is past the end of the PyArrayObject struct, which will have unpredictable consequences. (Most likely segfaults, but no guarantees.) This applies even to code which simply tries to check whether a mask is present. So I think the preconditions for leaving this code as-is for 1.7 are that we must agree: * We are willing to require a recompile of any C-level ndarray subclasses (do any exist?) As long as it's only subclasses I think this may be OK. Not 100% sure on this one though. * We are willing to make absolutely no guarantees about future compatibility for code which uses APIs marked experimental That is what I understand experimental to mean. Could stay, could change - no guarantees. Earlier you said it meant some changes are to be expected, but not complete removal, which seems different from absolutely no guarantees: http://www.mail-archive.com/numpy-discussion@scipy.org/msg36833.html So I just wanted to double-check whether you're revising that earlier opinion, or...? * We are willing for this breakage to occur in the form of random segfaults This is not OK of course. But it shouldn't apply to the Python API, which I think is the most important one here. Right, this part is specifically about ABI compatibility, not API compatibility -- segfaults would only occur for extension libraries that were compiled against one version of numpy and then used with a different version. - N ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NEP mask code and the 1.7 release
On Mon, Apr 23, 2012 at 12:57 PM, Nathaniel Smith n...@pobox.com wrote: Right, this part is specifically about ABI compatibility, not API compatibility -- segfaults would only occur for extension libraries that were compiled against one version of numpy and then used with a different version. Which makes me think -- the ABI will have changed by adding three new pointers to the end of the main struct, yes? Of the options on the table, do any of the others involve adding three new pointers? What I'm getting at is that while the API and symantics may change with a different NA system -- maybe the ABI won't change as much (even if those pointers mean something different, but the size of the struct could be constant). Or is this just a fantasy? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NEP mask code and the 1.7 release
On Mon, Apr 23, 2012 at 9:16 PM, Chris Barker chris.bar...@noaa.gov wrote: On Mon, Apr 23, 2012 at 12:57 PM, Nathaniel Smith n...@pobox.com wrote: Right, this part is specifically about ABI compatibility, not API compatibility -- segfaults would only occur for extension libraries that were compiled against one version of numpy and then used with a different version. Which makes me think -- the ABI will have changed by adding three new pointers to the end of the main struct, yes? No, re-read the original message :-). AFAICT the only place that *adding* the pointers will break backwards ABI compatibility is for C subclasses of ndarray, and it's not clear if any exist. Of the options on the table, do any of the others involve adding three new pointers? What I'm getting at is that while the API and symantics may change with a different NA system -- maybe the ABI won't change as much (even if those pointers mean something different, but the size of the struct could be constant). If the size of the struct stays the same but the meaning of the pointers changes, then that's probably not going to lead to any good results for code which tries to manipulate those pointers using the wrong semantics. Usually ABI changes are strictly greater than API changes... though of course it depends on how big exactly the change is. -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NEP mask code and the 1.7 release
On Mon, Apr 23, 2012 at 9:57 PM, Nathaniel Smith n...@pobox.com wrote: On Mon, Apr 23, 2012 at 6:18 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Mon, Apr 23, 2012 at 12:15 AM, Nathaniel Smith n...@pobox.com wrote: We need to decide what to do with the NA masking code currently in master, vis-a-vis the 1.7 release. While this code is great at what it is, we don't actually have consensus yet that it's the best way to give our users what they want/need -- or even an appropriate way. So we need to figure out how to release 1.7 without committing ourselves to supporting this design in the future. Background: what does the code currently in master do? It adds 3 pointers at the end of the PyArrayObject struct (which is better known as the numpy.ndarray object). These new struct members, and some accessors for them, are exposed as part of the public API. There are also a few additions to the Python-level API (mask= argument to np.array, skipna= argument to ufuncs, etc.) What does this mean for compatibility? The change in the ndarray struct is not as problematic as it might seem, compatibility-wise, since Python objects are almost always referred to by pointers. Since the initial part of the struct will continue to have the same memory layout, existing source and binary code that works with PyArrayObject *pointers* will continue to work unchanged. One place where the actual struct size matters is for any C-level ndarray subclasses, which will have their memory layout change, and thus will need to be recompiled. (Python-level ndarray subclasses will have their memory layout change as well -- e.g., they will have different __dictoffset__ values -- but it's unlikely that any existing Python code depends on such details.) What if we want to change our minds later? --- For the same reasons as given above, any new code which avoids referencing the new struct fields referring to masks, or using the new masking APIs, will continue to work even if the masking is later removed. Any new code which *does* refer to the new masking APIs, or references the fields directly, will break if masking is later removed. Specifically, source will fail to compile, and existing binaries will silently access memory that is past the end of the PyArrayObject struct, which will have unpredictable consequences. (Most likely segfaults, but no guarantees.) This applies even to code which simply tries to check whether a mask is present. So I think the preconditions for leaving this code as-is for 1.7 are that we must agree: * We are willing to require a recompile of any C-level ndarray subclasses (do any exist?) As long as it's only subclasses I think this may be OK. Not 100% sure on this one though. * We are willing to make absolutely no guarantees about future compatibility for code which uses APIs marked experimental That is what I understand experimental to mean. Could stay, could change - no guarantees. Earlier you said it meant some changes are to be expected, but not complete removal, which seems different from absolutely no guarantees: http://www.mail-archive.com/numpy-discussion@scipy.org/msg36833.html So I just wanted to double-check whether you're revising that earlier opinion, or...? Stay and change are both not the same as complete removal. But to spell it out: if we release a feature, I expect it to stay in some form. That still means we can change APIs (i.e. no compatibility for code written against the old API), but not removing the concept itself. If we're not even sure that the concept should stay, why bother releasing it as experimental? Experimental is for finding out what works well, not for whether or not we need some concept at all. * We are willing for this breakage to occur in the form of random segfaults This is not OK of course. But it shouldn't apply to the Python API, which I think is the most important one here. Right, this part is specifically about ABI compatibility, not API compatibility -- segfaults would only occur for extension libraries that were compiled against one version of numpy and then used with a different version. That's what I suspected, but not what your earlier email said. I understood your email as talking only about segfaults for code using the new NA C API. Breaking ABI compatibility is a no-go. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Masked Arrays in NumPy 1.x
Hi Paul, On Wed, Apr 11, 2012 at 8:57 PM, Paul Hobson pmhob...@gmail.com wrote: Travis et al, This isn't a reply to anything specific in your email and I apologize if there is a better thread or place to share this information. I've been meaning to participate in the discussion for a long time and never got around to it. The main thing I'd like to is convey my typical use of the numpy.ma module as an environmental engineer analyzing censored datasets --contaminant concentrations that are either at well understood values (not masked) or some unknown value below an upper bound (masked). My basic understanding is that this discussion revolved around how to treat masked data (ignored vs missing) and how to implement one, both, or some middle ground between those two concepts. If I'm off-base, just ignore all of the following. For my purposes, numpy.ma is implemented in a way very well suited to my needs. Here's a gist of a something that was *really* hard for me before I discovered numpy.ma and numpy in general. (this is a bit much, see below for the highlights) https://gist.github.com/2361814 The main message here is that I include the upper bounds of the unknown values (detection limits) in my array and use that to statistically estimate their values. I must be able to retrieve the masked detection limits throughout this process. Additionally the masks as currently implemented allow me sort first the undetected values, then the detected values (see __rosRanks in the gist). As boots-on-the-ground user of numpy, I'm ecstatic that this tool exists. I'm also pretty flexible and don't anticipated any major snags in my work if things change dramatically as the masked/missing/ignored functionality evolves. Thanks to everyone for the hard work and great tools, -Paul Hobson Thanks for this note -- it's getting feedback from people on how they're actually using numpy.ma is *very* helpful, because there's a lot more data out there on the missing data use case. But, I couldn't quite figure out what you're actually doing in this code. It looks like the measurements that you're masking out have some values hidden behind the mask, which you then make use of? Unfortunately, I don't know anything about environmental engineering or the method of Hirsch and Stedinger (1987). Could you elaborate a bit on what these masked values mean and how you process them? -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
Linux: Technically, everything you say is true. In practice, good luck convincing Linus or a subsystem maintainer to accept your patch when other people are raising substantive complaints. Here's an email I googled up in a few moments, in which Linus yells at people for trying to submit a patch to him without making sure that all interested parties have agreed: https://lkml.org/lkml/2009/9/14/481 Stuff regularly sits outside the kernel tree in limbo for *years* while people debate different approaches back and forth. To which I'd add: In fact, for [Linus'] decisions to be received as legitimate, they have to be consistent with the consensus of the opinions of participating developers as manifest on Linux mailing lists. It is not unusual for him to back down from a decision under the pressure of criticism from other developers. His position is based on the recognition of his fitness by the community of Linux developers and this type of authority is, therefore, constantly subject to withdrawal. His role is not that of a boss or a manager in the usual sense. In the final analysis, the direction of the project springs from the cumulative synthesis of modifications contributed by individual developers. http://shareable.net/blog/governance-of-open-source-george-dafermos-interview This is the model that I have for NumPy development. It is my view of how NumPy has evolved already and how Numarray, and Numeric evolved before it as well.I also feel like these things are fundamentally determined by the people involved and by the personalities and styles of those who participate. There certainly are globally applicable principles (like code review, building consensus, and mutual respect) that are worth emphasizing over and over again. If it helps let's write those down and say these are the principles we live by. I am suspicious that you can go beyond this in formalizing the process as you ultimately are at the mercy of the people involved and their judgment, anyway. I can also see that for the benefit of newcomers and occasional contributors it can be beneficial to have some documentation of the natural, emergent methods and interactions that apply to cooperative software development. But, I would hesitate to put some-kind of aura of authority around such a document that implies the processes cannot be violated if good judgment demands that they should be. That is the basis of my hesitation to spend much time on officially documenting our process Right now we are trying to balance difficult things: stable releases with experimental development. The fact that we had such differences of opinion last year on masked arrays / missing values and how to incorporate them into a common object model means that we should not have committed the code to master until we figured out a way to reconcile Nathaniel's concerns. That is my current view.I was very enthused that we had someone contributing large scale changes that clearly showed an ability to understand the code and contribute to it --- that hadn't happened in a while. I wanted to encourage that. I still do. I think the process itself has shown that you can have an impact on NumPy just by voicing your opinion. Clearly, you have more of an effect on NumPy by submitting pull requests, but NumPy development does listen carefully to the voices of users. Best, -Travis See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Masked Arrays in NumPy 1.x
Thank you very much for contributing this description.It is very helpful to see how people use numpy.ma in the wild. -Travis On Apr 11, 2012, at 2:57 PM, Paul Hobson wrote: Travis et al, This isn't a reply to anything specific in your email and I apologize if there is a better thread or place to share this information. I've been meaning to participate in the discussion for a long time and never got around to it. The main thing I'd like to is convey my typical use of the numpy.ma module as an environmental engineer analyzing censored datasets --contaminant concentrations that are either at well understood values (not masked) or some unknown value below an upper bound (masked). My basic understanding is that this discussion revolved around how to treat masked data (ignored vs missing) and how to implement one, both, or some middle ground between those two concepts. If I'm off-base, just ignore all of the following. For my purposes, numpy.ma is implemented in a way very well suited to my needs. Here's a gist of a something that was *really* hard for me before I discovered numpy.ma and numpy in general. (this is a bit much, see below for the highlights) https://gist.github.com/2361814 The main message here is that I include the upper bounds of the unknown values (detection limits) in my array and use that to statistically estimate their values. I must be able to retrieve the masked detection limits throughout this process. Additionally the masks as currently implemented allow me sort first the undetected values, then the detected values (see __rosRanks in the gist). As boots-on-the-ground user of numpy, I'm ecstatic that this tool exists. I'm also pretty flexible and don't anticipated any major snags in my work if things change dramatically as the masked/missing/ignored functionality evolves. Thanks to everyone for the hard work and great tools, -Paul Hobson On Mon, Apr 9, 2012 at 9:52 PM, Travis Oliphant tra...@continuum.io wrote: Hey all, I've been waiting for Mark Wiebe to arrive in Austin where he will spend several weeks, but I also know that masked arrays will be only one of the things he and I are hoping to make head-way on while he is in Austin. Nevertheless, we need to make progress on the masked array discussion and if we want to finalize the masked array implementation we will need to finish the design. I've caught up on most of the discussion including Mark's NEP, Nathaniel's NEP and other writings and the very-nice mailing list discussion that included a somewhat detailed discussion on the algebra of IGNORED. I think there are some things still to be decided. However, I think some things are pretty clear: 1) Masked arrays are going to be fundamental in NumPy and these should replace most people's use of numpy.ma. The numpy.ma code will remain as a compatibility layer 2) The reality of #1 and NumPy's general philosophy to date means that masked arrays in NumPy should support the common use-cases of masked arrays (including getting and setting of the mask from the Python and C-layers). However, the semantic of what the mask implies may change from what numpy.ma uses to having a True value meaning selected. 3) There will be missing-data dtypes in NumPy. Likely only a limited sub-set (string, bytes, int64, int32, float32, float64, complex64, complex32, and object) with an API that allows more to be defined if desired. These will most likely use Mark's nice machinery for managing the calculation structure without requiring new C-level loops to be defined. 4) I'm still not sure about whether the IGNORED concept is necessary or not.I really like the separation that was emphasized between implementation (masks versus bit-patterns) and operations (propagating versus non-propagating). Pauli even created another dimension which I don't totally grok and therefore can't remember. Pauli? Do you still feel that is a necessary construction? But, do we need the IGNORED concept to indicate what amounts to different default key-word arguments to functions that operate on NumPy arrays containing missing data (however that is represented)?My current weak view is that it is not really necessary. But, I could be convinced otherwise. I think the good news is that given Mark's hard-work and Nathaniel's follow-up we are really quite far along. I would love to get Nathaniel's opinion about what remains un-done in the current NumPy code-base. I would also appreciate knowing (from anyone with an interest) opinions of items 1-4 above and anything else I've left out. Thanks, -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion
Re: [Numpy-discussion] What is consensus anyway
Hi, On Mon, Apr 23, 2012 at 3:08 PM, Travis Oliphant tra...@continuum.io wrote: Linux: Technically, everything you say is true. In practice, good luck convincing Linus or a subsystem maintainer to accept your patch when other people are raising substantive complaints. Here's an email I googled up in a few moments, in which Linus yells at people for trying to submit a patch to him without making sure that all interested parties have agreed: https://lkml.org/lkml/2009/9/14/481 Stuff regularly sits outside the kernel tree in limbo for *years* while people debate different approaches back and forth. To which I'd add: In fact, for [Linus'] decisions to be received as legitimate, they have to be consistent with the consensus of the opinions of participating developers as manifest on Linux mailing lists. It is not unusual for him to back down from a decision under the pressure of criticism from other developers. His position is based on the recognition of his fitness by the community of Linux developers and this type of authority is, therefore, constantly subject to withdrawal. His role is not that of a boss or a manager in the usual sense. In the final analysis, the direction of the project springs from the cumulative synthesis of modifications contributed by individual developers. http://shareable.net/blog/governance-of-open-source-george-dafermos-interview This is the model that I have for NumPy development. It is my view of how NumPy has evolved already and how Numarray, and Numeric evolved before it as well. I also feel like these things are fundamentally determined by the people involved and by the personalities and styles of those who participate. There certainly are globally applicable principles (like code review, building consensus, and mutual respect) that are worth emphasizing over and over again. If it helps let's write those down and say these are the principles we live by. I am suspicious that you can go beyond this in formalizing the process as you ultimately are at the mercy of the people involved and their judgment, anyway. I think writing it down would help enormously. For example, if you do agree to Nathaniel's view of consensus - *in principle* - and we write that down and agree, we have a document to appeal to when we next run into trouble.Maybe the document could say something like: We strive for consensus [some refs here]. Any substantial new feature is subject to consensus. Only if all avenues for consensus have been documented, and exhausted, will we [vote, defer to Travis, or some other tie-breaking thing]. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
On Mon, Apr 23, 2012 at 3:08 PM, Travis Oliphant tra...@continuum.io wrote: Right now we are trying to balance difficult things: stable releases with experimental development. Perhaps a more formal development release system could help here. IIUC, numpy pretty much has two things: the latest release (and past ones) and master (and assorted experimentla branches). If someone develops a new feature, we can either: have them submit a pull request, and people with the where-with-all can pull it, compile, it, and start tesing it on their own -- hsitory shows that this is a small group. merge it with master -- and hope it gets the testing is should before it becomes part of a release, but: we are rightly heistant to put experimental stuff in master, and it really dont' get that much testing -- again only folks that are building master will even see it. Some projects have a more format development release system. wxPython, for instance has had for years development releases with odd numbers -- right now, the official release is 2.8.*, but there is a 2.9.* out there that is getting some use and testing. A couple of things help make this work: 1) Robin makes the effort to put out binaries for development releases -- it's easy to go get and give it a try. 2) there is the wxversion system that makes it easy to install a new versin of wx, and easily switch between them (it's actually broken on OS-X right now --- :-) ) -- this pre-dated virtualenv and friends, maybe virtualenv is enough for this now. Anyway, it's a thought -- I think some more rea-world use of new features before a real commitment to adopting them would be great. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
That is an excellent thought. We could make the odd numbered releases experimental and the even-numbered as stable. That makes some sense.What do others think? -Travis On Apr 23, 2012, at 5:46 PM, Chris Barker wrote: On Mon, Apr 23, 2012 at 3:08 PM, Travis Oliphant tra...@continuum.io wrote: Right now we are trying to balance difficult things: stable releases with experimental development. Perhaps a more formal development release system could help here. IIUC, numpy pretty much has two things: the latest release (and past ones) and master (and assorted experimentla branches). If someone develops a new feature, we can either: have them submit a pull request, and people with the where-with-all can pull it, compile, it, and start tesing it on their own -- hsitory shows that this is a small group. merge it with master -- and hope it gets the testing is should before it becomes part of a release, but: we are rightly heistant to put experimental stuff in master, and it really dont' get that much testing -- again only folks that are building master will even see it. Some projects have a more format development release system. wxPython, for instance has had for years development releases with odd numbers -- right now, the official release is 2.8.*, but there is a 2.9.* out there that is getting some use and testing. A couple of things help make this work: 1) Robin makes the effort to put out binaries for development releases -- it's easy to go get and give it a try. 2) there is the wxversion system that makes it easy to install a new versin of wx, and easily switch between them (it's actually broken on OS-X right now --- :-) ) -- this pre-dated virtualenv and friends, maybe virtualenv is enough for this now. Anyway, it's a thought -- I think some more rea-world use of new features before a real commitment to adopting them would be great. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
On Mon, Apr 23, 2012 at 5:02 PM, Travis Oliphant tra...@continuum.iowrote: That is an excellent thought. We could make the odd numbered releases experimental and the even-numbered as stable. That makes some sense.What do others think? I'm starting to think that a fork might be the best solution to the present problem. There is plenty of precedent for forks in FOSS, for example GCC, EGCS, Redhat 1.97, LLVM and emacs, xemacs. There are several semi-official forks of linux (Android, the real time Kernel, etc.) Zeromq just forked, OpenOffice forked, there was XFree86 forked to Xorg, etc. Linus encourages forks, so there is even authority for that ;) Of course, the further the fork diverges from the original the harder reintegration becomes, witness Android and wake-locks. But a fork would cure a lot of contention. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] A 1.6.2 release?
On Mon, Apr 23, 2012 at 11:05 AM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Mon, Apr 23, 2012 at 8:47 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Apr 23, 2012 at 12:22 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Sun, Apr 22, 2012 at 3:44 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Apr 22, 2012 at 5:25 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: Aiming for a RC on May 2nd and final release on May 16th would work for me. I count 280 BUG commits since 1.6.1, so we are going to need to thin those out. Indeed. We can discard all commits related to NA and datetime, and then we should find some balance between how important the fixes are and how much risk there is that they break something. I agree with the couple of backports you've done so far, but I propose to do the rest via PRs. Charles, did you have some practical way in mind to select these commits? We could split it up by time range or by submodules for example. I'd prefer the latter. You would be able to do a better job of the commits touching numpy/core than I. How about you do that one and the polynomial module, and I do the rest? I'll give it a shot. I thought the first thing I would try is a search on tickets. We'll also need to track things and I haven't thought of a good way to do that apart from making a list and checking things off. I don't think there was too much polynomial fixing, mostly new stuff, but I'd like to use the current documentation. I don't know how you manage that for releases. Nothing too fancy - I use the open tickets for the milestone at http://projects.scipy.org/numpy/report/3, plus the checklist at https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt and perhaps a small todo list in my inbox. Normally we only do bugfix releases for specific reasons, so besides those I just scan through the list of commits and pick only some relevant ones of which I'm sure that they won't give any problems. The fixed items under http://projects.scipy.org/numpy/query?status=closedgroup=resolutionmilestone=1.7.0 http://projects.scipy.org/numpy/query?status=closedgroup=resolutionmilestone=2.0.0 probably give the best overview. Argghhh... work ;) But thanks, that's a good starting point... Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
On Mon, Apr 23, 2012 at 4:39 PM, Charles R Harris charlesr.har...@gmail.com wrote: I'm starting to think that a fork might be the best solution to the present problem. If you are referring to the traditional concept of a fork, and not to the type we frequently make on GitHub, then I'm surprised that no one has objected already. What would a fork solve? To paraphrase the regexp saying: after forking, we'll simply have two problems. It's really not that hard to focus our attention on technical issues and to reach consensus. Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
On Mon, Apr 23, 2012 at 4:02 PM, Travis Oliphant tra...@continuum.io wrote: That is an excellent thought. We could make the odd numbered releases experimental and the even-numbered as stable. That makes some sense. What do others think? I think the concern with that is manpower: it effectively requires maintaining two complete projects alive in parallel. As far as I know, a number projects that used to have that model have backed off (the linux kernel included) to better enable a limited team to focus on development. I'm skeptical that numpy has the manpower to sustain that approach. Cheers, f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What is consensus anyway
On Mon, Apr 23, 2012 at 8:49 PM, Stéfan van der Walt ste...@sun.ac.za wrote: If you are referring to the traditional concept of a fork, and not to the type we frequently make on GitHub, then I'm surprised that no one has objected already. What would a fork solve? To paraphrase the regexp saying: after forking, we'll simply have two problems. I concur with you here: github 'forks', yes, as many as possible! Hopefully every one of those will produce one or more PRs :) But a fork in the sense of a divergent parallel project? I think that would only be indicative of a complete failure to find a way to make progress here, and I doubt we're anywhere near that state. That forks are *possible* is indeed a valuable and important option in open source software, because it means that a truly dysfunctional original project team/direction can't hold a community hostage forever. But that doesn't mean that full-blown forks should be considered lightly, as they also carry enormous costs. I see absolutely nothing in the current scenario to even remotely consider that a full-blown fork would be a good idea, and I hope I'm right. It seems to me we're making progress on problems that led to real difficulties last year, but from multiple parties I see signs that give me reason to be optimistic that the project is getting better, not worse. Cheers, f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
