[Numpy-discussion] summary of "office Hours" open discusison April 25
Office Hours 25April 2018 12:00 -13:00 PDT Present: Matti Picus, Allan Haldane, Ralf Gommers, Matthew Brett, Tyler Reddy, Stéfan van der Walt, Hameer Abbasi Some of the people were not present for the entire discussion, audio was a little flaky at times. Topics: Grant background overview Matti has been browsing through issues and pull-requests to try to get a handle on common themes and community pain points. - Policy questions: - Do we close duplicate issues? (answer - Yes, referencing the other issue, as long as they are true duplicates ) - Do we close tutorial-like issues that are documented?(answer - Yes, maybe improving documentation) - Common theme - there are many issues about overflow, mainly about int32. Maybe add a mode or command switch for warning on int32 overflow? - Requested topic for discussion - improving CI and MacOS testing - How to filter CI issues on github? There is a component:build label but it is not CI specific - What about MacOS testing - should it be sending notices? (answer - Probably) - Running ASV benchmarking (https://asv.readthedocs.io/en/latest/). It is done with SciPy, but it is fragile, not done nightly; need ability to run branches more robustly documentation on SciPy site https://github.com/scipy/scipy/tree/master/benchmarks - Hameer: f2py during testing is the system one, not the internal one Most of the remaining discussion was a meta-discussion about how the community will continue to decide priorities and influence how the full-time developers spend their time. - Setting up a community-driven roadmap would be useful - Be aware of the risks of having devoted developer time on a community project - Influence can be subtle: ideally, community writes roadmap, instead of simply commenting on proposal - Can we distill past lessons to inform future decisions? - In general, how to determine community priorities? - Constant communication paramount, looks like things are going in the right direction. Furher resources to consider: - How did Jupyter organize their roadmap (ask Brian Granger)? - How did Pandas run the project with a full time maintainer (Jeff Reback)? - Can we copy other projects' management guidelines? We did not set a time for another online discussion, since it was felt that maybe near/during the sprint in May would be appropriate. I apologize for any misrepresentation. Matti Picus ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] axis and keepdims arguments for generalized ufuncs
Hi All, When introducing the ``axes`` argument for generalized ufuncs, the plan was to eventually also add ``axis`` and ``keepdims`` for reduction-like gufuncs. I have now attempted to do so in https://github.com/numpy/numpy/pull/11018 It is not completely feature-compatible with reductions in that one cannot (yet) pass in a tuple or None to ``axis``. Comments most welcome. All the best, Marten ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions
I thought a bit further about this proposal: a disadvantage for matmul specifically is that is does not solve the need for `matvec`, `vecmat`, and `vecvec` gufuncs. That said, it might make sense to implement those as "pseudo-ufuncs" that just add a 1 in the right place and call `matmul`... -- Marten ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions
On Sun, Apr 29, 2018 at 2:48 AM Matti Picus wrote: > The proposed solution to issue #9029 is to extend the meaning of a > signature so "syntax like (n?,k),(k,m?)->(n?,m?) could mean that n and m > are optional dimensions; if missing in the input, they're treated as 1, and > then dropped from the output" I agree that this is an elegant fix for matmul, but are there other use-cases for "optional dimensions" in gufuncs? It feels a little wrong to add gufunc features if we can only think of one function that can use them. ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions
I think I’m -1 on this - this just makes things harder on the implementers of _array_ufunc__ who now might have to work out which signature matches. I’d prefer the solution where np.matmul is a wrapper around one of three gufuncs (or maybe just around one with axis insertion) - this is similar to how np.linalg already works. Eric On Mon, 30 Apr 2018 at 14:34 Stephan Hoyer wrote: > On Sun, Apr 29, 2018 at 2:48 AM Matti Picus wrote: > >> The proposed solution to issue #9029 is to extend the meaning of a >> signature so "syntax like (n?,k),(k,m?)->(n?,m?) could mean that n and m >> are optional dimensions; if missing in the input, they're treated as 1, and >> then dropped from the output" > > > I agree that this is an elegant fix for matmul, but are there other > use-cases for "optional dimensions" in gufuncs? > > It feels a little wrong to add gufunc features if we can only think of one > function that can use them. > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions
On 04/29/2018 05:46 AM, Matti Picus wrote: > In looking to solve issue #9028 "no way to override matmul/@ if > __array_ufunc__ is set", it seems there is consensus around the idea of > making matmul a true gufunc, but matmul can behave differently for > different combinations of array and vector: > > (n,k),(k,m)->(n,m) > (n,k),(k) -> (n) > (k),(k,m)->(m) > > Currently there is no way to express that in the ufunc signature. The > proposed solution to issue #9029 is to extend the meaning of a signature > so "syntax like (n?,k),(k,m?)->(n?,m?) could mean that n and m are > optional dimensions; if missing in the input, they're treated as 1, and > then dropped from the output" Additionally, there is an open pull > request #5015 "Add frozen dimensions to gufunc signatures" to allow > signatures like '(3),(3)->(3)'. How much harder would it be to implement multiple-dispatch for gufunc signatures, instead of modifying the signature to include `?` ? There was some discussion of this last year: http://numpy-discussion.10968.n7.nabble.com/Changes-to-generalized-ufunc-core-dimension-checking-tp42618p42638.html That sounded like a clean solution to me, although I'm a bit ignorant of the gufunc internals and the compatibility constraints. I assume gufuncs already have code to match the signature to the array dims, so it sounds fairly straightforward (I say without looking at any code) to do this in a loop over alternate signatures until one works. Allan ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions
On 01/05/18 01:45, Allan Haldane wrote: On 04/29/2018 05:46 AM, Matti Picus wrote: In looking to solve issue #9028 "no way to override matmul/@ if __array_ufunc__ is set", it seems there is consensus around the idea of making matmul a true gufunc, but matmul can behave differently for different combinations of array and vector: (n,k),(k,m)->(n,m) (n,k),(k) -> (n) (k),(k,m)->(m) Currently there is no way to express that in the ufunc signature. The proposed solution to issue #9029 is to extend the meaning of a signature so "syntax like (n?,k),(k,m?)->(n?,m?) could mean that n and m are optional dimensions; if missing in the input, they're treated as 1, and then dropped from the output" Additionally, there is an open pull request #5015 "Add frozen dimensions to gufunc signatures" to allow signatures like '(3),(3)->(3)'. How much harder would it be to implement multiple-dispatch for gufunc signatures, instead of modifying the signature to include `?` ? There was some discussion of this last year: http://numpy-discussion.10968.n7.nabble.com/Changes-to-generalized-ufunc-core-dimension-checking-tp42618p42638.html That sounded like a clean solution to me, although I'm a bit ignorant of the gufunc internals and the compatibility constraints. I assume gufuncs already have code to match the signature to the array dims, so it sounds fairly straightforward (I say without looking at any code) to do this in a loop over alternate signatures until one works. Allan ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion I will take a look at multiple-dispatch for gufuncs. The discussion also suggests using an axis kwarg when calling a gufunc for which there is PR #1108 (https://github.com/numpy/numpy/pull/11018) discussion). Matti ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
