Re: [Numpy-discussion] Numpy 1.11.0b2 released

[Andreas Mueller]

Thanks, that is very helpful!

On 01/30/2016 01:40 PM, Jeff Reback wrote:

just my 2c

it's fairly straightforward to add a test to the Travis matrix to grab 
numpy wheels built numpy wheels (works for conda or pip installs).


so in pandas we r testing 2.7/3.5 against numpy master continuously

https://github.com/pydata/pandas/blob/master/ci/install-3.5_NUMPY_DEV.sh

On Jan 30, 2016, at 1:16 PM, Nathaniel Smith > wrote:


On Jan 30, 2016 9:27 AM, "Ralf Gommers" > wrote:

>
>
>
> On Fri, Jan 29, 2016 at 11:39 PM, Nathaniel Smith > wrote:

>>
>> It occurs to me that the best solution might be to put together a 
.travis.yml for the release branches that does: "for pkg in 
IMPORTANT_PACKAGES: pip install $pkg; python -c 'import pkg; pkg.test()'"
>> This might not be viable right now, but will be made more viable 
if pypi starts allowing official Linux wheels, which looks likely to 
happen before 1.12... (see PEP 513)

>>
>> On Jan 29, 2016 9:46 AM, "Andreas Mueller" > wrote:

>> >
>> > Is this the point when scikit-learn should build against it?
>>
>> Yes please!
>>
>> > Or do we wait for an RC?
>>
>> This is still all in flux, but I think we might actually want a 
rule that says it can't become an RC until after we've tested 
scikit-learn (and a list of similarly prominent packages). On the 
theory that RC means "we think this is actually good enough to 
release" :-). OTOH I'm not sure the alpha/beta/RC distinction is very 
helpful; maybe they should all just be betas.

>>
>> > Also, we need a scipy build against it. Who does that?
>>
>> Like Julian says, it shouldn't be necessary. In fact using old 
builds of scipy and scikit-learn is even better than rebuilding them, 
because it tests numpy's ABI compatibility -- if you find you *have* 
to rebuild something then we *definitely* want to know that.

>>
>> > Our continuous integration doesn't usually build scipy or numpy, 
so it will be a bit tricky to add to our config.
>> > Would you run our master tests? [did we ever finish this 
discussion?]

>>
>> We didn't, and probably should... :-)
>
> Why would that be necessary if scikit-learn simply tests 
pre-releases of numpy as you suggested earlier in the thread (with 
--pre)?

>
> There's also https://github.com/MacPython/scipy-stack-osx-testing 
by the way, which could have scikit-learn and scikit-image added to it.

>
> That's two options that are imho both better than adding more 
workload for the numpy release manager. Also from a principled point 
of view, packages should test with new versions of their 
dependencies, not the other way around.


Sorry, that was unclear. I meant that we should finish the 
discussion, not that we should necessarily be the ones running the 
tests. "The discussion" being this one:


https://github.com/numpy/numpy/issues/6462#issuecomment-148094591
https://github.com/numpy/numpy/issues/6494

I'm not saying that the release manager necessarily should be running 
the tests (though it's one option). But the 1.10 experience seems to 
indicate that we need *some* process for the release manager to make 
sure that some basic downstream testing has happened. Another option 
would be keeping a checklist of downstream projects and making sure 
they've all checked in and confirmed that they've run tests before 
making the release.


-n

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org 
https://mail.scipy.org/mailman/listinfo/numpy-discussion



___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.11.0b2 released

[Andreas Mueller]

On 02/01/2016 04:25 PM, Ralf Gommers wrote:


It would be nice but its not realistic, I doubt most upstreams
that are
not themselves major downstreams are even subscribed to this list.


I'm pretty sure that some core devs from all major scipy stack 
packages are subscribed to this list.
Well, I don't think anyone else from sklearn picked up on this, and I 
myself totally forgot the issue for the last two weeks.



I think continuously testing against numpy master might actually be 
feasible for us, but I'm not entirely sure
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Linking other libm-Implementation

[Nils Becker]

2016-02-09 18:02 GMT+01:00 Gregor Thalhammer :
>> It is not suitable as a standard for numpy.
>
> Why should numpy not provide fast transcendental math functions? For
linear algebra it supports fast implementations, even non-free (MKL).
Wouldn’t it be nice if numpy outperforms C?

Floating point operations that make use of vector extensions of modern
processors may behave subtly different. This especially concerns floating
point exceptions, where sometimes silent infinities are generated instead
of raising a divide-by-zero exception (best description I could find on the
spot:
https://randomascii.wordpress.com/2012/04/21/exceptional-floating-point/,
also see the notes on C99-compliance of the new vector expressions in
glibc: https://sourceware.org/glibc/wiki/libmvec).
I think the default in numpy should strive to be mostly standard compliant.
But of course an option to activate vector math operations would be nice -
if that is necessary with packages like numexpr is another question.
One other point is the extended/long double type which is normally not
supported by those libraries (as vector extensions cannot handle them).

> Intel publishes accuracy/performance charts for VML/MKL:
>
https://software.intel.com/sites/products/documentation/doclib/mkl/vm/functions/_accuracyall.html
>
> For GNU libc it is more difficult to find similarly precise data, I only
could find:
>
http://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html

On Tue, Feb 9, 2016 at 7:06 AM, Daπid  wrote:
> I did some digging, and I found this:
>
>
http://julia-programming-language.2336112.n4.nabble.com/Is-the-accuracy-of-Julia-s-elementary-functions-exp-sin-known-td32736.html

Thank you for looking that up! I did not knew about the stuff published by
Intel yet.

2016-02-09 20:13 GMT+01:00 Matthew Brett :
> So GNU libm has max error <= 0.5 ULP, openlibm has <= 1 ULP, and OSX
> is (almost always) somewhere in-between.
>
> So, is <= 1 ULP good enough?

Calculating transcendental functions correctly rounded is very, very hard
and to my knowledge there is no complete libm implementation that
guarantees the necessary accuracy for all possible inputs. One effort
was/is the Correctly Rounded LibM (crlibm [1]) which tried to prove the
accuracy of their algorithms. However, the performance impact to achieve
that last ulp in all rounding modes can be severe.
Assessing accuracy of a function implementation is hard. Testing all
possible inputs is not feasible (2^32/64 for single/double) and proving
accuracy bounds may be even harder.
Most of the time one samples accuracy with random numbers from a certain
range. This generates tables like the ones for GNU libm or Intel.
This is a kind of "faithful" accuracy as you believe that the accuracy you
tested on a sample extends to the whole argument range. The error in worst
case may be (much) bigger.

That being said, I believe the values given by GNU libm for example are
very trustworthy. libm is not always correctly rounded (which would be <=
0.5ulp in round-to-nearest), however, the error bounds given in the table
seem to cover all worst cases.
Common single-argument functions (sin, cos) are correctly rounded and even
complex two-argument functions (cpow) are at most 5ulp off. I do not think
that other implementations are more accurate.
So libm is definitely good enough, accuracy-wise.

In any case I would like to build a testing framework to compare some libms
and check accuracy/performance (at least Intel has a history of
underestimating their error bounds in transcendental functions [2]). crlibm
offers worst-case arguments for some functions which could be used to
complement randomized sampling. Maybe I have some time in the next weeks...

[1] http://lipforge.ens-lyon.fr/www/crlibm/
[2]
https://randomascii.wordpress.com/2014/10/09/intel-underestimates-error-bounds-by-1-3-quintillion/
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoC?

[Ralf Gommers]

On Tue, Feb 9, 2016 at 1:02 AM, Chris Barker  wrote:

> As you can see in the timeline:
>
> https://developers.google.com/open-source/gsoc/timeline
>
> We are now in the stage where mentoring organizations are getting their
> act together. So the question now is -- are there folks that want to mentor
> for numpy projects? It can be rewarding, but it's a pretty big commitment
> as well, and, I suppose depending on the project, would require some good
> knowledge of the innards of numpy -- there are not a lot of those folks out
> there that have that background.
>

Note that we have always done a combined numpy/scipy ideas page and
submission. For really good students numpy may be the right challenge, but
in general scipy is easier to get started on. So we have difficult project
ideas for both, but easy/intermediate ones will most likely be for scipy.


>
> So to students, I suggest you keep an eye out, and engage a little later
> on in the process.
>
> That being said, if you have a idea for a numpy improvement you'd like to
> work on , by all means propose it and maybe you'll get a mentor or two
> excited.
>
> -CHB
>
>
>
>
>
> On Mon, Feb 8, 2016 at 3:33 PM, SMRUTI RANJAN SAHOO 
> wrote:
>
>> sir actually i am interested  very much . so can you help me about this
>> or suggest some , so that i  can contribute .
>>
>
Hi Smruti, I suggest you look at the numpy or scipy issues on Github, and
start with one labeled "easy-fix".


>
>> Thanks  & Regards,
>> Smruti Ranjan Sahoo
>>
>> On Tue, Feb 9, 2016 at 1:58 AM, Chris Barker 
>> wrote:
>>
>>>
>>> I think the real challenge is having folks with the time to really put
>>> into mentoring, but if folks want to do it -- numpy could really benefit.
>>>
>>> Maybe as a python.org sub-project?
>>>
>>
Under the PSF umbrella has always worked very well, both in terms of
communication quality and of getting the amount of slots we wanted, so yes.


>
>>> https://wiki.python.org/moin/SummerOfCode/2016
>>>
>>> Deadlines are approaching -- so I thought I'd ping the list and see if
>>> folks are interested.
>>> ANyone interested in Google Summer of Code this year?
>>>
>>
Yes, last year we had quite a productive GSoC, so I had planned to organize
it along the same lines again (with an updated ideas page of course).

Are you maybe interested in co-organizing or mentoring Chris? Updating the
ideas page, proposal reviewing and interviewing students via video calls
can be time-consuming, and mentoring definitely is, so the more the merrier.

Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoC?

[Ralf Gommers]

On Wed, Feb 10, 2016 at 11:48 PM, Chris Barker 
wrote:

> Thanks Ralf,
>
> Note that we have always done a combined numpy/scipy ideas page and
>> submission. For really good students numpy may be the right challenge, but
>> in general scipy is easier to get started on.
>>
>
> yup -- good idea. Is there a page ready to go, or do we need to get one
> up? (I don't even know where to put it...)
>

This is last year's page:
https://github.com/scipy/scipy/wiki/GSoC-2015-project-ideas

Some ideas have been worked on, others are still relevant. Let's copy this
page to -2016- and start editing it and adding new ideas. I'll start right
now actually.


>
>
>> Under the PSF umbrella has always worked very well, both in terms of
>> communication quality and of getting the amount of slots we wanted, so yes.
>>
>
> hmm, looking here:
>
> https://wiki.python.org/moin/SummerOfCode/2016#Sub-orgs
>
> it seems it's time to get started. and I _think_ our ideas page can go on
> that Wiki.
>
>
>> Are you maybe interested in co-organizing or mentoring Chris? Updating
>> the ideas page, proposal reviewing and interviewing students via video
>> calls can be time-consuming, and mentoring definitely is, so the more the
>> merrier.
>>
>
> I would love to help -- though I don't think I can commit to being a
> full-on mentor.
>
> If we get a couple people to agree to mentor,
>

That's always the tricky part. We normally let people indicate whether
they're interested in mentoring for specific project ideas on the ideas
page.


> then we can get ourselves setup up with the PSF.
> 
>

That's the easiest part, takes one email and one wiki page edit:)

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[Charles R Harris]

On Wed, Feb 10, 2016 at 2:58 PM, Pauli Virtanen  wrote:

> 10.02.2016, 04:09, Charles R Harris kirjoitti:
> > I'm pleased to announce the release of NumPy 1.11.0b3. This beta contains
> [clip]
> > Please test, hopefully this will be that last beta needed.
>
> FWIW, https://travis-ci.org/pv/testrig/builds/108384173


Thanks Pauli, very interesting.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[Pauli Virtanen]

10.02.2016, 04:09, Charles R Harris kirjoitti:
> I'm pleased to announce the release of NumPy 1.11.0b3. This beta contains
[clip]
> Please test, hopefully this will be that last beta needed.

FWIW, https://travis-ci.org/pv/testrig/builds/108384173


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoC?

[Chris Barker]

Thanks Ralf,

Note that we have always done a combined numpy/scipy ideas page and
> submission. For really good students numpy may be the right challenge, but
> in general scipy is easier to get started on.
>

yup -- good idea. Is there a page ready to go, or do we need to get one up?
(I don't even know where to put it...)


> Under the PSF umbrella has always worked very well, both in terms of
> communication quality and of getting the amount of slots we wanted, so yes.
>

hmm, looking here:

https://wiki.python.org/moin/SummerOfCode/2016#Sub-orgs

it seems it's time to get started. and I _think_ our ideas page can go on
that Wiki.


> Are you maybe interested in co-organizing or mentoring Chris? Updating the
> ideas page, proposal reviewing and interviewing students via video calls
> can be time-consuming, and mentoring definitely is, so the more the merrier.
>

I would love to help -- though I don't think I can commit to being a
full-on mentor.

If we get a couple people to agree to mentor, then we can get ourselves
setup up with the PSF.

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoC?

[Chris Barker]

>
> We might consider adding "improve duck typing for numpy arrays"
>

care to elaborate on that one?

I know it come up on here that it would be good to have some code in numpy
itself that made it easier to make array-like objects (I.e. do indexing the
same way) Is that what you mean?

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Deprecating `numpy.iterable`

[Joseph Fox-Rabinovitz]

I have created a PR to deprecate `np.iterable`
(https://github.com/numpy/numpy/pull/7202). It is a very old function,
introduced as a utility in 2005
(https://github.com/numpy/numpy/commit/052a7b2e3276a303be1083022fc24d43084d2e14),
and there is no good reason for it to be part of the public API. It is
used internally 10 times within numpy. I have repaced those usages
with a private function `np.lib.function_base._iterable` and added a
`DeprecationWarning` to the public function.

Is there anyone that objects to deprecating this function?

Regards,

-Joseph
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] NumPy 1.11.0b3 released.

[josef.pktd]

On Wed, Feb 10, 2016 at 5:36 PM, Charles R Harris  wrote:

>
>
> On Wed, Feb 10, 2016 at 2:58 PM, Pauli Virtanen  wrote:
>
>> 10.02.2016, 04:09, Charles R Harris kirjoitti:
>> > I'm pleased to announce the release of NumPy 1.11.0b3. This beta
>> contains
>> [clip]
>> > Please test, hopefully this will be that last beta needed.
>>
>> FWIW, https://travis-ci.org/pv/testrig/builds/108384173
>
>
> Thanks Pauli, very interesting.
>


Thanks Pauli, me too

is this intended?:

return np.r_[[np.nan] * head, x, [np.nan] * tail]
TypeError: 'numpy.float64' object cannot be interpreted as an index


In the old times of Python 2.x, statsmodels avoided integers so we don't
get accidental integer division.
Python wanted float() everywhere. Looks like numpy wants int() everywhere.
(fixed in statsmodels master)


Josef






>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoC?

[Ralf Gommers]

On Wed, Feb 10, 2016 at 11:55 PM, Ralf Gommers 
wrote:

>
>
> On Wed, Feb 10, 2016 at 11:48 PM, Chris Barker 
> wrote:
>
>> Thanks Ralf,
>>
>> Note that we have always done a combined numpy/scipy ideas page and
>>> submission. For really good students numpy may be the right challenge, but
>>> in general scipy is easier to get started on.
>>>
>>
>> yup -- good idea. Is there a page ready to go, or do we need to get one
>> up? (I don't even know where to put it...)
>>
>
> This is last year's page:
> https://github.com/scipy/scipy/wiki/GSoC-2015-project-ideas
>
> Some ideas have been worked on, others are still relevant. Let's copy this
> page to -2016- and start editing it and adding new ideas. I'll start right
> now actually.
>

OK first version:
https://github.com/scipy/scipy/wiki/GSoC-2016-project-ideas
I kept some of the ideas from last year, but removed all potential mentors
as the same people may not be available this year - please re-add
yourselves where needed.

And to everyone who has a good idea, and preferably is willing to mentor
for that idea: please add it to that page.

Ralf



>
>
>>
>>
>>> Under the PSF umbrella has always worked very well, both in terms of
>>> communication quality and of getting the amount of slots we wanted, so yes.
>>>
>>
>> hmm, looking here:
>>
>> https://wiki.python.org/moin/SummerOfCode/2016#Sub-orgs
>>
>> it seems it's time to get started. and I _think_ our ideas page can go on
>> that Wiki.
>>
>>
>>> Are you maybe interested in co-organizing or mentoring Chris? Updating
>>> the ideas page, proposal reviewing and interviewing students via video
>>> calls can be time-consuming, and mentoring definitely is, so the more the
>>> merrier.
>>>
>>
>> I would love to help -- though I don't think I can commit to being a
>> full-on mentor.
>>
>> If we get a couple people to agree to mentor,
>>
>
> That's always the tricky part. We normally let people indicate whether
> they're interested in mentoring for specific project ideas on the ideas
> page.
>
>
>> then we can get ourselves setup up with the PSF.
>> 
>>
>
> That's the easiest part, takes one email and one wiki page edit:)
>
> Ralf
>
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] GSoC?

[Stephan Hoyer]

On Wed, Feb 10, 2016 at 3:02 PM, Ralf Gommers 
wrote:

> OK first version:
> https://github.com/scipy/scipy/wiki/GSoC-2016-project-ideas
> I kept some of the ideas from last year, but removed all potential mentors
> as the same people may not be available this year - please re-add
> yourselves where needed.
>
> And to everyone who has a good idea, and preferably is willing to mentor
> for that idea: please add it to that page.
>
> Ralf
>

I removed the "Improve Numpy datetime functionality" project, since the
relevant improvements have mostly already made it into NumPy 1.11.

We might consider adding "improve duck typing for numpy arrays" if any GSOC
students are true masochists ;). I could potentially be a mentor for this
one, though of course Nathaniel is the obvious choice.

Stephan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion