Re: [Numpy-discussion] Proposal to drop python 2.4 support in numpy 1.8
So when upgrading everything you prefer to keep the bugs in 2.6 that were squashed in 2.7? Who has taught IT managers that older and more buggy versions of software are more professional and better for corporate environments? Sturla Den 14. des. 2012 kl. 05:14 skrev Raul Cota r...@virtualmaterials.com: +1 from me For what is worth, we are just moving forward from Python 2.2 / Numeric and are going to 2.6 and it has been rather painful because of the several little details of extensions and other subtleties. I believe we will settle there for a while. For companies like ours, it is a big problem to upgrade versions. There is always this or that hiccup that works great in a version but not so much in another and we also have all sorts of extensions. Raul On 13/12/2012 9:34 AM, Charles R Harris wrote: Time to raise this topic again. Opinions welcome. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal to drop python 2.4 support in numpy 1.8
On 14 Dec 2012 04:14, Raul Cota r...@virtualmaterials.com wrote: +1 from me For what is worth, we are just moving forward from Python 2.2 / Numeric and are going to 2.6 and it has been rather painful because of the several little details of extensions and other subtleties. I believe we will settle there for a while. For companies like ours, it is a big problem to upgrade versions. There is always this or that hiccup that works great in a version but not so much in another and we also have all sorts of extensions. Unfortunately (and I know this is a tradeoff), one consequence of this strategy is that you give up the chance to influence numpy development and avoid those hiccups in the first place. We try to catch things, but there's a *lot* more we can do if a bug gets noticed before it makes it into a final release, or multiple final releases... (This is why 1.7 has been dragging on - people testing the RCs found a number of places that it broke there code, so we're fixing numpy instead of them having to fix their code.) -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Failure in test_iterator.py at Travis
I only checked this build: https://secure.travis-ci.org/#!/certik/numpy/jobs/3656960 But that log clearly shows 'python setup.py install' being used instead of 'pip install'. How certain are you that your branch actually has my fix? -n On 14 Dec 2012 03:49, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Thu, Dec 13, 2012 at 7:16 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Dec 13, 2012 at 8:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Dec 13, 2012 at 7:23 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote: Hi, Another weird bug sometimes happen in numpy/core/tests/test_iterator.py, it looks like this: == FAIL: test_iterator.test_iter_array_cast -- Traceback (most recent call last): File /home/travis/virtualenv/python2.5/lib/python2.5/site-packages/nose/case.py, line 197, in runTest self.test(*self.arg) File /home/travis/virtualenv/python2.5/lib/python2.5/site-packages/numpy/core/tests/test_iterator.py, line 836, in test_iter_array_cast assert_equal(i.operands[0].strides, (-96,8,-32)) File /home/travis/virtualenv/python2.5/lib/python2.5/site-packages/numpy/testing/utils.py, line 252, in assert_equal assert_equal(actual[k], desired[k], 'item=%r\n%s' % (k,err_msg), verbose) File /home/travis/virtualenv/python2.5/lib/python2.5/site-packages/numpy/testing/utils.py, line 314, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: item=0 ACTUAL: 96 DESIRED: -96 -- But the problem is that there is no numpy/core/tests/test_iterator.py file in current branches This error was triggered for example by these PRs: https://github.com/numpy/numpy/pull/2765 https://github.com/numpy/numpy/pull/2815 and here are links to the failing Travis tests: https://travis-ci.org/certik/numpy/builds/3656959 https://travis-ci.org/numpy/numpy/builds/3330234 Any idea what is happening here and how to fix it? That should have been fixed by Nathaniel's travis fix. Hmm... And a quick check here shows pip not removing a previous 1.6.2 install. Maybe it is a pip version problem, pip 1.0.2 here. That's what I thought as well. I think we need to update the Travis script to manually remove the installed numpy. Btw, Travis seems to be using pip 1.2.1, at least according to: https://travis-ci.org/numpy/numpy/jobs/3330236 Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Building numpy with OpenBLAS
Hi. I'm trying to build numpy (1.6.2 and master from git) with OpenBLAS on Ubuntu server 11.10. I succeed with this just once and performance boost was really big for me, but unfortunately something went wrong with my application and I had to reinstall numpy. After that I couldn't reproduce this result and even just perform faster than default numpy installation with no external libraries anyhow. Now things went even worse. I assume that numpy built with BLAS and LAPACK should do dot operation faster than clean installation on relatively large matirces (say 2000 x 2000). Here I don't use OpenBLAS anyway. I install libblas-dev and liblapack-dev by apt-get and after that build numpy from sources / by pip (that doesn't matter for the result). Building tool reports that BLAS and LAPACK are detected on my system, so says numpy.distutils.system_info after installation. But matrix multiplication by dot takes the same time as clean installation (12 s vs 0.16 s with OpenBLAS). That's the first thing I'm wondering about. Nevertheless I tried to compile numpy with OpenBLAS only. I have forced this by setting ATLAS= BLAS=/usr/lib/libopenblas.a LAPACK=/usr/lib/libopenblas.a as I saw somewhere in the internet. I had installed numpy exactly this way at the first time when I was lucky. But now it doesn't work for me. I tryied installing OpenBLAS from sources and as libopenblas-dev ubuntu package. So how can I fix this? Many thanks in advance. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Building numpy with OpenBLAS
On 14.12.2012 10:17, Sergey Bartunov wrote: Now things went even worse. I assume that numpy built with BLAS and LAPACK should do dot operation faster than clean installation on relatively large matirces (say 2000 x 2000). Here I don't use OpenBLAS anyway. No, _dotblas is only built against ATLAS, MKL or Apple's accelerate framework. So with OpenBLAS you have to call e.g. DGEMM from OpenBLAS yourself instead of using np.dot. So how can I fix this? Many thanks in advance. You might fix NumPy to build _dotblas against OpenBLAS as well :) Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] np.seterr doesn't work for masked array?
On Fri, Dec 14, 2012 at 1:57 PM, Chao YUE chaoyue...@gmail.com wrote: Dear all, I tried to capture the zero divide error when I divide a masked array by another. It seems that np.seterr is not working for masked array? when I do np.divide on two masked array, it directly put the zero divides part as being masked. The np.seterr works if the two arrays for dividing are not masked arrays. could anyone explain? thanks!! numpy.ma uses np.seterr(divide='ignore', invalid='ignore') for most of its operations, so it is overriding your settings. This is usually desirable since many of the masked values will be trip these errors spuriously even though they will be masked out in the result. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Support for python 2.4 dropped. Should we drop 2.5 also?
We (STScI) are ending support for Python 2.5 in our stsci_python project and told our users as much last July. I have no objections to ending support for Python 2.5. Chris On Thu, Dec 13, 2012 at 12:38 PM, Charles R Harris charlesr.har...@gmail.com wrote: The previous proposal to drop python 2.4 support garnered no opposition. How about dropping support for python 2.5 also? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] an easy way to know if a functions works or not for masked array?
Dear numpy users, I think since long I am confused by if a function works as expected or not for masked array. like np.reshape works for masked array, but not np.sum (I mean, I expect np.sum to drop the masked elements when do summing, of course we have np.ma.sum). So I always use fuctions preceding by np.ma to make sure there is nothing going woring if I expect there will be masked array participating in the calculation in the data process chain. When I handle masked array, Before I use a function, normally I check if there is an equivalent np.ma function and if there is, I use np.mafunction; Then I check how the documentation says about np.func and np.ma.func and see there is anything mentioned explicitly on the handling of masked array. Howevery, In most cases I will try with simple arrays to test both np.ma.func and np.func before I use some function. but this is sometimes time consuming. Does anyone have similar situation? thanks! Chao -- *** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] np.seterr doesn't work for masked array?
Thanks. You mean actually when numpy handle masked array, it will first treat all the base data, and then apply the mask after the treatment. and normally the base data of maksed elements will very likely to intrigure these errors, and you will see a lot errory warning or print in the process, and will make it impossible to see the error information your want to see for those elements that are not masked but still can intriguer the error you would like to see or check. I didn't realize this. It's a good to overwrite the error setting. thanks for your explanation. Chao On Fri, Dec 14, 2012 at 3:15 PM, Robert Kern robert.k...@gmail.com wrote: On Fri, Dec 14, 2012 at 1:57 PM, Chao YUE chaoyue...@gmail.com wrote: Dear all, I tried to capture the zero divide error when I divide a masked array by another. It seems that np.seterr is not working for masked array? when I do np.divide on two masked array, it directly put the zero divides part as being masked. The np.seterr works if the two arrays for dividing are not masked arrays. could anyone explain? thanks!! numpy.ma uses np.seterr(divide='ignore', invalid='ignore') for most of its operations, so it is overriding your settings. This is usually desirable since many of the masked values will be trip these errors spuriously even though they will be masked out in the result. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- *** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] np.seterr doesn't work for masked array?
On Fri, Dec 14, 2012 at 2:40 PM, Chao YUE chaoyue...@gmail.com wrote: Thanks. You mean actually when numpy handle masked array, it will first treat all the base data, and then apply the mask after the treatment. and normally the base data of maksed elements will very likely to intrigure these errors, and you will see a lot errory warning or print in the process, and will make it impossible to see the error information your want to see for those elements that are not masked but still can intriguer the error you would like to see or check. I didn't realize this. It's a good to overwrite the error setting. Precisely. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Building numpy with OpenBLAS
Hi Sergey: I recently ran into similar problems with ACML. See an original bug report (https://github.com/numpy/numpy/issues/2728) documentation fix (https://github.com/numpy/numpy/pull/2809). Personally, I ended up using a patch similar to https://github.com/numpy/numpy/pull/2751 to force NumPy to respect site.cfg (so that I could put the libacml in the [blas_opt] [lapack_opt] sections). But this seems unlikely to get merged into NumPy as it changes the behavior of site.cfg. Instead I think we should discuss adding a have cblas flag of some sort to the [blas] section so that the user can still get _dotblas to compile. -Brad On Friday, December 14, 2012 at 1:17 AM, Sergey Bartunov wrote: Hi. I'm trying to build numpy (1.6.2 and master from git) with OpenBLAS on Ubuntu server 11.10. I succeed with this just once and performance boost was really big for me, but unfortunately something went wrong with my application and I had to reinstall numpy. After that I couldn't reproduce this result and even just perform faster than default numpy installation with no external libraries anyhow. Now things went even worse. I assume that numpy built with BLAS and LAPACK should do dot operation faster than clean installation on relatively large matirces (say 2000 x 2000). Here I don't use OpenBLAS anyway. I install libblas-dev and liblapack-dev by apt-get and after that build numpy from sources / by pip (that doesn't matter for the result). Building tool reports that BLAS and LAPACK are detected on my system, so says numpy.distutils.system_info after installation. But matrix multiplication by dot takes the same time as clean installation (12 s vs 0.16 s with OpenBLAS). That's the first thing I'm wondering about. Nevertheless I tried to compile numpy with OpenBLAS only. I have forced this by setting ATLAS= BLAS=/usr/lib/libopenblas.a LAPACK=/usr/lib/libopenblas.a as I saw somewhere in the internet. I had installed numpy exactly this way at the first time when I was lucky. But now it doesn't work for me. I tryied installing OpenBLAS from sources and as libopenblas-dev ubuntu package. So how can I fix this? Many thanks in advance. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org (mailto:NumPy-Discussion@scipy.org) http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Failure in test_iterator.py at Travis
On Fri, Dec 14, 2012 at 12:32 AM, Nathaniel Smith n...@pobox.com wrote: I only checked this build: https://secure.travis-ci.org/#!/certik/numpy/jobs/3656960 But that log clearly shows 'python setup.py install' being used instead of 'pip install'. How certain are you that your branch actually has my fix? Right. So is this one (using python setup.py install): https://travis-ci.org/numpy/numpy/jobs/3330235 No wonder that it fails. Since I rebased this on top of master and the master has this fix, and so does the 1.7 branch, I can't explain it. I know that the master at certik/numpy github does not have that fix (since I didn't push in there lately), but I don't see how that could matter, unless there is a bug at Travis. The branches that I created the pull requests from should have that fix. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Failure in test_iterator.py at Travis
The top of the build log has the actual git command they used to check out the source - it's some clever GitHub thing that gives the same thing as pressing the green button would iirc. You could copy the commands from the log to check that out locally though and see what the .travis.tml actually says. And if it's wrong poke around in the git history to figure out why. -n On 14 Dec 2012 17:20, Ondřej Čertík ondrej.cer...@gmail.com wrote: On Fri, Dec 14, 2012 at 12:32 AM, Nathaniel Smith n...@pobox.com wrote: I only checked this build: https://secure.travis-ci.org/#!/certik/numpy/jobs/3656960 But that log clearly shows 'python setup.py install' being used instead of 'pip install'. How certain are you that your branch actually has my fix? Right. So is this one (using python setup.py install): https://travis-ci.org/numpy/numpy/jobs/3330235 No wonder that it fails. Since I rebased this on top of master and the master has this fix, and so does the 1.7 branch, I can't explain it. I know that the master at certik/numpy github does not have that fix (since I didn't push in there lately), but I don't see how that could matter, unless there is a bug at Travis. The branches that I created the pull requests from should have that fix. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Failure in test_iterator.py at Travis
On Fri, Dec 14, 2012 at 9:49 AM, Nathaniel Smith n...@pobox.com wrote: The top of the build log has the actual git command they used to check out the source - it's some clever GitHub thing that gives the same thing as pressing the green button would iirc. You could copy the commands from the log to check that out locally though and see what the .travis.tml actually says. And if it's wrong poke around in the git history to figure out why. You are right! I followed the one: https://travis-ci.org/numpy/numpy/jobs/3330235 and .travis.yml indeed contains the old python setup.py install! I just don't understand why. My strong suspicion is that the actual commands are *not* equivalent to the github green merge button. It took the latest patches from the PR (which do not contain the .travis.yml fix), and merged into: commit 089bfa5865cd39e2b40099755e8563d8f0d04f5f Merge: 1065cc5 df2958e Author: Ralf Gommers ralf.gomm...@googlemail.com Date: Sat Nov 24 02:54:48 2012 -0800 Merge pull request #2766 from g2p/master Assume we can use sys.stdout.fileno() and friends. I have no idea why. Unless --- it is merging into the master, which was current at the time the PR was created. That would explain it this particular failure. Ok. And finally, the one at certik/numpy fails, because that one is checking out the sources from certik/numpy, but those do not have the improvement. It is then the Travis's bug, that it links to certik/numpy log for PRs at numpy/numpy. So I think that all is explained now. Ondrej ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] MKL licenses for core scientific Python projects
Hi all, Intel has offered to provide free MKL licenses for main contributors to scientific Python projects - at least those listed at numfocus.org/projects/. Licenses for all OSes that are required can be provided, the condition is that they're used for building/testing our projects and not for broader purposes. If you're interested, please let me know your full name and what OS you need a license for. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal to drop python 2.4 support in numpy 1.8
Point well taken. It is always a tradeoff / balancing act where you can have 'anything' but not 'everything'. Where would the fun be if we could have everything :) ? . In our situation, there were a couple of extensions that did not work (at least out of the box) in Python 2.7. Raul On 14/12/2012 1:09 AM, Sturla Molden wrote: So when upgrading everything you prefer to keep the bugs in 2.6 that were squashed in 2.7? Who has taught IT managers that older and more buggy versions of software are more professional and better for corporate environments? Sturla Den 14. des. 2012 kl. 05:14 skrev Raul Cota r...@virtualmaterials.com: +1 from me For what is worth, we are just moving forward from Python 2.2 / Numeric and are going to 2.6 and it has been rather painful because of the several little details of extensions and other subtleties. I believe we will settle there for a while. For companies like ours, it is a big problem to upgrade versions. There is always this or that hiccup that works great in a version but not so much in another and we also have all sorts of extensions. Raul On 13/12/2012 9:34 AM, Charles R Harris wrote: Time to raise this topic again. Opinions welcome. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] MKL licenses for core scientific Python projects
Ralf, Do these licenses allow fully free distribution of binaries? And are those binaries themselves redistributive? I.e. with py2exe and friends? If so, that could be nice. -Chris On Dec 14, 2012, at 1:01 PM, Ralf Gommers ralf.gomm...@gmail.com wrote: Hi all, Intel has offered to provide free MKL licenses for main contributors to scientific Python projects - at least those listed at numfocus.org/projects/. Licenses for all OSes that are required can be provided, the condition is that they're used for building/testing our projects and not for broader purposes. If you're interested, please let me know your full name and what OS you need a license for. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
