[Numpy-discussion] testing

[Chip Parker]

Let's see if the instructions enclosed in
http://www.jamesh.id.au/articles/mailman-spamassassin/ which was
written in 2003 are still biting us.

-- 
Chip Parker
DevOps Engineer
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Ondřej Čertík]

Hi Travis,

On Sun, Sep 13, 2015 at 4:51 PM, Travis Oliphant  wrote:
> Hey all,
>
> I just wanted to clarify, that I am very excited about a few ideas I have
> --- but I don't have time myself to engage in the community process to get
> these changes into NumPy. However, those are real processes --- I've
> been coaching a few people in those processes for the past several years
> already.
>
> So, rather than do nothing, what I'm looking to do is to work with a few
> people who I can share my ideas with, get excited about the ideas, and then
> who will work with the community to get them implemented.   That's what I
> was announcing and talking about yesterday --- looking for interested people
> who want to work on NumPy *with* the NumPy community.
>
> In my enthusiasm, I realize that some may have mis-understood my intention.
> There is no 'imminent' fork, nor am I planning on doing some crazy amount of
> work that I then try to force on other developers of NumPy.
>
> What I'm planning to do is find people to train on NumPy code base (people
> to increase the diversity of the developers would be ideal -- but hard to
> accomplish).  I plan to train them on NumPy based on my experience, and on
> what I think should be done --- and then have *them* work through the
> community process and engage with others to get consensus (hopefully not
> losing too much in translation in the process --- but instead getting even
> better).
>
> During that process I will engage as a member of the community and help
> write NEPs and other documents and help clarify where it makes sense as I
> can.   I will be filtering for people that actually want to see NumPy get
> better.Until I identify the people and work with them, it will be hard
> to tell how this will best work.   So, stay tuned.
>
> If all goes well, what you should see in a few weeks time are specific
> proposals, a branch or two, and the beginnings of some pull requests.If
> you don't see that, then I will not have found the right people to help me,
> and we will all continue to go back to searching.
>
> While I'm expecting the best, in the worst case, we get additional people
> who know the NumPy code base and can help squash bugs as well as implement
> changes that are desired.Three things are needed if you want to
> participate in this:  1) A willingness to work with the open source
> community, 2) a deep knowledge of C and in-particular CPython's brand of C,
> and 3) a willingness to engage with me, do a mind-meld and dump around the
> NumPy code base, and then improve on what is in my head with the rest of the
> community.


I don't have time to do the work myself, but I'll be happy to help in
terms of the community. Travis has paid me in the past to do the NumPy
1.7 release, and implement/fix some things that Travis had in mind. I
had no prior involvement with the NumPy codebase / community, and
within few months I had to get myself familiar with the code, fix
release critical bugs, get my PRs accepted by the community,
eventually get push access (from the core developers, not from Travis)
so that I can merge other people's patches, and most importantly, make
the community allow me to actually do the release, even though I came
from outside. As far as I know, Travis didn't have to pull any strings
for me, and as far as I know, there was no friction in the community
either --- there was a job to be done, we got it done, and that was
it.

So I'll be happy to share my experience and help the person (if coming
from outside) how to do this. In this case a push access is not
necessary, so it's just about sending high quality PRs and good
communication.

Ondrej
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Matthew Brett]

On Mon, Sep 14, 2015 at 9:42 AM, Matthew Brett  wrote:
> On Mon, Sep 14, 2015 at 3:47 AM, Julian Taylor
>  wrote:
>> as due to the many incompatiblities in 1.10 many will likely not be able
>> to update anytime soon, so I think putting out another 1.9.3 bugfix
>> release would be a good idea.
>> I can probably do the release management for it, though I haven't been
>> keeping up with bugfixes recently so, please comment on important issues
>> you would want fixed.
>> The np.where upcast regression and np.nanmedian issues come to my mind
>> as should be fixed.
>
> OK - fair enough - it would be good to do this soon as possible, as
> Python 3.5 is currently the default download from the Python.org site
> (rather than Python 2.7) and most other binary installers depend on
> numpy.   Is there anything I can do to help?

For example, how about a rush-out 1.9.3 release with just the gzip
patch, and a very-soon release with further patches?

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Christoph Gohlke]

On 9/14/2015 3:47 AM, Julian Taylor wrote:

as due to the many incompatiblities in 1.10 many will likely not be able
to update anytime soon, so I think putting out another 1.9.3 bugfix
release would be a good idea.
I can probably do the release management for it, though I haven't been
keeping up with bugfixes recently so, please comment on important issues
you would want fixed.
The np.where upcast regression and np.nanmedian issues come to my mind
as should be fixed.

On 09/14/2015 11:21 AM, Carl Kleffner wrote:

I would like to add patches for the mingwpy windows build as well. There
is no Python-3.5 build so far.

Carlkl

2015-09-14 10:46 GMT+02:00 Robert Kern >:

 On Mon, Sep 14, 2015 at 9:32 AM, Matthew Brett
 > wrote:
 >
 > Hi,
 >
 > On Mon, Sep 14, 2015 at 1:22 AM, David Cournapeau > wrote:
 > >
 > >
 > > On Mon, Sep 14, 2015 at 9:18 AM, Matthew Brett >
 > > wrote:
 > >>
 > >> Hi,
 > >>
 > >> I'm just building numpy 1.9.2 for Python 3.5 (just released).
 > >>
 > >> In order to get the tests to pass on Python 3.5, I need to cherry pick
 > >> commit 7d6aa8c onto the 1.9.2 tag position.
 > >>
 > >> Does anyone object to me uploading a wheel built from this patched
 > >> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to get
 > >> the ball rolling for Python 3.5 binary wheels.
 > >
 > >
 > > Why not releasing this as 1.9.3 ? It does not need to be a full release
 > > (with binaries and all), but having multiple sources for a given tag is
 > > confusing.
 >
 > Generally OK with me, but it's quite a bit of extra work for very
 > little gain.  We'd have to tag, release a source tarball and OSX
 > wheels, at least.

 I think it's highly desirable that we also have a *source* release
 that builds on Python 3.5, irrespective of whether or not we have
 binary wheels for a couple of platforms up for Python 3.5. So I
 would encourage a quick 1.9.3 release that incorporates this patch.

 --
 Robert Kern



Support for Visual Studio 2015 and Intel Fortran 16 for Python 3.5 on 
Windows would also be nice.


I am using:

DEV: Replace deprecated options for ifort
  

remove /GL for vs2015 in check_long_double_representation
  

Enable Visual Studio 2015 C99 features
  

BLD: revert C99 complex for msvc14
  


Christoph
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

[Sturla Molden]

On 14/09/15 10:34, Antoine Pitrou wrote:


Currently we don't provide those APIs on the GPU, since MT is much too
costly there.

If Numpy wanted to switch to a different generator, and if Numba wanted
to remain compatible with Numpy, one of the PCG functions would be an
excellent choice (also for CPU performance, incidentally).


We have discussed allowing plugable PRNGs in NumPy on multiple occations.

I don't think NumPy should change its default PRNG. The Mersenne Twister 
MT19937 is state of the art of numerical work. It has excellent 
numerical accuracy, a huge period, and is very fast. For most users of 
NumPy, the MT19937 is the first and last word that needs to be said 
about pseudorandom number generators. We have the default that most 
users of NumPy will ever want, unless severe flaws are discovered in the 
algorithm (not very likely).


But there are occations where it does not work well. For example for 
parallel simulations it is (sometimes) nice to use DCMT instead of a 
vanilla MT19937. There are a lot of other generators that NumPy should 
consider as well, including the one you mention and Marsaglia's 
generators. We need to refactor randomkit to use a plugable entropy 
source instead of its internal MT19937. This is actually very easy to 
implement. In that case a user could just provide a Cython class or 
callback function (C, Fortran or Python) for generating a random ulong32.


The discussion on python-ideas refers to cryptography. That is not 
equally relevant for NumPy. But with a plugable design we can let NumPy 
users use os.urandom as well.



Sturla


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] First release candidate for the 1.5.0 series

[Thomas Caswell]

Please give it a try! (linux64 conda builds are available on the tacaswell
anaconda.org channel)

https://github.com/matplotlib/matplotlib/releases/tag/v1.5.0rc1

This release contains many new features.  The highlights include:

  - the object oriented API will now automatically re-draw the figure when
working in the command line
  - automatic unpacking of labeled data into most plotting functions
  - arbitrary style cycling
  - four new perceptually linear color maps
  - mouse-over for pixel values with `imshow`
  - many new rcparams

In addition there are significant improvements to `nbagg` and a complete
overhaul of the c++ bindings to AGG.

Please see the drafts of the [full whats new](
http://matplotlib.org/devdocs/users/whats_new.html#new-in-matplotlib-1-5)
and [api changes](
http://matplotlib.org/devdocs/api/api_changes.html#changes-in-1-5-0)
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] new ufunc implementations for object arrays

[Allan Haldane]

Hello all,

I've just submitted a PR which overhauls the implementation of ufuncs
for object types.
https://github.com/numpy/numpy/pull/6320

The motivation for this is that many ufuncs (eg all transcendental
functions) can't handle objects. This PR will also make object arrays
more customizable, as all ufuncs can now be overridden by the user by
defining methods on the object.

I'd like to see if there's enough interest/agreement for me to finish
it, and also to have more eyes look over the pure-python ufunc
implementations since I am sure I have missed some edge cases. I would
love suggestions, and I need help for a few cases I couldn't figure out
(eg np.nextafter, np.spacing, np.cbrt. Also, np.expm1 np.signbit and
others are a bit suspect).

Allan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] testing

[Chip Parker]

This would be an email.

-- 
Chip Parker
DevOps Engineer
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Nathaniel Smith]

Hi all,

Thanks, Travis, for the followup. I know some people were confused or
concerned by some points in Travis’s recent emails (as was I,
initially), but after checking in with Travis and the NumPy steering
council, it sounds like the main points of possible confusion are
actually things where we at least are actually on the same page. So to
avoid any uncertainty or miscommunication, I just want to reiterate
some hopefully-uncontroversial points here:

- There is currently no plan whatsoever for NumPy to drop Python 2
support; something like ~80% of our users are still using Python 2,
and we anticipate that both Python 2 and Python 3 will continue to
receive full support for the foreseeable future.

- While it is possible that there will eventually be a
compatibility-breaking “NumPy 2.0” release, it won’t happen without a
community consensus that this is a good idea, and that consensus has
not yet been reached.

- As a clarification on how NumPy governance works: While we all
continue to be grateful to Travis for his past contributions, in terms
of formal authority he stepped down from his leadership role in the
project several years ago, and the project has since switched to a
community-driven governance model [1]. Under this model, all
contributors participate as equal peers, and (outside of specific
exceptional circumstances) all contributors should be assumed to be
speaking only on behalf of themselves, not the project as a whole.

- Discussion about the best way to improve NumPy’s dtype system is
ongoing, and several approaches are under consideration. No proposal
will be accepted without review and consensus by the broader NumPy
community. We welcome anyone who’s interested in these issues to join
us on the mailing list -- the more input we have, the better the
result will be :-).

Apologies for the wide distribution of this message; I suggest any
followups be directed to numpy-discussion@scipy.org only.

Thanks,
- Nathaniel

[1] http://thread.gmane.org/gmane.comp.python.numeric.general/61106
(we’ll move it into the repository soon, we promise!)

On Sun, Sep 13, 2015 at 3:51 PM, Travis Oliphant  wrote:
> Hey all,
>
> I just wanted to clarify, that I am very excited about a few ideas I have
> --- but I don't have time myself to engage in the community process to get
> these changes into NumPy. However, those are real processes --- I've
> been coaching a few people in those processes for the past several years
> already.
>
> So, rather than do nothing, what I'm looking to do is to work with a few
> people who I can share my ideas with, get excited about the ideas, and then
> who will work with the community to get them implemented.   That's what I
> was announcing and talking about yesterday --- looking for interested people
> who want to work on NumPy *with* the NumPy community.
>
> In my enthusiasm, I realize that some may have mis-understood my intention.
> There is no 'imminent' fork, nor am I planning on doing some crazy amount of
> work that I then try to force on other developers of NumPy.
>
> What I'm planning to do is find people to train on NumPy code base (people
> to increase the diversity of the developers would be ideal -- but hard to
> accomplish).  I plan to train them on NumPy based on my experience, and on
> what I think should be done --- and then have *them* work through the
> community process and engage with others to get consensus (hopefully not
> losing too much in translation in the process --- but instead getting even
> better).
>
> During that process I will engage as a member of the community and help
> write NEPs and other documents and help clarify where it makes sense as I
> can.   I will be filtering for people that actually want to see NumPy get
> better.Until I identify the people and work with them, it will be hard
> to tell how this will best work.   So, stay tuned.
>
> If all goes well, what you should see in a few weeks time are specific
> proposals, a branch or two, and the beginnings of some pull requests.If
> you don't see that, then I will not have found the right people to help me,
> and we will all continue to go back to searching.
>
> While I'm expecting the best, in the worst case, we get additional people
> who know the NumPy code base and can help squash bugs as well as implement
> changes that are desired.Three things are needed if you want to
> participate in this:  1) A willingness to work with the open source
> community, 2) a deep knowledge of C and in-particular CPython's brand of C,
> and 3) a willingness to engage with me, do a mind-meld and dump around the
> NumPy code base, and then improve on what is in my head with the rest of the
> community.
>
> Thanks,
>
> -Travis
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
Nathaniel J. Smith -- http://vorpus.org

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

[Sturla Molden]

On 14/09/15 10:26, Robert Kern wrote:


I want fast, multiple independent streams on my
current hardware first, and PCG gives that to me.


DCMT is good for that as well.

It should be possible to implement a pluggable design of NumPy's mtrand. 
Basically call a function pointer instead of rk_double. That way any 
(P)RNG can be plugged in. Then we could e.g. allow a Python callable, a 
Cython class or an f2py function as callback. We could ship multiple 
PRNGs with NumPy, and we could allow users to supply their own.


Sturla

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

[Sturla Molden]

On 14/09/15 10:34, Antoine Pitrou wrote:


If Numpy wanted to switch to a different generator, and if Numba wanted
to remain compatible with Numpy, one of the PCG functions would be an
excellent choice (also for CPU performance, incidentally).


Is Apache license ok in NumPy?

(Not sure, thus asking. The PCG suite is Apache licensed.)





___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Matthew Brett]

On Mon, Sep 14, 2015 at 3:47 AM, Julian Taylor
 wrote:
> as due to the many incompatiblities in 1.10 many will likely not be able
> to update anytime soon, so I think putting out another 1.9.3 bugfix
> release would be a good idea.
> I can probably do the release management for it, though I haven't been
> keeping up with bugfixes recently so, please comment on important issues
> you would want fixed.
> The np.where upcast regression and np.nanmedian issues come to my mind
> as should be fixed.

OK - fair enough - it would be good to do this soon as possible, as
Python 3.5 is currently the default download from the Python.org site
(rather than Python 2.7) and most other binary installers depend on
numpy.   Is there anything I can do to help?

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] First release candidate for the 1.5.0 series

[Matthew Brett]

Hi,

On Mon, Sep 14, 2015 at 8:59 PM, Thomas Caswell  wrote:
> Please give it a try! (linux64 conda builds are available on the tacaswell
> anaconda.org channel)
>
> https://github.com/matplotlib/matplotlib/releases/tag/v1.5.0rc1
>
> This release contains many new features.  The highlights include:
>
>   - the object oriented API will now automatically re-draw the figure when
> working in the command line
>   - automatic unpacking of labeled data into most plotting functions
>   - arbitrary style cycling
>   - four new perceptually linear color maps
>   - mouse-over for pixel values with `imshow`
>   - many new rcparams
>
> In addition there are significant improvements to `nbagg` and a complete
> overhaul of the c++ bindings to AGG.
>
> Please see the drafts of the [full whats
> new](http://matplotlib.org/devdocs/users/whats_new.html#new-in-matplotlib-1-5)
> and [api
> changes](http://matplotlib.org/devdocs/api/api_changes.html#changes-in-1-5-0)

After some struggle, RC1 wheels up at http://wheels.scipy.org/

Built via : https://travis-ci.org/MacPython/matplotlib-wheels/builds/80587383

If you're testing Python 3.5 you'll need the patched numpy 1.9.2 wheel
from nipy.bic.berkeley.edu :

pip install  -f https://nipy.bic.berkeley.edu/scipy_installers numpy

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ANN: Numexpr 2.4.4 is out

[Francesc Alted]

=
 Announcing Numexpr 2.4.4
=

Numexpr is a fast numerical expression evaluator for NumPy.  With it,
expressions that operate on arrays (like "3*a+4*b") are accelerated
and use less memory than doing the same calculation in Python.

It wears multi-threaded capabilities, as well as support for Intel's
MKL (Math Kernel Library), which allows an extremely fast evaluation
of transcendental functions (sin, cos, tan, exp, log...)  while
squeezing the last drop of performance out of your multi-core
processors.  Look here for a some benchmarks of numexpr using MKL:

https://github.com/pydata/numexpr/wiki/NumexprMKL

Its only dependency is NumPy (MKL is optional), so it works well as an
easy-to-deploy, easy-to-use, computational engine for projects that
don't want to adopt other solutions requiring more heavy dependencies.

What's new
==

This is a maintenance release which contains several bug fixes, like
better testing on Python3 platform and some harmless data race.  Among
the enhancements, AppVeyor support is here and OMP_NUM_THREADS is
honored as a fallback in case NUMEXPR_NUM_THREADS is not set.

In case you want to know more in detail what has changed in this
version, see:

https://github.com/pydata/numexpr/blob/master/RELEASE_NOTES.rst


Where I can find Numexpr?
=

The project is hosted at GitHub in:

https://github.com/pydata/numexpr

You can get the packages from PyPI as well (but not for RC releases):

http://pypi.python.org/pypi/numexpr

Share your experience
=

Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.

Enjoy data!

-- 
Francesc Alted
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Chris Barker]

Travis,

I'm sure you appreciate that this might all look a bit scary, given the
recent discussion about numpy governance.

But it's an open-source project, and I, at least, fully understand that
going through a big process is NOT the way to get a new idea tried out and
implemented. So I think think this is a great development -- I know I want
to see something like this dtype work done.

So, as someone who has been around this community for a long time, and
dependent on Numeric, numarray, and numpy over the years, this looks like a
great development.

And, in fact, with the new governance effort -- I think less scary --
people can go off and work on a branch or fork, do good stuff, and we, as a
community, can be assured that API (or even ABI) changes won't be thrust
upon us unawares :-)

As for the technical details -- I get a bit lost, not fully understanding
the current dtype system either, but do your ideas take us in the direction
of having dtypes independent of the container and ufunc machinery -- and
thus easier to create new dtypes (even in Python?) 'cause that would be
great.

I hope you find the partner you're looking for -- that's a challenge!

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ANN: python-blosc 1.2.8 released

[Francesc Alted]

=
Announcing python-blosc 1.2.8
=

What is new?


This is a maintenance release.  Internal C-Blosc has been upgraded to
1.7.0 (although new bitshuffle support has not been made public, as it
seems not ready for production yet).

Also, there is support for bytes-like objects that support the buffer
interface as input to ``compress`` and ``decompress``. On Python 2.x
this includes unicode, on Python 3.x it doesn't.  Thanks to Valentin
Haenel.

Finally, a memory leak in ``decompress```has been hunted and fixed.  And
new tests have been added to catch possible leaks in the future.  Thanks
to Santi Villalba.

For more info, you can have a look at the release notes in:

https://github.com/Blosc/python-blosc/blob/master/RELEASE_NOTES.rst

More docs and examples are available in the documentation site:

http://python-blosc.blosc.org


What is it?
===

Blosc (http://www.blosc.org) is a high performance compressor
optimized for binary data.  It has been designed to transmit data to
the processor cache faster than the traditional, non-compressed,
direct memory fetch approach via a memcpy() OS call.

Blosc is the first compressor that is meant not only to reduce the size
of large datasets on-disk or in-memory, but also to accelerate object
manipulations that are memory-bound
(http://www.blosc.org/docs/StarvingCPUs.pdf).  See
http://www.blosc.org/synthetic-benchmarks.html for some benchmarks on
how much speed it can achieve in some datasets.

Blosc works well for compressing numerical arrays that contains data
with relatively low entropy, like sparse data, time series, grids with
regular-spaced values, etc.

python-blosc (http://python-blosc.blosc.org/) is the Python wrapper for
the Blosc compression library.

There is also a handy tool built on Blosc called Bloscpack
(https://github.com/Blosc/bloscpack). It features a commmand line
interface that allows you to compress large binary datafiles on-disk.
It also comes with a Python API that has built-in support for
serializing and deserializing Numpy arrays both on-disk and in-memory at
speeds that are competitive with regular Pickle/cPickle machinery.


Installing
==

python-blosc is in PyPI repository, so installing it is easy:

$ pip install -U blosc  # yes, you must omit the 'python-' prefix


Download sources


The sources are managed through github services at:

http://github.com/Blosc/python-blosc


Documentation
=

There is Sphinx-based documentation site at:

http://python-blosc.blosc.org/


Mailing list


There is an official mailing list for Blosc at:

bl...@googlegroups.com
http://groups.google.es/group/blosc


Licenses


Both Blosc and its Python wrapper are distributed using the MIT license.
See:

https://github.com/Blosc/python-blosc/blob/master/LICENSES

for more details.



  **Enjoy data!**

-- 
Francesc Alted
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

[Nathaniel Smith]

On Mon, Sep 14, 2015 at 7:56 AM, Sturla Molden  wrote:
> On 14/09/15 10:34, Antoine Pitrou wrote:
>
>> Currently we don't provide those APIs on the GPU, since MT is much too
>> costly there.
>>
>> If Numpy wanted to switch to a different generator, and if Numba wanted
>> to remain compatible with Numpy, one of the PCG functions would be an
>> excellent choice (also for CPU performance, incidentally).
>
>
> We have discussed allowing plugable PRNGs in NumPy on multiple occations.
>
> I don't think NumPy should change its default PRNG. The Mersenne Twister
> MT19937 is state of the art of numerical work. It has excellent numerical
> accuracy, a huge period, and is very fast. For most users of NumPy, the
> MT19937 is the first and last word that needs to be said about pseudorandom
> number generators.

There are obviously trade-offs here and I haven't looked at the
details enough to have an opinion on whether the benefits are *enough*
that changing the default would be worth it, but it's simply not true
that MT19937 is state of the art. There are faster generators with
better numerical properties and more features (including, but not
only, the PCG family).

It looks like the random123 Philox generator is somewhat popular on
GPUs at this point too (e.g.
https://uk.mathworks.com/help/distcomp/examples/generating-random-numbers-on-a-gpu.html).

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Matthew Brett]

Hi Christoph,

On Mon, Sep 14, 2015 at 11:06 AM, Christoph Gohlke  wrote:
> On 9/14/2015 3:47 AM, Julian Taylor wrote:
>>
>> as due to the many incompatiblities in 1.10 many will likely not be able
>> to update anytime soon, so I think putting out another 1.9.3 bugfix
>> release would be a good idea.
>> I can probably do the release management for it, though I haven't been
>> keeping up with bugfixes recently so, please comment on important issues
>> you would want fixed.
>> The np.where upcast regression and np.nanmedian issues come to my mind
>> as should be fixed.
>>
>> On 09/14/2015 11:21 AM, Carl Kleffner wrote:
>>>
>>> I would like to add patches for the mingwpy windows build as well. There
>>> is no Python-3.5 build so far.
>>>
>>> Carlkl
>>>
>>> 2015-09-14 10:46 GMT+02:00 Robert Kern >> >:
>>>
>>>  On Mon, Sep 14, 2015 at 9:32 AM, Matthew Brett
>>>  > wrote:
>>>  >
>>>  > Hi,
>>>  >
>>>  > On Mon, Sep 14, 2015 at 1:22 AM, David Cournapeau
>>> > wrote:
>>>  > >
>>>  > >
>>>  > > On Mon, Sep 14, 2015 at 9:18 AM, Matthew Brett
>>> >
>>>  > > wrote:
>>>  > >>
>>>  > >> Hi,
>>>  > >>
>>>  > >> I'm just building numpy 1.9.2 for Python 3.5 (just released).
>>>  > >>
>>>  > >> In order to get the tests to pass on Python 3.5, I need to
>>> cherry pick
>>>  > >> commit 7d6aa8c onto the 1.9.2 tag position.
>>>  > >>
>>>  > >> Does anyone object to me uploading a wheel built from this
>>> patched
>>>  > >> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help
>>> to get
>>>  > >> the ball rolling for Python 3.5 binary wheels.
>>>  > >
>>>  > >
>>>  > > Why not releasing this as 1.9.3 ? It does not need to be a full
>>> release
>>>  > > (with binaries and all), but having multiple sources for a given
>>> tag is
>>>  > > confusing.
>>>  >
>>>  > Generally OK with me, but it's quite a bit of extra work for very
>>>  > little gain.  We'd have to tag, release a source tarball and OSX
>>>  > wheels, at least.
>>>
>>>  I think it's highly desirable that we also have a *source* release
>>>  that builds on Python 3.5, irrespective of whether or not we have
>>>  binary wheels for a couple of platforms up for Python 3.5. So I
>>>  would encourage a quick 1.9.3 release that incorporates this patch.
>>>
>>>  --
>>>  Robert Kern
>>>
>
> Support for Visual Studio 2015 and Intel Fortran 16 for Python 3.5 on
> Windows would also be nice.
>
> I am using:
>
> DEV: Replace deprecated options for ifort
>   
>
> remove /GL for vs2015 in check_long_double_representation
>   
>
> Enable Visual Studio 2015 C99 features
>   
>
> BLD: revert C99 complex for msvc14
>   

Would you mind making a branch for these, preferably starting from my
current branch:

git://github.com/matthew-brett/numpy.git branch prepare-1.9.3 ?

I had a quick go, but the merge conflicts needed more understanding
than I had of the Windows build.

Thanks a lot,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] The process I intend to follow for any proposed changes to NumPy

[Travis Oliphant]

Hey Chris (limiting to NumPy only),

I've had some great conversations with Nathaniel in the past few days and
I'm glad he posted his thoughts so that there is no confusion about
governance or what I was implying.

With respect to governance, I'm very supportive of what everyone is doing
in organizing a governance document and approach and appreciate the effort
of Nathaniel and others to move this forward.   Nothing I said was meant to
imply differently.   I'm sorry if it made anyone nervous.

I'm a very enthusiastic person when I get an idea of what to do.   I like
to see things implemented.   In this case, it also turns out that in terms
of overall architecture, my ideas are actually very similar to Nathaniel's
ideas.   That's a good sign.We have different tactical approaches as to
how to move forward, but I think it's a good thing to note that we see a
very similar path forward.Nothing will be done in NumPy itself except
via pull-request and review.

My approach for the ideas I'm pursuing will be to organize people around
two new prototype packages I'm calling memtype and gufunc.   The purpose of
these is to allow playing with the design and ideas quickly before looking
at how to put them into NumPy itself --- there will also be some training
involved in getting people up to speed.There was a long discussion
today at this BIDS data-structures for data-science summit part of which
talked about how to improve NumPy's dtype system. I would love to these
independent objects evolve into independent packages that could even go
into Python standard library. Not everyone agrees that is the best
idea, but regardless of whether this happens or not, the intent is to do
work that could go into NumPy now.

I look forward to the activity.

-Travis




On Mon, Sep 14, 2015 at 10:46 AM, Chris Barker 
wrote:

> Travis,
>
> I'm sure you appreciate that this might all look a bit scary, given the
> recent discussion about numpy governance.
>
> But it's an open-source project, and I, at least, fully understand that
> going through a big process is NOT the way to get a new idea tried out and
> implemented. So I think think this is a great development -- I know I want
> to see something like this dtype work done.
>
> So, as someone who has been around this community for a long time, and
> dependent on Numeric, numarray, and numpy over the years, this looks like a
> great development.
>
> And, in fact, with the new governance effort -- I think less scary --
> people can go off and work on a branch or fork, do good stuff, and we, as a
> community, can be assured that API (or even ABI) changes won't be thrust
> upon us unawares :-)
>
> As for the technical details -- I get a bit lost, not fully understanding
> the current dtype system either, but do your ideas take us in the direction
> of having dtypes independent of the container and ufunc machinery -- and
> thus easier to create new dtypes (even in Python?) 'cause that would be
> great.
>
> I hope you find the partner you're looking for -- that's a challenge!
>
> -Chris
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

*Travis Oliphant*
*Co-founder and CEO*


@teoliphant
512-222-5440
http://www.continuum.io
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] method to calculate the magnitude squared

[Phillip Feldman]

In communications and signal processing, it is frequently necessary to
calculate the power of a signal.  This can be done with a function like the
following:

def magsq(z):
   """
   Return the magnitude squared of the real- or complex-valued input.
   """
   return z.real**2 + z.imag**2

A high percentage of the scripts that I write contain or import a function
like this.  It would be great if there were a built-in method in NumPy,
preferably with a name like `magsq`, `mag2`, or `msq`.

Phillip
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] method to calculate the magnitude squared

[R Schumacher]

At 09:16 PM 9/18/2015, you wrote:
In communications and signal processing, it is 
frequently necessary to calculate the power of a 
signal.  This can be done with a function like the following:


def magsq(z):
   """
   Return the magnitude squared of the real- or complex-valued input.
   """
   return z.real**2 + z.imag**2


Is that not the same as
np.abs(z)**2 ?

- Ray


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion