Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Julian Taylor]

as due to the many incompatiblities in 1.10 many will likely not be able 
to update anytime soon, so I think putting out another 1.9.3 bugfix 
release would be a good idea.
I can probably do the release management for it, though I haven't been 
keeping up with bugfixes recently so, please comment on important issues 
you would want fixed.
The np.where upcast regression and np.nanmedian issues come to my mind 
as should be fixed.

On 09/14/2015 11:21 AM, Carl Kleffner wrote:
> I would like to add patches for the mingwpy windows build as well. There
> is no Python-3.5 build so far.
>
> Carlkl
>
> 2015-09-14 10:46 GMT+02:00 Robert Kern  >:
>
> On Mon, Sep 14, 2015 at 9:32 AM, Matthew Brett
> > wrote:
> >
> > Hi,
> >
> > On Mon, Sep 14, 2015 at 1:22 AM, David Cournapeau  > wrote:
> > >
> > >
> > > On Mon, Sep 14, 2015 at 9:18 AM, Matthew Brett 
> >
> > > wrote:
> > >>
> > >> Hi,
> > >>
> > >> I'm just building numpy 1.9.2 for Python 3.5 (just released).
> > >>
> > >> In order to get the tests to pass on Python 3.5, I need to cherry 
> pick
> > >> commit 7d6aa8c onto the 1.9.2 tag position.
> > >>
> > >> Does anyone object to me uploading a wheel built from this patched
> > >> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to 
> get
> > >> the ball rolling for Python 3.5 binary wheels.
> > >
> > >
> > > Why not releasing this as 1.9.3 ? It does not need to be a full 
> release
> > > (with binaries and all), but having multiple sources for a given tag 
> is
> > > confusing.
> >
> > Generally OK with me, but it's quite a bit of extra work for very
> > little gain.  We'd have to tag, release a source tarball and OSX
> > wheels, at least.
>
> I think it's highly desirable that we also have a *source* release
> that builds on Python 3.5, irrespective of whether or not we have
> binary wheels for a couple of platforms up for Python 3.5. So I
> would encourage a quick 1.9.3 release that incorporates this patch.
>
> --
> Robert Kern
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org 
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Matthew Brett]

Hi,

I'm just building numpy 1.9.2 for Python 3.5 (just released).

In order to get the tests to pass on Python 3.5, I need to cherry pick
commit 7d6aa8c onto the 1.9.2 tag position.

Does anyone object to me uploading a wheel built from this patched
version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to get
the ball rolling for Python 3.5 binary wheels.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

[Robert Kern]

[Tim, ping me if you want to get dropped from the reply chain, as we are
liable to get more into numpy decision-making. I've dropped python-ideas.]

On Mon, Sep 14, 2015 at 4:34 AM, Tim Peters  wrote:
>
> [Robert Kern ]
> >> ...
> >> I'll also recommend the PCG paper (and algorithm) as the author's
> >> cross-PRNGs comparisons have been bandied about in this thread
already. The
> >> paper lays out a lot of the relevant issues and balances the various
> >> qualities that are important: statistical quality, speed, and security
(of
> >> various flavors).
> >>
> >>   http://www.pcg-random.org/paper.html
> >>
> >> I'm actually not that impressed with Random123. The core idea is nice
and
> >> clean, but the implementation is hideously complex.
>
> [Nathaniel Smith ]
> > I'm curious what you mean by this? In terms of the code, or...?

In terms of the code that would be necessary to implement this in a
simultaneously performant and cross-platform way in numpy.random.

> > I haven't looked at the code,

I recommend it! Approach it with the practical goal of "how do I stick this
in as a new core PRNG for numpy.random?"

> > but the simplest generator they
> > recommend in the paper is literally just
> >
> > def rng_stream(seed):
> > counter = 0
> > while True:
> > # AES128 takes a 128 bit key and 128 bits of data and returns
> > 128 bits of encrypted data
> > val = AES128(key=seed, data=counter)
> > yield low_64_bits(val)
> > yield high_64_bits(val)
> > counter += 1
>
> I assume it's because if you expand what's required to _implement_
> AES128() in C, it does indeed look pretty hideously complex.  On HW
> implementing AES primitives, of course the code can be much simpler.
>
> But to be fair, if integer multiplication and/or addition weren't
> implemented in HW, and we had to write to C code emulating them via
> bit-level fiddling, the code for any of the PCG algorithms would look
> hideously complex too.
>
> But being fair isn't much fun ;-)

Actually, I meant all of the crap *around* it, the platform-compatibility
testing to see if you have such a hardware instruction or not, and C++
template shenanigans in the surrounding code. It's possible that the
complexity is only due to flexibility, but it was too complex for me to
begin understanding *why* it's so complex before I succumbed to ennui and
moved on to some other productive use of my free time. At least some of the
complexity is due to needing software implementations of reduced-round
crypto for decent performance in the absence of the hardware instruction.
Performing well in the absence of the hardware instruction is very
important to me as I do not seem to have the AES-NI instruction available
on my mid-2012 Macbook Pro. Exposing counter-mode AES128 as a core PRNG is
a nice idea, but it's just low on my wishlist. I want fast, multiple
independent streams on my current hardware first, and PCG gives that to me.

On the "if I had to implement all of the bit-fiddling myself" count, I'd
still prefer PCG. It has a similar problem with 128-bit integers that may
or may not be natively provided (should you want a 128-bit state), and I
have to say that the software implementation of that bit-fiddling is much
nicer to work with than reduced-round Threefry. It's reference
implementation is also overly complex for practical use as it is showing
off the whole parameterized PCG family to support the paper, even the
explicitly unrecommended members. But it's relatively straightforward to
disentangle the desirable members; their implementations, including the
supporting 128-bit arithmetic code, are small and clean.

--
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Antoine Pitrou]

On Mon, 14 Sep 2015 01:32:13 -0700
Matthew Brett  wrote:
> >>
> >> Does anyone object to me uploading a wheel built from this patched
> >> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to get
> >> the ball rolling for Python 3.5 binary wheels.
> >
> > Why not releasing this as 1.9.3 ? It does not need to be a full release
> > (with binaries and all), but having multiple sources for a given tag is
> > confusing.
> 
> Generally OK with me, but it's quite a bit of extra work for very
> little gain.  We'd have to tag, release a source tarball and OSX
> wheels, at least.

It's always bad to have two silent versions of a single release. People
can never know up front which version they got, since it's not written
anywhere. That's why the right policy is to bump the version number in
some way (be it by incrementing it or adding a ".patchXXX" at the end).

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Carl Kleffner]

I would like to add patches for the mingwpy windows build as well. There is
no Python-3.5 build so far.

Carlkl

2015-09-14 10:46 GMT+02:00 Robert Kern :

> On Mon, Sep 14, 2015 at 9:32 AM, Matthew Brett 
> wrote:
> >
> > Hi,
> >
> > On Mon, Sep 14, 2015 at 1:22 AM, David Cournapeau 
> wrote:
> > >
> > >
> > > On Mon, Sep 14, 2015 at 9:18 AM, Matthew Brett <
> matthew.br...@gmail.com>
> > > wrote:
> > >>
> > >> Hi,
> > >>
> > >> I'm just building numpy 1.9.2 for Python 3.5 (just released).
> > >>
> > >> In order to get the tests to pass on Python 3.5, I need to cherry pick
> > >> commit 7d6aa8c onto the 1.9.2 tag position.
> > >>
> > >> Does anyone object to me uploading a wheel built from this patched
> > >> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to get
> > >> the ball rolling for Python 3.5 binary wheels.
> > >
> > >
> > > Why not releasing this as 1.9.3 ? It does not need to be a full release
> > > (with binaries and all), but having multiple sources for a given tag is
> > > confusing.
> >
> > Generally OK with me, but it's quite a bit of extra work for very
> > little gain.  We'd have to tag, release a source tarball and OSX
> > wheels, at least.
>
> I think it's highly desirable that we also have a *source* release that
> builds on Python 3.5, irrespective of whether or not we have binary wheels
> for a couple of platforms up for Python 3.5. So I would encourage a quick
> 1.9.3 release that incorporates this patch.
>
> --
> Robert Kern
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[David Cournapeau]

On Mon, Sep 14, 2015 at 9:18 AM, Matthew Brett 
wrote:

> Hi,
>
> I'm just building numpy 1.9.2 for Python 3.5 (just released).
>
> In order to get the tests to pass on Python 3.5, I need to cherry pick
> commit 7d6aa8c onto the 1.9.2 tag position.
>
> Does anyone object to me uploading a wheel built from this patched
> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to get
> the ball rolling for Python 3.5 binary wheels.
>

Why not releasing this as 1.9.3 ? It does not need to be a full release
(with binaries and all), but having multiple sources for a given tag is
confusing.

David


>
> Cheers,
>
> Matthew
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Matthew Brett]

Hi,

On Mon, Sep 14, 2015 at 1:22 AM, David Cournapeau  wrote:
>
>
> On Mon, Sep 14, 2015 at 9:18 AM, Matthew Brett 
> wrote:
>>
>> Hi,
>>
>> I'm just building numpy 1.9.2 for Python 3.5 (just released).
>>
>> In order to get the tests to pass on Python 3.5, I need to cherry pick
>> commit 7d6aa8c onto the 1.9.2 tag position.
>>
>> Does anyone object to me uploading a wheel built from this patched
>> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to get
>> the ball rolling for Python 3.5 binary wheels.
>
>
> Why not releasing this as 1.9.3 ? It does not need to be a full release
> (with binaries and all), but having multiple sources for a given tag is
> confusing.

Generally OK with me, but it's quite a bit of extra work for very
little gain.  We'd have to tag, release a source tarball and OSX
wheels, at least.

The patch being cherry-picked is just deleting some legacy
monkey-patching of gzip:

https://github.com/numpy/numpy/commit/7d6aa8c721d5274ac57d0c87685d472cb1fd7948

and it's difficult for me to imagine this could cause a problem with
source differences.

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-ideas] Should our default random number generator be secure?

[Antoine Pitrou]

On Mon, 14 Sep 2015 09:26:58 +0100
Robert Kern  wrote:
> 
> Actually, I meant all of the crap *around* it, the platform-compatibility
> testing to see if you have such a hardware instruction or not, and C++
> template shenanigans in the surrounding code. It's possible that the
> complexity is only due to flexibility, but it was too complex for me to
> begin understanding *why* it's so complex before I succumbed to ennui and
> moved on to some other productive use of my free time. At least some of the
> complexity is due to needing software implementations of reduced-round
> crypto for decent performance in the absence of the hardware instruction.
> Performing well in the absence of the hardware instruction is very
> important to me as I do not seem to have the AES-NI instruction available
> on my mid-2012 Macbook Pro. Exposing counter-mode AES128 as a core PRNG is
> a nice idea, but it's just low on my wishlist. I want fast, multiple
> independent streams on my current hardware first, and PCG gives that to me.

Using AES also means emulating it on a GPU will be quite hard.

For the record, Numba is currently using a Mersenne Twister on the CPU,
to emulate Numpy's behaviour (although some of our distributions may be
different):

>>> def f(x):
... np.random.seed(x)
... l = []
... for i in range(10): l.append(np.random.random())
... return l
... 
>>> g = numba.jit(nopython=True)(f)
>>> f(10)
[0.771320643266746, 0.0207519493594015, 0.6336482349262754,
0.7488038825386119, 0.4985070123025904, 0.22479664553084766,
0.19806286475962398, 0.7605307121989587, 0.16911083656253545,
0.08833981417401027]
>>> g(10)
[0.771320643266746, 0.0207519493594015, 0.6336482349262754,
0.7488038825386119, 0.4985070123025904, 0.22479664553084766,
0.19806286475962398, 0.7605307121989587, 0.16911083656253545,
0.08833981417401027]


Currently we don't provide those APIs on the GPU, since MT is much too
costly there.

If Numpy wanted to switch to a different generator, and if Numba wanted
to remain compatible with Numpy, one of the PCG functions would be an
excellent choice (also for CPU performance, incidentally).

Regards

Antoine.


___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] OK to upload patched 1.9.2 for Python 3.5?

[Robert Kern]

On Mon, Sep 14, 2015 at 9:32 AM, Matthew Brett 
wrote:
>
> Hi,
>
> On Mon, Sep 14, 2015 at 1:22 AM, David Cournapeau 
wrote:
> >
> >
> > On Mon, Sep 14, 2015 at 9:18 AM, Matthew Brett 
> > wrote:
> >>
> >> Hi,
> >>
> >> I'm just building numpy 1.9.2 for Python 3.5 (just released).
> >>
> >> In order to get the tests to pass on Python 3.5, I need to cherry pick
> >> commit 7d6aa8c onto the 1.9.2 tag position.
> >>
> >> Does anyone object to me uploading a wheel built from this patched
> >> version to pypi as 1.9.2 for Python 3.5 on OSX?   It would help to get
> >> the ball rolling for Python 3.5 binary wheels.
> >
> >
> > Why not releasing this as 1.9.3 ? It does not need to be a full release
> > (with binaries and all), but having multiple sources for a given tag is
> > confusing.
>
> Generally OK with me, but it's quite a bit of extra work for very
> little gain.  We'd have to tag, release a source tarball and OSX
> wheels, at least.

I think it's highly desirable that we also have a *source* release that
builds on Python 3.5, irrespective of whether or not we have binary wheels
for a couple of platforms up for Python 3.5. So I would encourage a quick
1.9.3 release that incorporates this patch.

--
Robert Kern
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion