Re: [Numpy-discussion] ANN: NumExpr3 Alpha

[Francesc Alted]

Yes, Julian is doing an amazing work on getting rid of temporaries inside
NumPy.  However, NumExpr still has the advantage of using multi-threading
right out of the box, as well as integration with Intel VML.  Hopefully
these features will eventually arrive to NumPy, but meanwhile there is
still value in pushing NumExpr.

Francesc

2017-02-19 18:21 GMT+01:00 Marten van Kerkwijk :

> Hi All,
>
> Just a side note that at a smaller scale some of the benefits of
> numexpr are coming to numpy: Julian Taylor has been working on
> identifying temporary arrays in
> https://github.com/numpy/numpy/pull/7997. Julian also commented
> (https://github.com/numpy/numpy/pull/7997#issuecomment-246118772) that
> with PEP 523 in python 3.6, this should indeed become a lot easier.
>
> All the best,
>
> Marten
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 
Francesc Alted
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Could we simplify backporting?

[Marten van Kerkwijk]

Hi All,

In gh-8594, a question came up how to mark things that should be
backported and Chuck commented [1]:

> Our backport policy is still somewhat ad hoc, especially as I the only one 
> who has been doing release. What I currently do is set the milestone to the 
> earlier version, so I will find the PR when looking for backports, then do a 
> backport, label it as such, set the milestone on the backported version, and 
> remove the milestone from the original. I'm not completely happy with the 
> process, so if you have better ideas I'd like to hear them. One option I've 
> considered is a `backported` label in addition to the `backport` label, then 
> use the latter for things to be backported.

It seems that continuing to set the milestone to a bug-release version
if required was a good idea; it is little effort to anyone and keeps
things clear. For the rest, might it be possible to make things more
automated? E.g., might it be possible to have some travis magic that
does a trial merge & test? Could one somehow merge to multiple
branches at the same time?

I have no idea myself really how these things work, but maybe some of you do!

All the best,

Marten
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] PowerPC testing servers

[Matthew Brett]

Hi,

On Thu, Feb 16, 2017 at 12:50 AM, Ralf Gommers  wrote:
>
>
> On Thu, Feb 16, 2017 at 8:58 AM, Matthew Brett 
> wrote:
>>
>> On Wed, Feb 15, 2017 at 7:55 PM, Ralf Gommers 
>> wrote:
>> >
>> >
>> > On Thu, Feb 16, 2017 at 8:45 AM, Matthew Brett 
>> > wrote:
>> >>
>> >> On Wed, Feb 15, 2017 at 7:37 PM, Ralf Gommers 
>> >> wrote:
>> >> >
>> >> >
>> >> > On Thu, Feb 16, 2017 at 8:02 AM, Matthew Brett
>> >> > 
>> >> > wrote:
>> >> >>
>> >> >> Hey,
>> >> >>
>> >> >> A recent post to the wheel-builders mailing list pointed out some
>> >> >> links to places providing free PowerPC hosting for open source
>> >> >> projects, if they agree to a submitted request:
>> >> >>
>> >> >>
>> >> >>
>> >> >> https://mail.python.org/pipermail/wheel-builders/2017-February/000257.html
>> >> >>
>> >> >> It would be good to get some testing going on these architectures.
>> >> >> Shall we apply for hosting, as the numpy organization?
>> >> >
>> >> >
>> >> > Those are bare VMs it seems. Remembering the Buildbot and Mailman
>> >> > horrors, I
>> >> > think we should be very reluctant to taking responsibility for
>> >> > maintaining
>> >> > CI on anything that's not hosted and can be controlled with a simple
>> >> > config
>> >> > file in our repo.
>> >>
>> >> Not sure what you mean about mailman - maybe the Enthought servers we
>> >> didn't have access to?
>> >
>> >
>> > We did have access (for most of the time), it's just that no one is
>> > interested in putting in lots of hours on sysadmin duties.
>> >
>> >>
>> >> For buildbot, I've been maintaining about 12
>> >> crappy old machines for about 7 years now [1] - I'm happy to do the
>> >> same job for a couple of properly hosted PPC machines.
>> >
>> >
>> > That's awesome persistence. The NumPy and SciPy buildbots certainly
>> > weren't
>> > maintained like that, half of them were offline or broken for long
>> > periods
>> > usually.
>>
>> Right - they do need persistence, and to have someone who takes
>> responsibility for them.
>>
>> >>
>> >>  At least we'd
>> >> have some way of testing for these machines, if we get stuck - even if
>> >> that involved spinning up a VM and installing the stuff we needed from
>> >> the command line.
>> >
>> >
>> > I do see the value of testing on more platforms of course. It's just
>> > about
>> > logistics/responsibilities. If you're saying that you'll do the
>> > maintenance,
>> > and want to apply for resources using the NumPy name, that's much better
>> > I
>> > think then making "the numpy devs" collectively responsible.
>>
>> Yes, exactly.  I'm happy to take responsibility for them, I just
>> wanted to make sure that numpy devs could get at them if I'm not
>> around for some reason.
>
>
> In that case, +1 from me!

OK - IBM have kindly given me access to a testing machine, via my own
SSH public key.   Would it make sense to have a Numpy key, with
several people having access to the private key and passphrase?

Cheers,

Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Fortran order in recarray.

[Alex Rogozhnikov]

Hi, 

a question about numpy.recarray:
There is a parameter order in constructor 
https://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.recarray.html 
,
 but it seems to have no effect:

import numpy
x = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[1000], order='C')
y = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[1000], order='F')
print numpy.array(x.ctypes.get_strides()) # [16]
print numpy.array(y.ctypes.get_strides()) # [16]

is this an intended behavior or bug?

Thanks,
Alex.___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Fortran order in recarray.

[Chris Barker]

On Tue, Feb 21, 2017 at 3:05 PM, Alex Rogozhnikov <
alex.rogozhni...@yandex.ru> wrote:

> a question about numpy.recarray:
> There is a parameter order in constructor https://docs.scipy.org/doc/
> numpy-1.10.1/reference/generated/numpy.recarray.html, but it seems to
> have no effect:
> x = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[1000],
> order='C')
>

you are creating a 1D array here -- there is no difference between Fortran
and C order for a 1D array. For 2D:

In [2]: x = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[10,10],
order='C')


In [3]: x.strides
Out[3]: (160, 16)


In [4]: y = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[10,10],
order='F')


In [5]: y.strides
Out[5]: (16, 160)

note the easier way to get the strides, too :-)

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Fortran order in recarray.

[Alex Rogozhnikov]

Ah, got it. Thanks, Chris!
I thought recarray can be only one-dimensional (like tables with named columns).

Maybe it's better to ask directly what I was looking for: 
something that works like a table with named columns (but no labelling for 
rows), and keeps data (of different dtypes) in a column-by-column way (and this 
is numpy, not pandas). 

Is there such a magic thing?

Alex.


> 22 февр. 2017 г., в 2:10, Chris Barker  написал(а):
> 
> 
> 
> On Tue, Feb 21, 2017 at 3:05 PM, Alex Rogozhnikov  > wrote:
> a question about numpy.recarray:
> There is a parameter order in constructor 
> https://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.recarray.html
>  
> ,
>  but it seems to have no effect:
> x = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[1000], order='C')
> 
> you are creating a 1D array here -- there is no difference between Fortran 
> and C order for a 1D array. For 2D:
> 
> In [2]: x = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[10,10], 
> order='C')
> 
> 
> In [3]: x.strides
> Out[3]: (160, 16)
> 
> 
> In [4]: y = numpy.recarray(dtype=[('a', int), ('b', float)], shape=[10,10], 
> order='F')
> 
> 
> In [5]: y.strides
> Out[5]: (16, 160)
> 
> note the easier way to get the strides, too :-)
> 
> -CHB
> 
> 
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
> 
> chris.bar...@noaa.gov 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Fortran order in recarray.

[Nathaniel Smith]

On Feb 21, 2017 3:24 PM, "Alex Rogozhnikov" 
wrote:

Ah, got it. Thanks, Chris!
I thought recarray can be only one-dimensional (like tables with named
columns).

Maybe it's better to ask directly what I was looking for:
something that works like a table with named columns (but no labelling for
rows), and keeps data (of different dtypes) in a column-by-column way (and
this is numpy, not pandas).

Is there such a magic thing?


Well, that's what pandas is for...

A dict of arrays?

-n
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion