[Numpy-discussion] Proposal for a new function: np.moveaxis

[Stephan Hoyer]

I've put up a pull request implementing a new function, np.moveaxis, as an
alternative to np.transpose and np.rollaxis:
https://github.com/numpy/numpy/pull/6630

This functionality has been discussed (even the exact function name)
several times over the years, but it never made it into a pull request. The
most pressing issue is that the behavior of np.rollaxis is not intuitive to
most users:
https://mail.scipy.org/pipermail/numpy-discussion/2010-September/052882.html
https://github.com/numpy/numpy/issues/2039
http://stackoverflow.com/questions/29891583/reason-why-numpy-rollaxis-is-so-confusing

In this pull request, I also allow the source and destination axes to be
sequences as well as scalars. This does not add much complexity to the
code, solves some additional use cases and makes np.moveaxis a proper
generalization of the other axes manipulation routines (see the pull
requests for details).

Best of all, it already works on ndarray duck types (like masked array and
dask.array), because they have already implemented transpose.

I think np.moveaxis would be a useful addition to NumPy -- I've found
myself writing helper functions with a subset of its functionality several
times over the past few years. What do you think?

Cheers,
Stephan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] querying backend information

[Ralf Gommers]

On Thu, Nov 5, 2015 at 5:11 AM, Nathaniel Smith  wrote:

> On Wed, Nov 4, 2015 at 4:40 PM, Stefan Seefeld 
> wrote:
> > Hello,
> >
> > is there a way to query Numpy for information about backends (BLAS,
> > LAPACK, etc.) that it was compiled against, including compiler / linker
> > flags that were used ?
> > Consider the use-case where instead of calling a function such as
> > numpy.dot() I may want to call the appropriate backend directly using
> > the C API as an optimization technique. Is there a straight-forward way
> > to do that ?
> >
> > In a somewhat related line of thought: Is there a way to see what
> > backends are available during Numpy compile-time ? I'm looking for a
> > list of flags to pick ATLAS/OpenBLAS/LAPACK/MKL or any other backend
> > that might be available, combined with variables (compiler and linker
> > flags, notably) I might have to set. Is that information available at
> all ?
>
> NumPy does reveal some information about its configuration and
> numpy.distutils does provide helper methods, but I'm not super
> familiar with it so I'll let others answer that part.
>

np.show_config()

Gives:

lapack_opt_info:
libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/atlas-base/atlas', '/usr/lib/atlas-base']
define_macros = [('NO_ATLAS_INFO', -1)]
language = f77
include_dirs = ['/usr/include/atlas']
openblas_lapack_info:
  NOT AVAILABLE



It's a function with no docstring and not in the html docs (I think), so
that can certainly be improved.

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] querying backend information

[Nathaniel Smith]

On Wed, Nov 4, 2015 at 4:40 PM, Stefan Seefeld  wrote:
> Hello,
>
> is there a way to query Numpy for information about backends (BLAS,
> LAPACK, etc.) that it was compiled against, including compiler / linker
> flags that were used ?
> Consider the use-case where instead of calling a function such as
> numpy.dot() I may want to call the appropriate backend directly using
> the C API as an optimization technique. Is there a straight-forward way
> to do that ?
>
> In a somewhat related line of thought: Is there a way to see what
> backends are available during Numpy compile-time ? I'm looking for a
> list of flags to pick ATLAS/OpenBLAS/LAPACK/MKL or any other backend
> that might be available, combined with variables (compiler and linker
> flags, notably) I might have to set. Is that information available at all ?

NumPy does reveal some information about its configuration and
numpy.distutils does provide helper methods, but I'm not super
familiar with it so I'll let others answer that part.

Regarding the idea of "cutting out the middleman" and calling directly
into the appropriate backend via the C API, NumPy doesn't currently
expose any interface for doing this. There are some discussions with
Antoine from a few months back about this (and given that you work at
the same place I'm guessing the motivation is the same? :-)). For some
reason I'm failing to find the archives now, but the summary from off
the top of my head is: SciPy does expose an interface for this (via
cython and its PyCapsule tricks -- see [1]), NumPy is unlikely to
because we're wary of adding extra public interfaces and can't
guarantee that we even have a full BLAS/LAPACK available (sometimes we
fall back on a minimal vendored subset that's just enough for our
needs), you probably don't want to try and get into the business of
dynamically hunting down BLAS/LAPACK because it will be brittle and
expose you to all kinds of cross-platform linker issues, and if you
want to pull the clever stuff that scipy is doing out of scipy and put
it into its own dedicated blas/lapack package, then well, we need one
of those anyway [2].

-n

[1] 
https://github.com/scipy-conference/scipy_proceedings_2015/blob/master/papers/ian_henriksen/cython_blas_lapack_api.rst
[2] e.g. 
https://mail.scipy.org/pipermail/numpy-discussion/2015-January/072123.html

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] querying backend information

[Stefan Seefeld]

Hello,

is there a way to query Numpy for information about backends (BLAS,
LAPACK, etc.) that it was compiled against, including compiler / linker
flags that were used ?
Consider the use-case where instead of calling a function such as
numpy.dot() I may want to call the appropriate backend directly using
the C API as an optimization technique. Is there a straight-forward way
to do that ?

In a somewhat related line of thought: Is there a way to see what
backends are available during Numpy compile-time ? I'm looking for a
list of flags to pick ATLAS/OpenBLAS/LAPACK/MKL or any other backend
that might be available, combined with variables (compiler and linker
flags, notably) I might have to set. Is that information available at all ?

Thanks,
Stefan

-- 

  ...ich hab' noch einen Koffer in Berlin...

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] deprecate fromstring() for text reading?

[Derek Homeier]

On 3 Nov 2015, at 6:03 pm, Chris Barker - NOAA Federal  
wrote:
> 
> I was more aiming to point out a situation where the NumPy's text file reader 
> was significantly better than the Pandas version, so we would want to make 
> sure that we properly benchmark any significant changes to NumPy's text 
> reading code. Who knows where else NumPy beats Pandas?
> Indeed. For this example, I think a fixed-with reader really is a different 
> animal, and it's probably a good idea to have a high performance one in 
> Numpy. Among other things, you wouldn't want it to try to auto-determine data 
> types or anything like that.
> 
> I think what's on the table now is to bring in a new delimited reader -- I.e. 
> CSV in its various flavors.
> 
To add my own handful of change or at least another data point, I had been 
looking into both
the pandas and the Astropy fast readers as a fast loadtxt/genfromtxt 
replacement; at the time
I found the Astropy cparser source somewhat easier to dig into, although 
looking now Pandas'
parser.pyx seems clear enough as well.
Some comparison of the two can be found at
http://astropy.readthedocs.org/en/stable/io/ascii/fast_ascii_io.html#speed-gains

Unfortunately the Astropy fast reader currently does not support fixed-width 
format either, and
adding this functionality would require modifications to the tokenizer C code - 
not sure how
extensive.

Cheers,
Derek

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New behavior of allclose

[Nathan Goldbaum]

Yup, https://github.com/numpy/numpy/issues/6196

On Wed, Nov 4, 2015 at 1:45 PM, Charles R Harris 
wrote:

>
>
> On Wed, Nov 4, 2015 at 12:42 PM, Nathan Goldbaum 
> wrote:
>
>> Oh oops, this is about np.allcose, not np.assert_allclose. Sorry for the
>> noise...
>>
>
> Probably related ;) Did you open an issue for it?
>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New behavior of allclose

[Charles R Harris]

On Wed, Nov 4, 2015 at 12:42 PM, Nathan Goldbaum 
wrote:

> Oh oops, this is about np.allcose, not np.assert_allclose. Sorry for the
> noise...
>

Probably related ;) Did you open an issue for it?

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New behavior of allclose

[Charles R Harris]

On Wed, Nov 4, 2015 at 12:40 PM, Benjamin Root  wrote:

> I am not sure I understand what you mean. Specifically that np.isclose
> will return a memmap if one of the inputs is a memmap. The result is a
> brand new array, right? So, what is that result memmapping from? Also, how
> does this impact np.allclose()? That function returns a scalar True/False,
> so what is the change in behavior there?
>
> By the way, the docs for isclose in 1.10.1 does not mention any behavior
> changes.
>

Yep, it is a new issue, see #6475


Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New behavior of allclose

[Nathan Goldbaum]

Oh oops, this is about np.allcose, not np.assert_allclose. Sorry for the
noise...

On Wed, Nov 4, 2015 at 1:36 PM, Nathan Goldbaum 
wrote:

> I actually brought this up before 1.10 came out:
> https://github.com/numpy/numpy/issues/6196
>
> The behavior change brought out a bug in our use of allclose, so while it
> was annoying in the sense that our test suite started failing in a new way,
> it was good in that our tests are now more correct.
>
> On Wed, Nov 4, 2015 at 1:28 PM, Charles R Harris <
> charlesr.har...@gmail.com> wrote:
>
>> Hi All,
>>
>> This is to open a discussion of a change of behavior of `np.allclose`.
>> That function uses `isclose` in numpy 1.10 with the result that array
>> subtypes are preserved whereas before they were not. In particular, memmaps
>> are returned when at least one of the inputs is a memmap. By and large I
>> think this is a good thing, OTOH, it is a change in behavior. It is easy to
>> fix, just run `np.array(result, copy=False)` on the current `result`, but I
>> thought I'd raise the topic on the list in case there is a good argument to
>> change things.
>>
>> Chuck
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New behavior of allclose

[Benjamin Root]

I am not sure I understand what you mean. Specifically that np.isclose will
return a memmap if one of the inputs is a memmap. The result is a brand new
array, right? So, what is that result memmapping from? Also, how does this
impact np.allclose()? That function returns a scalar True/False, so what is
the change in behavior there?

By the way, the docs for isclose in 1.10.1 does not mention any behavior
changes.

Ben Root

On Wed, Nov 4, 2015 at 2:28 PM, Charles R Harris 
wrote:

> Hi All,
>
> This is to open a discussion of a change of behavior of `np.allclose`.
> That function uses `isclose` in numpy 1.10 with the result that array
> subtypes are preserved whereas before they were not. In particular, memmaps
> are returned when at least one of the inputs is a memmap. By and large I
> think this is a good thing, OTOH, it is a change in behavior. It is easy to
> fix, just run `np.array(result, copy=False)` on the current `result`, but I
> thought I'd raise the topic on the list in case there is a good argument to
> change things.
>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] New behavior of allclose

[Nathan Goldbaum]

I actually brought this up before 1.10 came out:
https://github.com/numpy/numpy/issues/6196

The behavior change brought out a bug in our use of allclose, so while it
was annoying in the sense that our test suite started failing in a new way,
it was good in that our tests are now more correct.

On Wed, Nov 4, 2015 at 1:28 PM, Charles R Harris 
wrote:

> Hi All,
>
> This is to open a discussion of a change of behavior of `np.allclose`.
> That function uses `isclose` in numpy 1.10 with the result that array
> subtypes are preserved whereas before they were not. In particular, memmaps
> are returned when at least one of the inputs is a memmap. By and large I
> think this is a good thing, OTOH, it is a change in behavior. It is easy to
> fix, just run `np.array(result, copy=False)` on the current `result`, but I
> thought I'd raise the topic on the list in case there is a good argument to
> change things.
>
> Chuck
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] New behavior of allclose

[Charles R Harris]

Hi All,

This is to open a discussion of a change of behavior of `np.allclose`. That
function uses `isclose` in numpy 1.10 with the result that array subtypes
are preserved whereas before they were not. In particular, memmaps are
returned when at least one of the inputs is a memmap. By and large I think
this is a good thing, OTOH, it is a change in behavior. It is easy to fix,
just run `np.array(result, copy=False)` on the current `result`, but I
thought I'd raise the topic on the list in case there is a good argument to
change things.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion