[Numpy-discussion] Argonne is hiring a postdoc in radar forward modelling using Python

[Scott Collis]

Dear Numpy Users,
Argonne National Lab is hiring a postdoc working with the team behind 
Py-ART.


Please take a look and use this link to apply and direct any questions 
towards me.


http://careers.peopleclick.com/careerscp/client_argonnelab/post_doc/en_US/gateway.do?functionName=viewFromLink=en-us=3702=Facebook=NETWORKING_SITE

Long shot I know, but we found our key developer using this list last 
time :)


Cheers,
Scott
--
--
Dr Scott Collis
ARM Precipitation Radar Translator
Environmental Science Division
Argonne National Laboratory
Mb: +1 630 235 8025
Of: +1 630 252 0550
Become a Py-ART user today! http://arm-doe.github.io/pyart/ 

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] isfortran compatibility in numpy 1.10.

[Sebastian Berg]

I bet it has all been said already, but to note just in case. In numpy itself 
we use it mostly to determine the memory order of the *output* and not for 
safty purpose. That is the macro of course and I think yelling people to use 
flags.fnc in python is better.
 
- Sebastian


On Mon Nov 2 08:52:01 2015 GMT+0330, Sturla Molden wrote:
> Charles R Harris  wrote:
> 
> >1. Return `a.flags.f_contiguous`. This differs for 1-D arrays, but is
> >most consistent with the name isfortran.
> 
> If the idea is to determine if an array can safely be passed to Fortran,
> this is the correct one.
> 
> >2. Return `a.flags.f_contiguous and a.ndim > 1`, which would be backward
> >compatible.
> 
> This one is just wrong.
> 
> A compromize might be to raise an exception in the case of a.ndim<2. 
> 
> Sturla
> 
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] isfortran compatibility in numpy 1.10.

[Charles R Harris]

On Mon, Nov 2, 2015 at 11:28 AM, Sebastian Berg 
wrote:

> I bet it has all been said already, but to note just in case. In numpy
> itself we use it mostly to determine the memory order of the *output* and
> not for safty purpose. That is the macro of course and I think yelling
> people to use flags.fnc in python is better.
>

Probably all the Numpy uses of `PyArray_ISFORTRAN` should be audited. My
guess is that it will be found to be incorrect in some (most?) of the
places.

Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ANN: numexpr 2.4.6 released

[Francesc Alted]

Hi,

This is a quick release fixing some reported problems in the 2.4.5 version
that I announced a few hours ago.  Hope I have fixed the main issues now.
Now, the official announcement:

=
 Announcing Numexpr 2.4.6
=

Numexpr is a fast numerical expression evaluator for NumPy.  With it,
expressions that operate on arrays (like "3*a+4*b") are accelerated
and use less memory than doing the same calculation in Python.

It wears multi-threaded capabilities, as well as support for Intel's
MKL (Math Kernel Library), which allows an extremely fast evaluation
of transcendental functions (sin, cos, tan, exp, log...)  while
squeezing the last drop of performance out of your multi-core
processors.  Look here for a some benchmarks of numexpr using MKL:

https://github.com/pydata/numexpr/wiki/NumexprMKL

Its only dependency is NumPy (MKL is optional), so it works well as an
easy-to-deploy, easy-to-use, computational engine for projects that
don't want to adopt other solutions requiring more heavy dependencies.

What's new
==

This is a quick maintenance version that offers better handling of
MSVC symbols (#168, Francesc Alted), as well as fising some
UserWarnings in Solaris (#189, Graham Jones).

In case you want to know more in detail what has changed in this
version, see:

https://github.com/pydata/numexpr/blob/master/RELEASE_NOTES.rst

Where I can find Numexpr?
=

The project is hosted at GitHub in:

https://github.com/pydata/numexpr

You can get the packages from PyPI as well (but not for RC releases):

http://pypi.python.org/pypi/numexpr

Share your experience
=

Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.


Enjoy data!

-- 
Francesc Alted
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ANN: numexpr 2.4.5 released

[Francesc Alted]

=
 Announcing Numexpr 2.4.5
=

Numexpr is a fast numerical expression evaluator for NumPy.  With it,
expressions that operate on arrays (like "3*a+4*b") are accelerated
and use less memory than doing the same calculation in Python.

It wears multi-threaded capabilities, as well as support for Intel's
MKL (Math Kernel Library), which allows an extremely fast evaluation
of transcendental functions (sin, cos, tan, exp, log...)  while
squeezing the last drop of performance out of your multi-core
processors.  Look here for a some benchmarks of numexpr using MKL:

https://github.com/pydata/numexpr/wiki/NumexprMKL

Its only dependency is NumPy (MKL is optional), so it works well as an
easy-to-deploy, easy-to-use, computational engine for projects that
don't want to adopt other solutions requiring more heavy dependencies.

What's new
==

This is a maintenance release where an important bug in multithreading
code has been fixed (#185 Benedikt Reinartz, Francesc Alted).  Also,
many harmless warnings (overflow/underflow, divide by zero and others)
in the test suite have been silenced  (#183, Francesc Alted).

In case you want to know more in detail what has changed in this
version, see:

https://github.com/pydata/numexpr/blob/master/RELEASE_NOTES.rst

Where I can find Numexpr?
=

The project is hosted at GitHub in:

https://github.com/pydata/numexpr

You can get the packages from PyPI as well (but not for RC releases):

http://pypi.python.org/pypi/numexpr

Share your experience
=

Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.


Enjoy data!

-- 
Francesc Alted
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [NumPy/Swig] Return NumPy array with same size as input array (no additional length argument)

[Chris Barker]

On Fri, Oct 30, 2015 at 11:15 PM, laurentes 
wrote:

> Using Swig, I don't manage to (properly) create the Python Binding for the
> following C-like function:
>
> void add_array(double* input_array1, double* input_array2, double*
> output_array, int length);
>
> where the three arrays have all the same length.
>

do you have to use SWIG? this would be really easy in Cython

cdef cdef extern from "your_header.h":
void add_array(double* input_array1,
   double* input_array2,
   double* output_array,
   int length)


def py_add_array( np.ndarray[double, ndim=1] arr1,
  np.ndarray[double, ndim=1] arr2):

cdef int length

if arr1.shape != arr2.shape:
raise ValueError("Arrays must be the same size")

length = arr1.shape[0]

cdef np.ndarray[double, ndim=1] out_arr = np.empty((length),
dtype=np.float64)

add_array([0],
  [0],
  _arr[0],
  length)

return out_arr


Untested and from memory -- but you get the idea.

-CHB


> >
> > This is similar to  this thread
> > <
> http://numpy-discussion.10968.n7.nabble.com/Numpy-SWIG-td25709.html#a25710
> >
> > , which has never been fully addressed online.
> >
> > From Python, I would like to be able to call:
> >
> add_array(input_array1, input_array2)
>
> which would return me a newly allocated NumPy array (output_array) with the
> result.
>
> In my Swig file, I've first used the wrapper function trick described  here
> <
> http://web.mit.edu/6.863/spring2011/packages/numpy_src/doc/swig/doc/numpy_swig.html#a-common-example
> >
> , that is:
>
> %apply (double* IN_ARRAY1, int DIM1) {(double* input_array1, int length1),
> (double* input_array2, int length2)};
> %apply (double* ARGOUT_ARRAY1, int DIM1) {(double* output_array, int
> length3)};
>
> %rename (add_array) my_add_array;
> %exception my_add_array {
> $action
> if (PyErr_Occurred()) SWIG_fail;
> }
> %inline %{
> void my_add_array(double* input_array1, int length1, double* input_array2,
> int length2, double* output_array, int length3) {
>   if (length1 != length2 || length1 != length3) {
>   PyErr_Format(PyExc_ValueError,
>"Arrays of lengths (%d,%d,%d) given",
>length1, length2, length3);
>   }
>   else {
> add_array(input_array1, input_array2, output_array, length1);
>   }
> }
> %}
>
> This allows me to call the function from Python using
> add_array(input_array1, input_array2, length). But the third argument of
> this function is useless and this function does not look 'Pythonic'.
>
> Could someone help me to modify my Swig file, such that only the first two
> arguments are required for the Python API?
>
> Thanks a lot,
> Laurent
>
>
>
> --
> View this message in context:
> http://numpy-discussion.10968.n7.nabble.com/NumPy-Swig-Return-NumPy-array-with-same-size-as-input-array-no-additional-length-argument-tp41601.html
> Sent from the Numpy-discussion mailing list archive at Nabble.com.
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Numpy style docstring support in Sphinx and PyCharm

[Ralf Gommers]

Hi all,

Just noticed this:
   http://sphinx-doc.org/latest/ext/napoleon.html
   http://www.jetbrains.com/pycharm/whatsnew/index.html#GDocstrings
Slowly conquering the docstring world:)

Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

[Chris Barker]

On Tue, Oct 27, 2015 at 8:25 AM, Nathan Goldbaum 
wrote:

> Interestingly, conda actually does "setup.py install" in the recipe for
> numpy:
>

indeed -- many, many conda packages do setup.py install, whihc doesn't mean
it's a good idea --personally, I'm trying hard to switch them all to:

pip install ./

:-)

Which reminds me, the conda skelaton command craes a pip install build.sh
file -- I really need to submit a PR for that ...

There are two cases where a 'pip install' run might go off and start
>> downloading packages without asking you:
>>
>
for my part, regular old setup.py isntall oftem goes off and istalls sutff
too - using easy_install, which really sucks...

This is making me want a setuptools-lite again  -- see the distutils SIG if
you're curious.

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] deprecate fromstring() for text reading?

[Chris Barker]

On Tue, Oct 27, 2015 at 7:30 AM, Benjamin Root  wrote:

> FWIW, when I needed a fast Fixed Width reader
>

was there potentially no whitespace between fields in that case? In which
case, it really isn a different use-case than delimited text -- if it's at
all common, a version written in C would be nice and fast. and nat hard to
do.

But fromstring never would have helped you with that anyway :-)

-CHB



> for a very large dataset last year, I found that np.genfromtext() was
> faster than pandas' read_fwf(). IIRC, pandas' text reading code fell back
> to pure python for fixed width scenarios.
>
> On Fri, Oct 23, 2015 at 8:22 PM, Chris Barker - NOAA Federal <
> chris.bar...@noaa.gov> wrote:
>
>> Grabbing the pandas csv reader would be great, and I hope it happens
>> sooner than later, though alas, I haven't the spare cycles for it either.
>>
>> In the meantime though, can we put a deprecation Warning in when using
>> fromstring() on text files? It's really pretty broken.
>>
>> -Chris
>>
>> On Oct 23, 2015, at 4:02 PM, Jeff Reback  wrote:
>>
>>
>>
>> On Oct 23, 2015, at 6:49 PM, Nathaniel Smith  wrote:
>>
>> On Oct 23, 2015 3:30 PM, "Jeff Reback"  wrote:
>> >
>> > On Oct 23, 2015, at 6:13 PM, Charles R Harris <
>> charlesr.har...@gmail.com> wrote:
>> >
>> >>
>> >>
>> >> On Thu, Oct 22, 2015 at 5:47 PM, Chris Barker - NOAA Federal <
>> chris.bar...@noaa.gov> wrote:
>> >>>
>> >>>
>>  I think it would be good to keep the usage to read binary data at
>> least.
>> >>>
>> >>>
>> >>> Agreed -- it's only the text file reading I'm proposing to deprecate.
>> It was kind of weird to cram it in there in the first place.
>> >>>
>> >>> Oh, fromfile() has the same issues.
>> >>>
>> >>> Chris
>> >>>
>> >>>
>>  Or is there a good alternative to `np.fromstring(,
>> dtype=...)`?  -- Marten
>> 
>>  On Thu, Oct 22, 2015 at 1:03 PM, Chris Barker 
>> wrote:
>> >
>> > There was just a question about a bug/issue with scipy.fromstring
>> (which is numpy.fromstring) when used to read integers from a text file.
>> >
>> >
>> https://mail.scipy.org/pipermail/scipy-user/2015-October/036746.html
>> >
>> > fromstring() is bugging and inflexible for reading text files --
>> and it is a very, very ugly mess of code. I dug into it a while back, and
>> gave up -- just to much of a mess!
>> >
>> > So we really should completely re-implement it, or deprecate it. I
>> doubt anyone is going to do a big refactor, so that means deprecating it.
>> >
>> > Also -- if we do want a fast read numbers from text files function
>> (which would be nice, actually), it really should get a new name anyway.
>> >
>> > (and the hopefully coming new dtype system would make it easier to
>> write cleanly)
>> >
>> > I'm not sure what deprecating something means, though -- have it
>> raise a deprecation warning in the next version?
>> >
>> >>
>> >> There was discussion at SciPy 2015 of separating out the text reading
>> abilities of Pandas so that numpy could include it. We should contact Jeff
>> Rebeck and see about moving that forward.
>> >
>> >
>> > IIRC Thomas Caswell was interested in doing this :)
>>
>> When he was in Berkeley a few weeks ago he assured me that every night
>> since SciPy he has dutifully been feeling guilty about not having done it
>> yet. I think this week his paltry excuse is that he's "on his honeymoon" or
>> something.
>>
>> ...which is to say that if someone has some spare cycles to take this
>> over then I think that might be a nice wedding present for him :-).
>>
>> (The basic idea is to take the text reading backend behind
>> pandas.read_csv and extract it into a standalone package that pandas could
>> depend on, and that could also be used by other packages like numpy (among
>> others -- I thing dato's SFrame package has a fork of this code as well?))
>>
>> -n
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>> I can certainly provide guidance on how/what to extract but don't have
>> spare cycles myself for this :(
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>> ___
>> NumPy-Discussion mailing list
>> NumPy-Discussion@scipy.org
>> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   

Re: [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

[Nathaniel Smith]

[Adding distutils-sig to the CC as a heads-up. The context is that
numpy is looking at deprecating the use of 'python setup.py install'
and enforcing the use of 'pip install .' instead, and running into
some issues that will probably need to be addressed if 'pip install .'
is going to become the standard interface to work with source trees.]

On Sun, Nov 1, 2015 at 3:16 PM, Ralf Gommers  wrote:
[...]
> Hmm, after some more testing I'm going to have to bring up a few concerns
> myself:
>
> 1. ``pip install .`` still has a clear bug; it starts by copying everything
> (including .git/ !) to a tempdir with shutil, which is very slow. And the
> fix for that will go via ``setup.py sdist``, which is still slow.

Ugh. If 'pip (install/wheel) .' is supposed to become the standard way
to build things, then it should probably build in-place by default.
Working in a temp dir makes perfect sense for 'pip install
' or 'pip install ', but if the user supplies an
actual named on-disk directory then presumably the user is expecting
this directory to be used, and to be able to take advantage of
incremental rebuilds etc., no?

> 2. ``pip install .`` silences build output, which may make sense for some
> usecases, but for numpy it just sits there for minutes with no output after
> printing "Running setup.py install for numpy". Users will think it hangs and
> Ctrl-C it. https://github.com/pypa/pip/issues/2732

I tend to agree with the commentary there that for end users this is
different but no worse than the current situation where we spit out
pages of "errors" that don't mean anything :-). I posted a suggestion
on that bug that might help with the apparent hanging problem.

> 3. ``pip install .`` refuses to upgrade an already installed development
> version. For released versions that makes sense, but if I'm in a git tree
> then I don't want it to refuse because 1.11.0.dev0+githash1 compares equal
> to 1.11.0.dev0+githash2. Especially after waiting a few minutes, see (1).

Ugh, this is clearly just a bug -- `pip install .` should always
unconditionally install, IMO. (Did you file a bug yet?) At least the
workaround is just 'pip uninstall numpy; pip install .', which is
still better the running 'setup.py install' and having it blithely
overwrite some files and not others.

The first and last issue seem like ones that will mostly only affect
developers, who should mostly have the ability to deal with these
weird issues (or just use setup.py install --force if that's what they
prefer)? This still seems like a reasonable trade-off to me if it also
has the effect of reducing the number of weird broken installs among
our thousands-of-times-larger userbase.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Distutils] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

[Nathaniel Smith]

On Nov 2, 2015 6:51 PM, "Robert Collins"  wrote:
>
> On 3 November 2015 at 14:57, Nathaniel Smith  wrote:
> > [Adding distutils-sig to the CC as a heads-up. The context is that
> > numpy is looking at deprecating the use of 'python setup.py install'
> > and enforcing the use of 'pip install .' instead, and running into
> > some issues that will probably need to be addressed if 'pip install .'
> > is going to become the standard interface to work with source trees.]
> >
> > On Sun, Nov 1, 2015 at 3:16 PM, Ralf Gommers 
wrote:
> > [...]
> >> Hmm, after some more testing I'm going to have to bring up a few
concerns
> >> myself:
> >>
> >> 1. ``pip install .`` still has a clear bug; it starts by copying
everything
> >> (including .git/ !) to a tempdir with shutil, which is very slow. And
the
> >> fix for that will go via ``setup.py sdist``, which is still slow.
> >
> > Ugh. If 'pip (install/wheel) .' is supposed to become the standard way
> > to build things, then it should probably build in-place by default.
> > Working in a temp dir makes perfect sense for 'pip install
> > ' or 'pip install ', but if the user supplies an
> > actual named on-disk directory then presumably the user is expecting
> > this directory to be used, and to be able to take advantage of
> > incremental rebuilds etc., no?
>
> Thats what 'pip install -e .' does. 'setup.py develop' -> 'pip install -e
.'

I'm not talking about in place installs, I'm talking about e.g. building a
wheel and then tweaking one file and rebuilding -- traditionally build
systems go to some effort to keep track of intermediate artifacts and reuse
them across builds when possible, but if you always copy the source tree
into a temporary directory before building then there's not much the build
system can do.

> >> 3. ``pip install .`` refuses to upgrade an already installed
development
> >> version. For released versions that makes sense, but if I'm in a git
tree
> >> then I don't want it to refuse because 1.11.0.dev0+githash1 compares
equal
> >> to 1.11.0.dev0+githash2. Especially after waiting a few minutes, see
(1).
> >
> > Ugh, this is clearly just a bug -- `pip install .` should always
> > unconditionally install, IMO. (Did you file a bug yet?) At least the
> > workaround is just 'pip uninstall numpy; pip install .', which is
> > still better the running 'setup.py install' and having it blithely
> > overwrite some files and not others.
>
> There is a bug open. https://github.com/pypa/pip/issues/536

Thanks!

-n
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion