[Numpy-discussion] guvectorize, a helper for writing generalized ufuncs

2016-09-13 Thread Stephan Hoyer
NumPy has the handy np.vectorize for turning Python code that operates on scalars into a function that vectorizes works like a ufunc, but no helper function for creating generalized ufuncs ( http://docs.scipy.org/doc/numpy/reference/c-api.generalized-ufuncs.html). np.apply_along_axis accomplishes

Re: [Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Chris Barker
We had a big long discussion about this on this list a while back (maybe 2 yrs ago???) please search the archives to find it. Though I'm pretty sure that we never did come to a conclusion. I think it stared with wanting better support ofr unicode in loadtxt and the like, and ended up delving into

Re: [Numpy-discussion] guvectorize, a helper for writing generalized ufuncs

2016-09-13 Thread Nathan Goldbaum
On Tue, Sep 13, 2016 at 11:47 AM, Stephan Hoyer wrote: > NumPy has the handy np.vectorize for turning Python code that operates on > scalars into a function that vectorizes works like a ufunc, but no helper > function for creating generalized ufuncs (http://docs.scipy.org/doc/

Re: [Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Stephan Hoyer
On Tue, Sep 13, 2016 at 11:05 AM, Lluís Vilanova wrote: > Whenever we repr an array using 'S', we can instead show a unicode in py3. > That > keeps the binary representation, but will always show the expected result > to > users, and it's only a handful of lines added to

Re: [Numpy-discussion] guvectorize, a helper for writing generalized ufuncs

2016-09-13 Thread Stephan Hoyer
On Tue, Sep 13, 2016 at 10:39 AM, Nathan Goldbaum wrote: > I'm curious whether you have a plan to deal with the python functional > call overhead. Numba gets around this by JIT-compiling python functions - > is there something analogous you can do in NumPy or will this

Re: [Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Lluís Vilanova
Chris Barker writes: > We had a big long discussion about this on this list a while back (maybe 2 yrs > ago???) please search the archives to find it. Though I'm pretty sure that we > never did come to a conclusion. I think it stared with wanting better support > ofr unicode in loadtxt and the

Re: [Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Chris Barker
On Tue, Sep 13, 2016 at 11:05 AM, Lluís Vilanova wrote: > Great, that's the type of info I wanted to get before going forward. I > guess > there's code relying on the binary representation of 'S' to do mmap's or > access > the array's raw contents. Is that right? yes,

Re: [Numpy-discussion] guvectorize, a helper for writing generalized ufuncs

2016-09-13 Thread Travis Oliphant
There has been some discussion on the Numba mailing list as well about a version of guvectorize that doesn't compile for testing and flexibility. Having this be inside NumPy itself seems ideal. -Travis On Tue, Sep 13, 2016 at 12:59 PM, Stephan Hoyer wrote: > On Tue, Sep 13,

Re: [Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Lluís Vilanova
Sebastian Berg writes: > On Di, 2016-09-13 at 15:02 +0200, Lluís Vilanova wrote: >> Hi! I'm giving a shot to issue #3184 [1], based on the observation >> that the >> string dtype ('S') under python 3 uses byte arrays instead of unicode >> (the only >> readable string type in python 3). >> >>

Re: [Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Lluís Vilanova
Sebastian Berg writes: > On Di, 2016-09-13 at 15:02 +0200, Lluís Vilanova wrote: >> Hi! I'm giving a shot to issue #3184 [1], based on the observation >> that the >> string dtype ('S') under python 3 uses byte arrays instead of unicode >> (the only >> readable string type in python 3). >> >>

Re: [Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Sebastian Berg
On Di, 2016-09-13 at 15:02 +0200, Lluís Vilanova wrote: > Hi! I'm giving a shot to issue #3184 [1], based on the observation > that the > string dtype ('S') under python 3 uses byte arrays instead of unicode > (the only > readable string type in python 3). > > This brings two major problems: > >

[Numpy-discussion] String & unicode arrays vs text loading in python 3

2016-09-13 Thread Lluís Vilanova
Hi! I'm giving a shot to issue #3184 [1], based on the observation that the string dtype ('S') under python 3 uses byte arrays instead of unicode (the only readable string type in python 3). This brings two major problems: * numpy code has to go through loops to open and read files as binary