[Numpy-discussion] Fwd: numpy test fails with Illegal instruction'
Dear numpy users, I am trying to get numpy to work on my computer, but so far no luck. When I run `numpy.test(verbose=10)` it crashes with test_polyfit (test_polynomial.TestDocs) ... Illegal instruction In the FAQ it states that I should provide the following information (running Ubuntu 12.04 64bit): os.name = 'posix' uname -r = 3.2.0-35-generic sys.platform = 'linux2' sys.version = '2.7.3 (default, Aug 1 2012, 05:14:39) \n[GCC 4.6.3]' Atlas is not installed (not required for numpy, only for scipy right?) It fails both when I install numpy 1.6.2 with `pip install numpy` and if I install the latest dev version from git. Can someone give me some pointers on how to solve this? I will be grateful for any help you can provide. Kind regards, Gerhard Burger ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Casting Bug or a Feature?
On Thu, Jan 17, 2013 at 2:34 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Hi, Actually, this behavior is already present in other languages, so I'm -1 on additional verbosity. Of course a += b is not the same as a = a + b. The first one modifies the object a, the second one creates a new object and puts it inside a. The behavior IS consistent. The inplace operation is standard, but my guess is that the silent downcasting is not. in python a = 1 a += 5.3 a 6.2998 a = 1 a *= 1j a 1j I have no idea about other languages. Josef Cheers, Matthieu 2013/1/17 Paul Anton Letnes paul.anton.let...@gmail.com On 17.01.2013 04:43, Patrick Marsh wrote: Thanks, everyone for chiming in. Now that I know this behavior exists, I can explicitly prevent it in my code. However, it would be nice if a warning or something was generated to alert users about the inconsistency between var += ... and var = var + ... Patrick I agree wholeheartedly. I actually, for a long time, used to believe that python would translate a += b to a = a + b and was bitten several times by this bug. A warning (which can be silenced if you desperately want to) would be really nice, imho. Keep up the good work, Paul ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Casting Bug or a Feature?
On 01/17/2013 01:27 PM, josef.p...@gmail.com wrote: On Thu, Jan 17, 2013 at 2:34 AM, Matthieu Brucher matthieu.bruc...@gmail.com wrote: Hi, Actually, this behavior is already present in other languages, so I'm -1 on additional verbosity. Of course a += b is not the same as a = a + b. The first one modifies the object a, the second one creates a new object and puts it inside a. The behavior IS consistent. The inplace operation is standard, but my guess is that the silent downcasting is not. in python a = 1 a += 5.3 a 6.2998 a = 1 a *= 1j a 1j I have no idea about other languages. I don't think the comparison with Python scalars is relevant since they are immutable: In [9]: a = 1 In [10]: b = a In [11]: a *= 1j In [12]: b Out[12]: 1 In-place operators exists for lists, but I don't know what the equivalent of a down-cast would be... In [3]: a = [0, 1] In [4]: b = a In [5]: a *= 2 In [6]: b Out[6]: [0, 1, 0, 1] Dag Sverre ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On 1/16/2013 11:41 PM, Nathaniel Smith wrote: On 16 Jan 2013 17:54, josef.p...@gmail.com mailto:josef.p...@gmail.com wrote: a = np.random.random_integers(0, 5, size=5) b = a.sort() b a array([0, 1, 2, 5, 5]) b = np.random.shuffle(a) b b = np.random.permutation(a) b array([0, 5, 5, 2, 1]) How do I remember if shuffle shuffles or permutes ? Do we have a list of functions that are inplace? I rather like the convention used elsewhere in Python of naming in-place operations with present tense imperative verbs, and out-of-place operations with past participles. So you have sort/sorted, reverse/reversed, etc. Here this would suggest we name these two operations as either shuffle() and shuffled(), or permute() and permuted(). I like this (tense) suggestion. It seems easy to remember. --jv -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
Hi, Le 14/01/2013 20:17, Alan G Isaac a écrit : a = np.tile(5,(1,2,3)) a array([[[5, 5, 5], [5, 5, 5]]]) np.tile(1,a.shape) array([[[1, 1, 1], [1, 1, 1]]]) I had not realized a scalar first argument was possible. I didn't know either ! I discovered this use in the thread of this discussion. Just like Ben, I've almost never used np.tile neither its cousin np.repeat... Now, in the process of rediscovering those two functions, I was just wondering whether it would make sense to repackage them in order to allow the simple functionality of initializing a non-empty array. In term of choosing the name (or actually the verb), I prefer repeat because it's a more familiar concept than tile. However, repeat may need more changes to make it work than tile. Indeed we currently have : tile(nan, (3,3)) # works fine, but is pretty slow for that purpose, And doesn't accept a dtype arg array([[ nan, nan, nan], [ nan, nan, nan], [ nan, nan, nan]]) Doesn't work for that purpose: repeat(nan, (3,3)) [...] ValueError: a.shape[axis] != len(repeats) So what people think of this green approach of recycling existing API into a slightly different function (without breaking current behavior of course) Best, Pierre signature.asc Description: OpenPGP digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fwd: numpy test fails with Illegal instruction'
On 17 January 2013 12:01, Gerhard Burger burger...@gmail.com wrote: When I run `numpy.test(verbose=10)` it crashes with test_polyfit (test_polynomial.TestDocs) ... Illegal instruction In the FAQ it states that I should provide the following information (running Ubuntu 12.04 64bit): os.name = 'posix' uname -r = 3.2.0-35-generic sys.platform = 'linux2' sys.version = '2.7.3 (default, Aug 1 2012, 05:14:39) \n[GCC 4.6.3]' Atlas is not installed (not required for numpy, only for scipy right?) It fails both when I install numpy 1.6.2 with `pip install numpy` and if I install the latest dev version from git. Very strange. I tried to reproduce this on 64-bit Ubuntu 12.04 (by removing my ATLAS, BLAS, LAPACK etc..) but couldn't: $ python -c import numpy; numpy.test() Running unit tests for numpy NumPy version 1.6.2 NumPy is installed in /home/scott/.virtualenvs/numpy-tmp/local/lib/python2.7/site-packages/numpy Python version 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] nose version 1.2.1 . -- Ran 3568 tests in 14.170s OK (KNOWNFAIL=5, SKIP=5) $ python -c import numpy; numpy.show_config() blas_info: NOT AVAILABLE lapack_info: NOT AVAILABLE atlas_threads_info: NOT AVAILABLE blas_src_info: NOT AVAILABLE lapack_src_info: NOT AVAILABLE atlas_blas_threads_info: NOT AVAILABLE lapack_opt_info: NOT AVAILABLE blas_opt_info: NOT AVAILABLE atlas_info: NOT AVAILABLE lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE atlas_blas_info: NOT AVAILABLE mkl_info: NOT AVAILABLE Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! best, Pierre signature.asc Description: OpenPGP digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fwd: numpy test fails with Illegal instruction'
I read somewhere that it could have to do with the sse instructions that your processor is capable of, but my processor is not that old, so I would think that is not the problem... On Thu, Jan 17, 2013 at 3:12 PM, Scott Sinclair scott.sinclair...@gmail.com wrote: On 17 January 2013 12:01, Gerhard Burger burger...@gmail.com wrote: When I run `numpy.test(verbose=10)` it crashes with test_polyfit (test_polynomial.TestDocs) ... Illegal instruction In the FAQ it states that I should provide the following information (running Ubuntu 12.04 64bit): os.name = 'posix' uname -r = 3.2.0-35-generic sys.platform = 'linux2' sys.version = '2.7.3 (default, Aug 1 2012, 05:14:39) \n[GCC 4.6.3]' Atlas is not installed (not required for numpy, only for scipy right?) It fails both when I install numpy 1.6.2 with `pip install numpy` and if I install the latest dev version from git. Very strange. I tried to reproduce this on 64-bit Ubuntu 12.04 (by removing my ATLAS, BLAS, LAPACK etc..) but couldn't: $ python -c import numpy; numpy.test() Running unit tests for numpy NumPy version 1.6.2 NumPy is installed in /home/scott/.virtualenvs/numpy-tmp/local/lib/python2.7/site-packages/numpy Python version 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] nose version 1.2.1 . -- Ran 3568 tests in 14.170s OK (KNOWNFAIL=5, SKIP=5) $ python -c import numpy; numpy.show_config() blas_info: NOT AVAILABLE lapack_info: NOT AVAILABLE atlas_threads_info: NOT AVAILABLE blas_src_info: NOT AVAILABLE lapack_src_info: NOT AVAILABLE atlas_blas_threads_info: NOT AVAILABLE lapack_opt_info: NOT AVAILABLE blas_opt_info: NOT AVAILABLE atlas_info: NOT AVAILABLE lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE atlas_blas_info: NOT AVAILABLE mkl_info: NOT AVAILABLE Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Do we want scalar casting to behave as it does at the moment?
Hi, On Wed, Jan 9, 2013 at 6:07 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 01/09/2013 06:22 PM, Chris Barker - NOAA Federal wrote: On Wed, Jan 9, 2013 at 7:09 AM, Nathaniel Smith n...@pobox.com wrote: This is a general issue applying to data which is read from real-world external sources. For example, digitizers routinely represent their samples as int8's or int16's, and you apply a scale and offset to get a reading in volts. This particular case is actually handled fine by 1.5, because int array + float scalar *does* upcast to float. It's width that's ignored (int8 versus int32), not the basic kind of data (int versus float). But overall this does sound like a problem -- but it's not a problem with the scalar/array rules, it's a problem with working with narrow width data in general. Exactly -- this is key. details asside, we essentially have a choice between an approach that makes it easy to preserver your values -- upcasting liberally, or making it easy to preserve your dtype -- requiring users to specifically upcast where needed. IIRC, our experience with earlier versions of numpy (and Numeric before that) is that all too often folks would choose a small dtype quite deliberately, then have it accidentally upcast for them -- this was determined to be not-so-good behavior. I think the HDF (and also netcdf...) case is a special case -- the small dtype+scaling has been chosen deliberately by whoever created the data file (to save space), but we would want it generally opaque to the consumer of the file -- to me, that means the issue should be adressed by the file reading tools, not numpy. If your HDF5 reader chooses the the resulting dtype explicitly, it doesn't matter what numpy's defaults are. If the user wants to work with the raw, unscaled arrays, then they should know what they are doing. +1. I think h5py should consider: File(my.h5)['int8_dset'].dtype == int64 File(my.h5, preserve_dtype=True)['int8_dset'].dtype == int8 Returning to this thread - did we have a decision? With further reflection, it seems to me we will have a tough time going back to the 1.5 behavior now - we might be shutting the stable door after the cat is out of the bag, if you see what I mean. Maybe we should change the question to the desirable behavior in the long term. I am starting to wonder if we should aim for making * scalar and array casting rules the same; * Python int / float scalars become int32 / 64 or float64; This has the benefit of being very easy to understand and explain. It makes dtypes predictable in the sense they don't depend on value. Those wanting to maintain - say - float32 will need to cast scalars to float32. Maybe the use-cases motivating the scalar casting rules - maintaining float32 precision in particular - can be dealt with by careful casting of scalars, throwing the burden onto the memory-conscious to maintain their dtypes. Or is there a way of using flags to ufuncs to emulate the 1.5 casting rules? Do y'all agree this is desirable in the long term? If so, how should we get there? It seems to me we're about 25 percent of the way there with the current scalar casting rule. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
Is it really better to have `permute` and `permuted` than to add a keyword? (Note that these are actually still ambiguous, except by convention.) Btw, two separate issues seem to be running side by side. i. should in-place operations return their result? ii. how can we signal that an operation is inplace? I expect NumPy to do inplace operations when feasible, so maybe they could take an `out` keyword with a None default. Possibly recognize `out=True` as asking for the original array object to be returned (mutated); `out='copy'` as asking for a copy to be created, operated upon, and returned; and `out=a` to ask for array `a` to be used for the output (without changing the original object, and with a return value of None). Alan Isaac ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On Thu, Jan 17, 2013 at 8:54 AM, Jim Vickroy jim.vick...@noaa.gov wrote: On 1/16/2013 11:41 PM, Nathaniel Smith wrote: On 16 Jan 2013 17:54, josef.p...@gmail.com wrote: a = np.random.random_integers(0, 5, size=5) b = a.sort() b a array([0, 1, 2, 5, 5]) b = np.random.shuffle(a) b b = np.random.permutation(a) b array([0, 5, 5, 2, 1]) How do I remember if shuffle shuffles or permutes ? Do we have a list of functions that are inplace? I rather like the convention used elsewhere in Python of naming in-place operations with present tense imperative verbs, and out-of-place operations with past participles. So you have sort/sorted, reverse/reversed, etc. Here this would suggest we name these two operations as either shuffle() and shuffled(), or permute() and permuted(). I like this (tense) suggestion. It seems easy to remember. --jv And another score for functions as verbs! :-P Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fwd: numpy test fails with Illegal instruction'
Solved it, did a backtrace with gdb and the error came somewhere from an old lapack version that was installed on my machine (I thought I wouldn't have these issues in a virtualenv). but anyway after I removed it, and installed numpy again, it ran without problems! On Thu, Jan 17, 2013 at 3:18 PM, Gerhard Burger burger...@gmail.com wrote: I read somewhere that it could have to do with the sse instructions that your processor is capable of, but my processor is not that old, so I would think that is not the problem... On Thu, Jan 17, 2013 at 3:12 PM, Scott Sinclair scott.sinclair...@gmail.com wrote: On 17 January 2013 12:01, Gerhard Burger burger...@gmail.com wrote: When I run `numpy.test(verbose=10)` it crashes with test_polyfit (test_polynomial.TestDocs) ... Illegal instruction In the FAQ it states that I should provide the following information (running Ubuntu 12.04 64bit): os.name = 'posix' uname -r = 3.2.0-35-generic sys.platform = 'linux2' sys.version = '2.7.3 (default, Aug 1 2012, 05:14:39) \n[GCC 4.6.3]' Atlas is not installed (not required for numpy, only for scipy right?) It fails both when I install numpy 1.6.2 with `pip install numpy` and if I install the latest dev version from git. Very strange. I tried to reproduce this on 64-bit Ubuntu 12.04 (by removing my ATLAS, BLAS, LAPACK etc..) but couldn't: $ python -c import numpy; numpy.test() Running unit tests for numpy NumPy version 1.6.2 NumPy is installed in /home/scott/.virtualenvs/numpy-tmp/local/lib/python2.7/site-packages/numpy Python version 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] nose version 1.2.1 . -- Ran 3568 tests in 14.170s OK (KNOWNFAIL=5, SKIP=5) $ python -c import numpy; numpy.show_config() blas_info: NOT AVAILABLE lapack_info: NOT AVAILABLE atlas_threads_info: NOT AVAILABLE blas_src_info: NOT AVAILABLE lapack_src_info: NOT AVAILABLE atlas_blas_threads_info: NOT AVAILABLE lapack_opt_info: NOT AVAILABLE blas_opt_info: NOT AVAILABLE atlas_info: NOT AVAILABLE lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE atlas_blas_info: NOT AVAILABLE mkl_info: NOT AVAILABLE Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On Thu, Jan 17, 2013 at 9:49 AM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Jan 17, 2013 at 8:54 AM, Jim Vickroy jim.vick...@noaa.gov wrote: On 1/16/2013 11:41 PM, Nathaniel Smith wrote: On 16 Jan 2013 17:54, josef.p...@gmail.com wrote: a = np.random.random_integers(0, 5, size=5) b = a.sort() b a array([0, 1, 2, 5, 5]) b = np.random.shuffle(a) b b = np.random.permutation(a) b array([0, 5, 5, 2, 1]) How do I remember if shuffle shuffles or permutes ? Do we have a list of functions that are inplace? I rather like the convention used elsewhere in Python of naming in-place operations with present tense imperative verbs, and out-of-place operations with past participles. So you have sort/sorted, reverse/reversed, etc. Here this would suggest we name these two operations as either shuffle() and shuffled(), or permute() and permuted(). I like this (tense) suggestion. It seems easy to remember. --jv And another score for functions as verbs! I don't thing the filled we discuss here is an action. The current ``fill`` is an inplace operation, operating on an existing array. ``filled`` would be the analog that returns a copy. However ``filled`` here is creating an object I still think ``array_filled`` is the most precise '''Create an array and initialize it with the ``value``, returning the array ''' my 2.5c Josef :-P Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On Wed, Jan 16, 2013 at 5:11 PM, eat e.antero.ta...@gmail.com wrote: Hi, In a recent thread http://article.gmane.org/gmane.comp.python.numeric.general/52772 it was proposed that .fill(.) should return self as an alternative for a trivial two-liner. I'm raising now the question: what if all in-place operations indeed could return self? How bad this would be? A 'strong' counter argument may be found at http://mail.python.org/pipermail/python-dev/2003-October/038855.html. But anyway, at least for me. it would be much more straightforward to implement simple mini dsl's ( http://en.wikipedia.org/wiki/Domain-specific_language) a much more straightforward manner. What do you think? I've read Guido about why he didn't like inplace operations returning self and found him convincing for a while. And then I listened to other folks express a preference for the freight train style and found them convincing also. I think it comes down to a preference for one style over another and I go back and forth myself. If I had to vote, I'd go for returning self, but I'm not sure it's worth breaking python conventions to do so. Chuck -eat P.S. FWIW, if this idea really gains momentum obviously I'm volunteering to create a PR of it. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On Thu, Jan 17, 2013 at 10:24 AM, josef.p...@gmail.com wrote: On Thu, Jan 17, 2013 at 9:49 AM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Jan 17, 2013 at 8:54 AM, Jim Vickroy jim.vick...@noaa.gov wrote: On 1/16/2013 11:41 PM, Nathaniel Smith wrote: On 16 Jan 2013 17:54, josef.p...@gmail.com wrote: a = np.random.random_integers(0, 5, size=5) b = a.sort() b a array([0, 1, 2, 5, 5]) b = np.random.shuffle(a) b b = np.random.permutation(a) b array([0, 5, 5, 2, 1]) How do I remember if shuffle shuffles or permutes ? Do we have a list of functions that are inplace? I rather like the convention used elsewhere in Python of naming in-place operations with present tense imperative verbs, and out-of-place operations with past participles. So you have sort/sorted, reverse/reversed, etc. Here this would suggest we name these two operations as either shuffle() and shuffled(), or permute() and permuted(). I like this (tense) suggestion. It seems easy to remember. --jv And another score for functions as verbs! I don't thing the filled we discuss here is an action. The current ``fill`` is an inplace operation, operating on an existing array. ``filled`` would be the analog that returns a copy. However ``filled`` here is creating an object I still think ``array_filled`` is the most precise '''Create an array and initialize it with the ``value``, returning the array ''' my 2.5c Josef Sorry, completely out of context. I shouldn't write emails, when I'm running in and out the office. Josef :-P Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] phase unwrapping (1d)
Hi Neal, Le 14/01/2013 15:39, Neal Becker a écrit : This code should explain all: import numpy as np arg = np.angle def nint (x): return int (x + 0.5) if x = 0 else int (x - 0.5) def unwrap (inp, y=np.pi, init=0, cnt=0): o = np.empty_like (inp) prev_o = init for i in range (len (inp)): o[i] = cnt * 2 * y + inp[i] delta = o[i] - prev_o if delta / y 1 or delta / y -1: n = nint (delta / (2*y)) o[i] -= 2*y*n cnt -= n prev_o = o[i] return o u = np.linspace (0, 400, 100) * np.pi/100 v = np.cos (u) + 1j * np.sin (u) plot (arg(v)) plot (arg(v) + arg (v)) plot (unwrap (arg (v))) plot (unwrap (arg (v) + arg (v))) I think your code does the job. I tried the following simplification, without the use of nint (which by the way could be replaced by int(floor(x)) I think) : def unwrap (inp, y=np.pi, init=0, cnt=0): o = np.empty_like (inp) prev_o = init for i in range (len (inp)): o[i] = cnt * 2 * y + inp[i] delta = o[i] - prev_o if delta / y 1: o[i] -= 2*y cnt -= 1 elif delta / y -1: o[i] += 2*y cnt += 1 prev_o = o[i] return o And now I understand the issue you described of phase changes of more than 2pi because the above indeed fail to unwrap (arg (v) + arg (v)). On the other hand np.unwrap handles it correctly. (I still don't know for the speed issue). Best, Pierre signature.asc Description: OpenPGP digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Casting Bug or a Feature?
On Wed, Jan 16, 2013 at 11:34 PM, Matthieu Brucher Of course a += b is not the same as a = a + b. The first one modifies the object a, the second one creates a new object and puts it inside a. The behavior IS consistent. Exactly -- if you ask me, the bug is that Python allows in_place operators for immutable objects -- they should be more than syntactic sugar. Of course, the temptation for += on regular numbers was just too much to resist. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On Thu, Jan 17, 2013 at 2:32 PM, Alan G Isaac alan.is...@gmail.com wrote: Is it really better to have `permute` and `permuted` than to add a keyword? (Note that these are actually still ambiguous, except by convention.) The convention in question, though, is that of English grammar. In practice everyone who uses numpy is a more-or-less skilled English speaker in any case, so re-using the conventions is helpful! Shake the martini! - an imperative command This is a complete statement all by itself. You can't say Hand me the shake the martini. In procedural languages like Python, there's a strong distinction between statements (whole lines, a = 1), which only matter because of their side-effects, and expressions (a + b) which have a value and can be embedded into a larger statement or expression ((a + b) + c). Shake the martini is clearly a statement, not an expression, and therefore clearly has a side-effect. shaken martini - a noun phrase Grammatically, this is like plain martini, you can use it anywhere you can use a noun. Hand me the martini, Hand me the shaken martini. In programming terms, it's an expression, not a statement. And side-effecting expressions are poor style, because when you read procedural code, you know each statement contains at least 1 side-effect, and it's much easier to figure out what's going on if each statement contains *exactly* one side-effect, and it's the top-most operation. This underlying readability guideline is actually baked much more deeply into Python than the sort/sorted distinction -- this is why in Python, 'a = 1' is *not* an expression, but a statement. C allows you to say things like b = (a = 1), but in Python you have to say a = 1; b = a. Btw, two separate issues seem to be running side by side. i. should in-place operations return their result? ii. how can we signal that an operation is inplace? I expect NumPy to do inplace operations when feasible, so maybe they could take an `out` keyword with a None default. Possibly recognize `out=True` as asking for the original array object to be returned (mutated); `out='copy'` as asking for a copy to be created, operated upon, and returned; and `out=a` to ask for array `a` to be used for the output (without changing the original object, and with a return value of None). Good point that numpy also has a nice convention with out= arguments for ufuncs. I guess that convention is, by default return a new array, but also allow one to modify the same (or another!) array in-place, by passing out=. So this would suggest that we'd have b = shuffled(a) shuffled(a, out=a) shuffled(a, out=b) shuffle(a) # same as shuffled(a, out=a) and if people are bothered by having both 'shuffled' and 'shuffle', then we drop 'shuffle'. (And the decision about whether to include the imperative form can be made on a case-by-case basis; having both shuffled and shuffle seems fine to me, but probably there are other cases where this is less clear.) There is also an argument that if out= is given, then we should always return None, in general. I'm having a lot of trouble thinking of any situation where it would be acceptable style (or even useful) to write something like: c = np.add(a, b, out=a) + 1 But, 'out=' is very large and visible (which makes the readability less terrible than it could be). And np.add always returns the out array when working out-of-place (so there's at least a weak countervailing convention). So I feel much more strongly that shuffle() should return None, than I do that np.add(out=...) should return None. A compromise position would be to make all new functions that take out= return None when out= is given, while leaving existing ufuncs and such as they are for now. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On 01/17/2013 05:33 PM, Nathaniel Smith wrote: On Thu, Jan 17, 2013 at 2:32 PM, Alan G Isaac alan.is...@gmail.com wrote: Is it really better to have `permute` and `permuted` than to add a keyword? (Note that these are actually still ambiguous, except by convention.) The convention in question, though, is that of English grammar. In practice everyone who uses numpy is a more-or-less skilled English speaker in any case, so re-using the conventions is helpful! Shake the martini! - an imperative command This is a complete statement all by itself. You can't say Hand me the shake the martini. In procedural languages like Python, there's a strong distinction between statements (whole lines, a = 1), which only matter because of their side-effects, and expressions (a + b) which have a value and can be embedded into a larger statement or expression ((a + b) + c). Shake the martini is clearly a statement, not an expression, and therefore clearly has a side-effect. shaken martini - a noun phrase Grammatically, this is like plain martini, you can use it anywhere you can use a noun. Hand me the martini, Hand me the shaken martini. In programming terms, it's an expression, not a statement. And side-effecting expressions are poor style, because when you read procedural code, you know each statement contains at least 1 side-effect, and it's much easier to figure out what's going on if each statement contains *exactly* one side-effect, and it's the top-most operation. This underlying readability guideline is actually baked much more deeply into Python than the sort/sorted distinction -- this is why in Python, 'a = 1' is *not* an expression, but a statement. C allows you to say things like b = (a = 1), but in Python you have to say a = 1; b = a. Btw, two separate issues seem to be running side by side. i. should in-place operations return their result? ii. how can we signal that an operation is inplace? I expect NumPy to do inplace operations when feasible, so maybe they could take an `out` keyword with a None default. Possibly recognize `out=True` as asking for the original array object to be returned (mutated); `out='copy'` as asking for a copy to be created, operated upon, and returned; and `out=a` to ask for array `a` to be used for the output (without changing the original object, and with a return value of None). Good point that numpy also has a nice convention with out= arguments for ufuncs. I guess that convention is, by default return a new array, but also allow one to modify the same (or another!) array in-place, by passing out=. So this would suggest that we'd have b = shuffled(a) shuffled(a, out=a) shuffled(a, out=b) shuffle(a) # same as shuffled(a, out=a) and if people are bothered by having both 'shuffled' and 'shuffle', then we drop 'shuffle'. (And the decision about whether to include the imperative form can be made on a case-by-case basis; having both shuffled and shuffle seems fine to me, but probably there are other cases where this is less clear.) In addition to the verb tense, I think it's important that mutators are methods whereas functions do not mutate their arguments: lst.sort() sorted(lst) So -1 on shuffle(a) and a.shuffled(). Dag Sverre There is also an argument that if out= is given, then we should always return None, in general. I'm having a lot of trouble thinking of any situation where it would be acceptable style (or even useful) to write something like: c = np.add(a, b, out=a) + 1 But, 'out=' is very large and visible (which makes the readability less terrible than it could be). And np.add always returns the out array when working out-of-place (so there's at least a weak countervailing convention). So I feel much more strongly that shuffle() should return None, than I do that np.add(out=...) should return None. A compromise position would be to make all new functions that take out= return None when out= is given, while leaving existing ufuncs and such as they are for now. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On Thu, Jan 17, 2013 at 6:08 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: In addition to the verb tense, I think it's important that mutators are methods whereas functions do not mutate their arguments: lst.sort() sorted(lst) Unfortunately this isn't really viable in a language like Python where you can't add methods to a class. (list.sort() versus sorted() has as much or more to do with the fact that sort's implementation only works on lists, while sorted takes an arbitrary iterable.) Even core python provides a function for in-place list randomization, not a method. Following the proposed rule would just mean that we couldn't provide in-place shuffles at all, which is clearly not going to be acceptable. -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] memory leak in 1.7
I've tracked down and fixed a memory leak in 1.7 and master. The pull request to check and backport is here: https://github.com/numpy/numpy/pull/2928 Thanks, Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
On 2013/01/17 4:13 AM, Pierre Haessig wrote: Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! I agree with your summary and conclusion. Eric best, Pierre ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
On Thu, Jan 17, 2013 at 5:04 PM, Eric Firing efir...@hawaii.edu wrote: On 2013/01/17 4:13 AM, Pierre Haessig wrote: Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! I agree with your summary and conclusion. Eric Can we at least have a np.nans() and np.infs() functions? This should cover an additional 4% of use-cases. Ben Root P.S. - I know they aren't verbs... ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Shouldn't all in-place operations simply return self?
On Thu, Jan 17, 2013 at 10:27 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Jan 16, 2013 at 5:11 PM, eat e.antero.ta...@gmail.com wrote: Hi, In a recent thread http://article.gmane.org/gmane.comp.python.numeric.general/52772 it was proposed that .fill(.) should return self as an alternative for a trivial two-liner. I'm raising now the question: what if all in-place operations indeed could return self? How bad this would be? A 'strong' counter argument may be found at http://mail.python.org/pipermail/python-dev/2003-October/038855.html. But anyway, at least for me. it would be much more straightforward to implement simple mini dsl's (http://en.wikipedia.org/wiki/Domain-specific_language) a much more straightforward manner. What do you think? I've read Guido about why he didn't like inplace operations returning self and found him convincing for a while. And then I listened to other folks express a preference for the freight train style and found them convincing also. I think it comes down to a preference for one style over another and I go back and forth myself. If I had to vote, I'd go for returning self, but I'm not sure it's worth breaking python conventions to do so. Chuck I'm -1 on breaking with Python convention without very good reasons. Ray ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
Hi, On Thu, Jan 17, 2013 at 10:10 PM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Jan 17, 2013 at 5:04 PM, Eric Firing efir...@hawaii.edu wrote: On 2013/01/17 4:13 AM, Pierre Haessig wrote: Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! I agree with your summary and conclusion. Eric Can we at least have a np.nans() and np.infs() functions? This should cover an additional 4% of use-cases. I'm a -0.5 on the new functions, just because they only save one line of code, and the use-case is fairly rare in my experience.. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
On Thu, Jan 17, 2013 at 2:10 PM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Jan 17, 2013 at 5:04 PM, Eric Firing efir...@hawaii.edu wrote: On 2013/01/17 4:13 AM, Pierre Haessig wrote: Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! I agree with your summary and conclusion. Eric Can we at least have a np.nans() and np.infs() functions? This should cover an additional 4% of use-cases. Ben Root P.S. - I know they aren't verbs... Would it be too weird or clumsy to extend the empty and empty_like functions to do the filling? np.empty((10, 10), fill=np.nan) np.empty_like(my_arr, fill=np.nan) -Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
Hi, On Thu, Jan 17, 2013 at 10:27 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jan 17, 2013 at 2:10 PM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Jan 17, 2013 at 5:04 PM, Eric Firing efir...@hawaii.edu wrote: On 2013/01/17 4:13 AM, Pierre Haessig wrote: Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! I agree with your summary and conclusion. Eric Can we at least have a np.nans() and np.infs() functions? This should cover an additional 4% of use-cases. Ben Root P.S. - I know they aren't verbs... Would it be too weird or clumsy to extend the empty and empty_like functions to do the filling? np.empty((10, 10), fill=np.nan) np.empty_like(my_arr, fill=np.nan) That sounds like a good idea to me. Someone wanting a fast way to fill an array will probably check out the 'empty' docstring first. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
2013/1/17 Matthew Brett matthew.br...@gmail.com: Hi, On Thu, Jan 17, 2013 at 10:27 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jan 17, 2013 at 2:10 PM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Jan 17, 2013 at 5:04 PM, Eric Firing efir...@hawaii.edu wrote: On 2013/01/17 4:13 AM, Pierre Haessig wrote: Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! I agree with your summary and conclusion. Eric Can we at least have a np.nans() and np.infs() functions? This should cover an additional 4% of use-cases. Ben Root P.S. - I know they aren't verbs... Would it be too weird or clumsy to extend the empty and empty_like functions to do the filling? np.empty((10, 10), fill=np.nan) np.empty_like(my_arr, fill=np.nan) That sounds like a good idea to me. Someone wanting a fast way to fill an array will probably check out the 'empty' docstring first. See you, Matthew +1 from me. Even though it *is* weird to have both empty and fill ;) -=- Olivier ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Do we want scalar casting to behave as it does at the moment?
On Thu, Jan 17, 2013 at 6:26 AM, Matthew Brett matthew.br...@gmail.com wrote: I am starting to wonder if we should aim for making * scalar and array casting rules the same; * Python int / float scalars become int32 / 64 or float64; aren't they already? I'm not sure what you are proposing. This has the benefit of being very easy to understand and explain. It makes dtypes predictable in the sense they don't depend on value. That is key -- I don't think casting should ever depend on value. Those wanting to maintain - say - float32 will need to cast scalars to float32. Maybe the use-cases motivating the scalar casting rules - maintaining float32 precision in particular - can be dealt with by careful casting of scalars, throwing the burden onto the memory-conscious to maintain their dtypes. IIRC this is how it worked back in the day (the Numeric day? -- and I'm pretty sure that in the long run it worked out badly. the core problem is that there are only python literals for a couple types, and it was oh so easy to do things like: my_arr = np,zeros(shape, dtype-float32) another_array = my_array * 4.0 and you'd suddenly get a float64 array. (of course, we already know all that..) I suppose this has the up side of being safe, and having scalar and array casting rules be the same is of course appealing, but you use a particular size dtype for a reason,and it's a real pain to maintain it. Casual users will use the defaults that match the Python types anyway. So in the in the spirit of practicality beats purity -- Id like accidental upcasting to be hard to do. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Do we want scalar casting to behave as it does at the moment?
On Thu, Jan 17, 2013 at 5:04 PM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: So in the in the spirit of practicality beats purity -- Id like accidental upcasting to be hard to do. and then: arr = arr + scalar would yield the same type as: arr += scalar so we buy some consistency! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Do we want scalar casting to behave as it does at the moment?
Hi, On Fri, Jan 18, 2013 at 1:04 AM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Thu, Jan 17, 2013 at 6:26 AM, Matthew Brett matthew.br...@gmail.com wrote: I am starting to wonder if we should aim for making * scalar and array casting rules the same; * Python int / float scalars become int32 / 64 or float64; aren't they already? I'm not sure what you are proposing. Sorry - yes that is what they are already, this sentence refers back to an earlier suggestion of mine on the thread, which I am discarding. This has the benefit of being very easy to understand and explain. It makes dtypes predictable in the sense they don't depend on value. That is key -- I don't think casting should ever depend on value. Those wanting to maintain - say - float32 will need to cast scalars to float32. Maybe the use-cases motivating the scalar casting rules - maintaining float32 precision in particular - can be dealt with by careful casting of scalars, throwing the burden onto the memory-conscious to maintain their dtypes. IIRC this is how it worked back in the day (the Numeric day? -- and I'm pretty sure that in the long run it worked out badly. the core problem is that there are only python literals for a couple types, and it was oh so easy to do things like: my_arr = np,zeros(shape, dtype-float32) another_array = my_array * 4.0 and you'd suddenly get a float64 array. (of course, we already know all that..) I suppose this has the up side of being safe, and having scalar and array casting rules be the same is of course appealing, but you use a particular size dtype for a reason,and it's a real pain to maintain it. Yes, I do understand that. The difference - as I understand it - is that back in the day, numeric did not have the the float32 etc scalars, so you could not do: another_array = my_array * np.float32(4.0) (please someone correct me if I'm wrong). Casual users will use the defaults that match the Python types anyway. I think what we are reading in this thread is that even experienced numpy users can find the scalar casting rules surprising, and that's a real problem, it seems to me. The person with a massive float32 array certainly should have the ability to control upcasting, but I think the default should be the least surprising thing, and that, it seems to me, is for the casting rules to be the same for arrays and scalars. In the very long term. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Casting Bug or a Feature?
2013/1/16 josef.p...@gmail.com: On Wed, Jan 16, 2013 at 10:43 PM, Patrick Marsh patrickmars...@gmail.com wrote: Thanks, everyone for chiming in. Now that I know this behavior exists, I can explicitly prevent it in my code. However, it would be nice if a warning or something was generated to alert users about the inconsistency between var += ... and var = var + ... Since I also got bitten by this recently in my code, I fully agree. I could live with an exception for lossy down casting in this case. About exceptions: someone mentioned in another thread about casting how having exceptions can make it difficult to write code. I've thought a bit more about this issue and I tend to agree, especially on code that used to work (in the sense of doing something -- not necessarily what you'd want -- without complaining). Don't get me wrong, when I write code I love when a library crashes and forces me to be more explicit about what I want, thus saving me the trouble of hunting down a tricky overflow / casting bug. However, in a production environment for instance, such an unexpected crash could have much worse consequences than an incorrect output. And although you may blame the programmer for not being careful enough about types, he couldn't expect it might crash the application back when this code was written Long story short, +1 for warning, -1 for exception, and +1 for a config flag that allows one to change to exceptions by default, if desired. -=- Olivier ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Do we want scalar casting to behave as it does at the moment?
I'd like to echo what Chris is saying. It was a big annoyance with Numeric to make it so hard to preserve the array type in ordinary expressions. Perry On Jan 17, 2013, at 8:04 PM, Chris Barker - NOAA Federal wrote: On Thu, Jan 17, 2013 at 6:26 AM, Matthew Brett matthew.br...@gmail.com wrote: I am starting to wonder if we should aim for making * scalar and array casting rules the same; * Python int / float scalars become int32 / 64 or float64; aren't they already? I'm not sure what you are proposing. This has the benefit of being very easy to understand and explain. It makes dtypes predictable in the sense they don't depend on value. That is key -- I don't think casting should ever depend on value. Those wanting to maintain - say - float32 will need to cast scalars to float32. Maybe the use-cases motivating the scalar casting rules - maintaining float32 precision in particular - can be dealt with by careful casting of scalars, throwing the burden onto the memory-conscious to maintain their dtypes. IIRC this is how it worked back in the day (the Numeric day? -- and I'm pretty sure that in the long run it worked out badly. the core problem is that there are only python literals for a couple types, and it was oh so easy to do things like: my_arr = np,zeros(shape, dtype-float32) another_array = my_array * 4.0 and you'd suddenly get a float64 array. (of course, we already know all that..) I suppose this has the up side of being safe, and having scalar and array casting rules be the same is of course appealing, but you use a particular size dtype for a reason,and it's a real pain to maintain it. Casual users will use the defaults that match the Python types anyway. So in the in the spirit of practicality beats purity -- Id like accidental upcasting to be hard to do. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Do we want scalar casting to behave as it does at the moment?
2013/1/17 Matthew Brett matthew.br...@gmail.com: Hi, On Fri, Jan 18, 2013 at 1:04 AM, Chris Barker - NOAA Federal chris.bar...@noaa.gov wrote: On Thu, Jan 17, 2013 at 6:26 AM, Matthew Brett matthew.br...@gmail.com wrote: I am starting to wonder if we should aim for making * scalar and array casting rules the same; * Python int / float scalars become int32 / 64 or float64; aren't they already? I'm not sure what you are proposing. Sorry - yes that is what they are already, this sentence refers back to an earlier suggestion of mine on the thread, which I am discarding. This has the benefit of being very easy to understand and explain. It makes dtypes predictable in the sense they don't depend on value. That is key -- I don't think casting should ever depend on value. Those wanting to maintain - say - float32 will need to cast scalars to float32. Maybe the use-cases motivating the scalar casting rules - maintaining float32 precision in particular - can be dealt with by careful casting of scalars, throwing the burden onto the memory-conscious to maintain their dtypes. IIRC this is how it worked back in the day (the Numeric day? -- and I'm pretty sure that in the long run it worked out badly. the core problem is that there are only python literals for a couple types, and it was oh so easy to do things like: my_arr = np,zeros(shape, dtype-float32) another_array = my_array * 4.0 and you'd suddenly get a float64 array. (of course, we already know all that..) I suppose this has the up side of being safe, and having scalar and array casting rules be the same is of course appealing, but you use a particular size dtype for a reason,and it's a real pain to maintain it. Yes, I do understand that. The difference - as I understand it - is that back in the day, numeric did not have the the float32 etc scalars, so you could not do: another_array = my_array * np.float32(4.0) (please someone correct me if I'm wrong). Casual users will use the defaults that match the Python types anyway. I think what we are reading in this thread is that even experienced numpy users can find the scalar casting rules surprising, and that's a real problem, it seems to me. The person with a massive float32 array certainly should have the ability to control upcasting, but I think the default should be the least surprising thing, and that, it seems to me, is for the casting rules to be the same for arrays and scalars. In the very long term. That would also be my preference, after banging my head against this problem for a while now, because it's simple and consistent. Since most of the related issues seem to come from integer arrays, a middle-ground may be the following: - Integer-type arrays get upcasted by scalars as in usual array / array operations. - Float/Complex-type arrays don't get upcasted by scalars except when the scalar is complex and the array is float. It makes the rule a bit more complex, but has the advantage of better preserving float types while getting rid of most issues related to integer overflows. -=- Olivier ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] New numpy functions: filled, filled_like
On Jan 17, 2013 8:01 PM, Olivier Delalleau sh...@keba.be wrote: 2013/1/17 Matthew Brett matthew.br...@gmail.com: Hi, On Thu, Jan 17, 2013 at 10:27 PM, Mark Wiebe mwwi...@gmail.com wrote: On Thu, Jan 17, 2013 at 2:10 PM, Benjamin Root ben.r...@ou.edu wrote: On Thu, Jan 17, 2013 at 5:04 PM, Eric Firing efir...@hawaii.edu wrote: On 2013/01/17 4:13 AM, Pierre Haessig wrote: Hi, Le 14/01/2013 20:05, Benjamin Root a écrit : I do like the way you are thinking in terms of the broadcasting semantics, but I wonder if that is a bit awkward. What I mean is, if one were to use broadcasting semantics for creating an array, wouldn't one have just simply used broadcasting anyway? The point of broadcasting is to _avoid_ the creation of unneeded arrays. But maybe I can be convinced with some examples. I feel that one of the point of the discussion is : although a new (or not so new...) function to create a filled array would be more elegant than the existing pair of functions np.zeros and np.ones, there are maybe not so many usecases for filled arrays *other than zeros values*. I can remember having initialized a non-zero array *some months ago*. For the anecdote it was a vector of discretized vehicule speed values which I wanted to be initialized with a predefined mean speed value prior to some optimization. In that usecase, I really didn't care about the performance of this initialization step. So my overall feeling after this thread is - *yes* a single dedicated fill/init/someverb function would give a slightly better API, - but *no* it's not important because np.empty and np.zeros covers 95 % usecases ! I agree with your summary and conclusion. Eric Can we at least have a np.nans() and np.infs() functions? This should cover an additional 4% of use-cases. Ben Root P.S. - I know they aren't verbs... Would it be too weird or clumsy to extend the empty and empty_like functions to do the filling? np.empty((10, 10), fill=np.nan) np.empty_like(my_arr, fill=np.nan) That sounds like a good idea to me. Someone wanting a fast way to fill an array will probably check out the 'empty' docstring first. See you, Matthew +1 from me. Even though it *is* weird to have both empty and fill ;) I'd almost prefer such a keyword be added to np.ones() to avoid that weirdness. (something like an array of ones where one equals X) Ray ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Casting Bug or a Feature?
On Thu, Jan 17, 2013 at 5:19 PM, Olivier Delalleau sh...@keba.be wrote: 2013/1/16 josef.p...@gmail.com: On Wed, Jan 16, 2013 at 10:43 PM, Patrick Marsh patrickmars...@gmail.com wrote: I could live with an exception for lossy down casting in this case. I'm not sure what the idea here is -- would you only get an exception if the value was such that the downcast would be lossy? If so, a major -1 The other option would be to always raise an exception if types would cause a downcast, i.e: arr = np.zeros(shape, dtype-uint8) arr2 = arr + 30 # this would raise an exception arr2 = arr + np.uint8(30) # you'd have to do this That sure would be clear and result if few errors of this type, but sure seems verbose and static language like to me. Long story short, +1 for warning, -1 for exception, and +1 for a config flag that allows one to change to exceptions by default, if desired. is this for value-dependent or any casting of this sort? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Fwd: numpy test fails with Illegal instruction'
On 17 January 2013 16:59, Gerhard Burger burger...@gmail.com wrote: Solved it, did a backtrace with gdb and the error came somewhere from an old lapack version that was installed on my machine (I thought I wouldn't have these issues in a virtualenv). but anyway after I removed it, and installed numpy again, it ran without problems! Virtualenv only creates an isolated Python install, it doesn't trick the Numpy build process into ignoring system libraries like LAPACK, ATLAS etc. Glad it's fixed. Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Do we want scalar casting to behave as it does at the moment?
On Thu, Jan 17, 2013 at 5:34 PM, Olivier Delalleau sh...@keba.be wrote: Yes, I do understand that. The difference - as I understand it - is that back in the day, numeric did not have the the float32 etc scalars, so you could not do: another_array = my_array * np.float32(4.0) (please someone correct me if I'm wrong). correct, it didn't have any scalars, but you could (and had to) still do something like: another_array = my_array * np.array(4.0, dtype=np.float32) a bit more verbose, but the verbosity wasn't the key issue -- it was doing anything special at all. Casual users will use the defaults that match the Python types anyway. I think what we are reading in this thread is that even experienced numpy users can find the scalar casting rules surprising, and that's a real problem, it seems to me. for sure -- but it's still relevant -- if you want non-default types, you need to understand the rules an be more careful. The person with a massive float32 array certainly should have the ability to control upcasting, but I think the default should be the least surprising thing, and that, it seems to me, is for the casting rules to be the same for arrays and scalars. In the very long term. A foolish consistency is the hobgoblin of little minds -- just kidding. But in all seriousness -- accidental upcasting really was a big old pain back in the day -- we are not making this up. We re using the term least surprising, but I now I was often surprised that I had lost my nice compact array. The user will need to think about it no matter how you slice it. Since most of the related issues seem to come from integer arrays, a middle-ground may be the following: - Integer-type arrays get upcasted by scalars as in usual array / array operations. - Float/Complex-type arrays don't get upcasted by scalars except when the scalar is complex and the array is float. I'm not sure that integer arrays are any more of an an issue, and having integer types and float typed behave differently is really asking for trouble! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
