Re: [Numpy-discussion] Default builds of OpenBLAS development branch are now fork safe

2014-03-28 Thread Nathaniel Smith
PL2?at=default > On Mar 28, 2014 7:34 PM, "Nathaniel Smith" wrote: > >> On 28 Mar 2014 20:26, "Robert Kern" wrote: >> > >> > It's only a problem in that the binary will not be BSD, and we do need >> to communicate that appropriately. It

Re: [Numpy-discussion] Default builds of OpenBLAS development branch are now fork safe

2014-03-28 Thread Nathaniel Smith
I thought OpenBLAS is usually used with reference lapack? On 28 Mar 2014 22:16, "Matthew Brett" wrote: > Hi, > > On Fri, Mar 28, 2014 at 1:28 PM, Sturla Molden > wrote: > > Nathaniel Smith wrote: > > > >> If the only problem with eigen turns out to b

Re: [Numpy-discussion] Dates and times and Datetime64 (again)

2014-03-29 Thread Nathaniel Smith
On Fri, Mar 28, 2014 at 9:30 PM, Sankarshan Mudkavi wrote: > > Hi Nathaniel, > > 1- You give as an example of "naive" datetime handling: > np.datetime64('2005-02-25T03:00Z') > np.datetime64('2005-02-25T03:00') > > This IIUC is incorrect. The Z modifier is a timezone offset, and for normal > "

Re: [Numpy-discussion] Dates and times and Datetime64 (again)

2014-03-29 Thread Nathaniel Smith
On 29 Mar 2014 20:57, "Chris Barker" wrote: > I think this is somewhat open for discussion -- yes, it's odd, but in the spirit of practicality beats purity, it seems OK. We could allow any TZ specifier for that matter -- that's kind of how "naive" or "local" timezone (non) handling works -- it's u

Re: [Numpy-discussion] Dates and times and Datetime64 (again)

2014-03-31 Thread Nathaniel Smith
On 31 Mar 2014 19:47, "Chris Barker" wrote: > > On Sat, Mar 29, 2014 at 3:08 PM, Nathaniel Smith wrote: >> >> On 29 Mar 2014 20:57, "Chris Barker" wrote: >> > I think this is somewhat open for discussion -- yes, it's odd, but in the spirit

Re: [Numpy-discussion] Confused by spec of numpy.linalg.solve

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 3:57 PM, Sebastian Berg wrote: > If `a` has exactly one dimension more then `b`, the first case is used. > Otherwise (..., M, K) is used instead. To make sure you always get the > expected result, it may be best to make sure that the number of > broadcasting (...) dimensions

Re: [Numpy-discussion] Dates and times and Datetime64 (again)

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 5:22 PM, Alexander Belopolsky wrote: > > On Tue, Apr 1, 2014 at 12:10 PM, Chris Barker wrote: >> >> It seems this committee of two has come to a consensus on naive -- and >> you're probably right, raise an exception if there is a time zone specifier. > > > Count me as +1 on

Re: [Numpy-discussion] ANN: NumPy 1.8.1 release

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 6:26 PM, Matthew Brett wrote: > I'm guessing that the LOAD_WITH_ALTERED_SEARCH_PATH means that a DLL loaded > via: > > hDLL = LoadLibraryEx(pathname, NULL, LOAD_WITH_ALTERED_SEARCH_PATH); > > will in turn (by default) search for its dependent DLLs in their own > directory.

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden wrote: > Haslwanter Thomas wrote: > >> Personally I cannot think of many applications where it would be desired >> to calculate the standard deviation with ddof=0. In addition, I feel that >> there should be consistency between standard modules such a

Re: [Numpy-discussion] Standard Deviation (std): Suggested change for "ddof" default value

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 9:51 PM, Ralf Gommers wrote: > > > > On Tue, Apr 1, 2014 at 10:08 PM, Nathaniel Smith wrote: >> >> On Tue, Apr 1, 2014 at 9:02 PM, Sturla Molden >> wrote: >> > Haslwanter Thomas wrote: >> > >> >> Pers

Re: [Numpy-discussion] Confused by spec of numpy.linalg.solve

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 9:50 PM, Sebastian Berg wrote: > On Di, 2014-04-01 at 16:25 +0100, Nathaniel Smith wrote: >> On Tue, Apr 1, 2014 at 3:57 PM, Sebastian Berg >> wrote: >> > If `a` has exactly one dimension more then `b`, the first case is used. >> > Otherwis

Re: [Numpy-discussion] ANN: NumPy 1.8.1 release

2014-04-01 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 11:58 PM, David Cournapeau wrote: > On Tue, Apr 1, 2014 at 6:43 PM, Nathaniel Smith wrote: >> >> On Tue, Apr 1, 2014 at 6:26 PM, Matthew Brett >> wrote: >> > I'm guessing that the LOAD_WITH_ALTERED_SEARCH_PATH means that a

Re: [Numpy-discussion] Resolving the associativity/precedence debate for @

2014-04-06 Thread Nathaniel Smith
On Tue, Apr 1, 2014 at 4:13 PM, Charles R Harris wrote: > > > > On Mon, Mar 24, 2014 at 6:33 PM, Nathaniel Smith wrote: >> >> On Mon, Mar 24, 2014 at 11:58 PM, Charles R Harris >> wrote: >> > On Mon, Mar 24, 2014 at 5:56 PM, Nathaniel Smith wrote: >

[Numpy-discussion] PEP 465 has been accepted / volunteers needed

2014-04-07 Thread Nathaniel Smith
Hey all, Guido just formally accepted PEP 465: https://mail.python.org/pipermail/python-dev/2014-April/133819.html http://legacy.python.org/dev/peps/pep-0465/#implementation-details Yay. The next step is to implement it, in CPython and in numpy. I have time to advise on this, but not to do i

Re: [Numpy-discussion] PEP 465 has been accepted / volunteers needed

2014-04-08 Thread Nathaniel Smith
On Tue, Apr 8, 2014 at 12:51 AM, Fernando Perez wrote: > On Mon, Apr 7, 2014 at 4:23 PM, Nathaniel Smith wrote: >> >> Hey all, >> >> Guido just formally accepted PEP 465: >> https://mail.python.org/pipermail/python-dev/2014-April/133819.html >> htt

[Numpy-discussion] Fwd: [Python-Dev] PEP 465: A dedicated infix operator for matrix multiplication

2014-04-10 Thread Nathaniel Smith
Hey all, Given the sometimes rocky history of collaboration between numerical Python and core Python, I thought it might be helpful to flag this posting for broader distribution -- it gives one perspective on how the core devs see things. (And is certainly consistent with my experience around PEP

[Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 12:21 PM, Carl Kleffner wrote: > a discussion about OpenBLAS on the octave maintainer list: > http://article.gmane.org/gmane.comp.gnu.octave.maintainers/38746 I'm getting the impression that OpenBLAS is being both a tantalizing opportunity and a practical thorn-in-the-side

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 6:05 PM, Sturla Molden wrote: > Sturla Molden wrote: > >> Making a totally new BLAS might seem like a crazy idea, but it might be the >> best solution in the long run. > > To see if this can be done, I'll try to re-implement cblas_dgemm and then > benchmark against MKL, Ac

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 7:53 PM, Julian Taylor wrote: > On 11.04.2014 18:03, Nathaniel Smith wrote: >> On Fri, Apr 11, 2014 at 12:21 PM, Carl Kleffner wrote: >>> a discussion about OpenBLAS on the octave maintainer list: >>> http://article.gmane.org/gmane.comp.g

Re: [Numpy-discussion] Dates and times and Datetime64 (again)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 11:25 PM, Sankarshan Mudkavi wrote: > So is the consensus that we don't accept any tags at all (not even > temporarily)? Would that break too much existing code? Well, we don't know. If anyone has any ideas on how to figure it out then they should speak up :-). Barring an

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 11:26 PM, Sturla Molden wrote: > On 11/04/14 20:47, Nathaniel Smith wrote: > >> Also, while Windows is maybe in the worst shape, all platforms would >> seriously benefit from the existence of a reliable speed-competitive >> binary-distribution-comp

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Sat, Apr 12, 2014 at 12:07 AM, Sturla Molden wrote: > On 12/04/14 00:39, Nathaniel Smith wrote: > >> The spawn mode is fine and all, but (a) the presence of something in >> 3.4 helps only a minority of users, (b) "spawn" is not a full >> replacement for fork;

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
On Fri, Apr 11, 2014 at 7:29 PM, Julian Taylor wrote: > x86 cpus are backward compatible with almost all instructions they ever > introduced, so one machine with the latest instruction set supported is > sufficient to test almost everything. > For that the runtime kernel selection must be tuneable

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-11 Thread Nathaniel Smith
Okay, I started taking notes here: https://github.com/numpy/numpy/wiki/BLAS-desiderata Please add as appropriate... -n On Sat, Apr 12, 2014 at 12:19 AM, Nathaniel Smith wrote: > On Fri, Apr 11, 2014 at 7:29 PM, Julian Taylor > wrote: >> x86 cpus are backward compatible wit

Re: [Numpy-discussion] Writing successful tests

2014-04-13 Thread Nathaniel Smith
On Mon, Apr 14, 2014 at 6:26 AM, wrote: > Greetings, > > I'm working on an additional function for numpy/lib/twodim_base.py. I'm > trying to add some tests for the new function, and > numpy/lib/tests/test_twodim_base.py seems like the right place for > them. > My problem is travis-ci tells me my

[Numpy-discussion] High-quality memory profiling for numpy in python 3.5 / volunteers needed

2014-04-15 Thread Nathaniel Smith
Hey all, The well known memory_profiler module [1] is super-useful, but has a fundamental limitation, which is the only way it can track allocations is by constantly polling the OS for the size of the total process address space. This is a crude and unreliable way of making measurements. In Pytho

Re: [Numpy-discussion] High-quality memory profiling for numpy in python 3.5 / volunteers needed

2014-04-15 Thread Nathaniel Smith
On Tue, Apr 15, 2014 at 12:06 PM, Julian Taylor wrote: >> Good news, though! python-dev is in favor of adding calloc() to the >> core allocation interfaces, which will let numpy join the party. See >> python-dev thread: >> https://mail.python.org/pipermail/python-dev/2014-April/133985.html >> >>

Re: [Numpy-discussion] High-quality memory profiling for numpy in python 3.5 / volunteers needed

2014-04-15 Thread Nathaniel Smith
On Tue, Apr 15, 2014 at 4:08 PM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > On Tue, Apr 15, 2014 at 3:07 PM, Nathaniel Smith wrote: >> On Tue, Apr 15, 2014 at 12:06 PM, Julian Taylor >> wrote: >>>> Good news, though! python-dev is in favor of

Re: [Numpy-discussion] Universal functions and introspection

2014-04-15 Thread Nathaniel Smith
I don't think the names exist currently except as mnemonics in the docstring. Patches to improve 'inspect' support would be welcome of course. -n On Tue, Apr 15, 2014 at 11:30 PM, Sylvain Corlay wrote: > Hi, > Unfortunately, numpy ufuncs are incompatible with the inspect module. > While ufunc.ni

Re: [Numpy-discussion] About the npz format

2014-04-16 Thread Nathaniel Smith
crc32 extremely fast, and I think zip might use adler32 instead which is even faster. OTOH compression is incredibly slow, unless you're using one of the 'just a little bit of compression' formats like blosc or lzo1. If your npz files are compressed then this is certainly the culprit. The zip form

Re: [Numpy-discussion] About the npz format

2014-04-17 Thread Nathaniel Smith
On 17 Apr 2014 01:57, "onefire" wrote: > > What I cannot understand is why savez takes more than 10 times longer than saving the data to a npy file. The only reason that I could come up with was the computation of the crc32. We can all make guesses but the solution is just to profile it :-). %pru

Re: [Numpy-discussion] High-quality memory profiling for numpy in python 3.5 / volunteers needed

2014-04-17 Thread Nathaniel Smith
On Wed, Apr 16, 2014 at 4:17 PM, R Hattersley wrote: > For some reason the Python issue 21223 didn't show any activity until I > logged in to post my patch. At which point I saw that haypo had already > submitted pretty much exactly the same patch. *sigh* That was pretty much a > waste of time the

Re: [Numpy-discussion] High-quality memory profiling for numpy in python 3.5 / volunteers needed

2014-04-17 Thread Nathaniel Smith
On 17 Apr 2014 15:09, "Aron Ahmadia" wrote: > > > On the one hand it would be nice to actually know whether posix_memalign is important, before making api decisions on this basis. > > FWIW: On the lightweight IBM cores that the extremely popular BlueGene machines were based on, accessing unaligned

Re: [Numpy-discussion] Slightly off-topic - accuracy of C exp function?

2014-04-23 Thread Nathaniel Smith
On Wed, Apr 23, 2014 at 6:22 AM, Matthew Brett wrote: > Hi, > > I'm exploring Mingw-w64 for numpy building, and I've found it gives a > slightly different answer for 'exp' than - say - gcc on OSX. > > The difference is of the order of the eps value for the output number > (2 * eps for a result of

Re: [Numpy-discussion] 64-bit windows numpy / scipy wheels for testing

2014-04-24 Thread Nathaniel Smith
On Fri, Apr 25, 2014 at 12:00 AM, Charles R Harris wrote: > Cool. After all these long years... Now all we need is a box running tests > for CI. There is http://www.appveyor.com/ though I haven't tried doing anything with it yet... (yes it says ".NET" at the top, but then at the bottom it says

Re: [Numpy-discussion] should rint return int?

2014-04-28 Thread Nathaniel Smith
On 28 Apr 2014 20:22, "Robert Kern" wrote: > C's rint() does not: > > http://linux.die.net/man/3/rint > > This is because there are many integers that are representable as > floats/doubles/long doubles that are well outside of the range of any > C integer type, e.g. 1e20. By the time you have a

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn wrote: > > Am 11 Apr 2014 um 19:05 schrieb Sturla Molden : > >> Sturla Molden wrote: >> >>> Making a totally new BLAS might seem like a crazy idea, but it might be the >>> best solution in the long run. >> >> To see if this can be done, I'll try to r

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Tue, Apr 29, 2014 at 12:52 AM, Sturla Molden wrote: > On 29/04/14 01:30, Nathaniel Smith wrote: > >> I finally read this paper: >> >> http://www.cs.utexas.edu/users/flame/pubs/blis2_toms_rev2.pdf >> >> and I have to say that I'm no longer so convinc

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Tue, Apr 29, 2014 at 1:10 AM, Julian Taylor wrote: > On 29.04.2014 02:05, Matthew Brett wrote: >> Hi, >> >> On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith wrote: >>> It would be really interesting if someone were to try hacking simple >>> runtime CPU

Re: [Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)

2014-04-28 Thread Nathaniel Smith
On Tue, Apr 29, 2014 at 1:05 AM, Matthew Brett wrote: > Hi, > > On Mon, Apr 28, 2014 at 4:30 PM, Nathaniel Smith wrote: >> On Mon, Apr 28, 2014 at 11:25 AM, Michael Lehn >> wrote: >>> >>> Am 11 Apr 2014 um 19:05 schrieb Sturla Molden : >>> >&g

Re: [Numpy-discussion] IDL vs Python parallel computing

2014-05-07 Thread Nathaniel Smith
On Wed, May 7, 2014 at 7:11 PM, Sturla Molden wrote: > On 03/05/14 23:56, Siegfried Gonzi wrote: > > I noticed IDL uses at least 400% (4 processors or cores) out of the box > > for simple things like reading and processing files, calculating the > > mean etc. > > The DMA controller is working a

Re: [Numpy-discussion] smoothing function

2014-05-15 Thread Nathaniel Smith
On Thu, May 15, 2014 at 1:04 PM, rodrigo koblitz wrote: > Buenos, > I'm reading Zuur book (ecology models with R) and try make it entire in > python. > Have this function in R: > M4 <- gam(So ∼ s(De) + factor(ID), subset = I1) > > the 's' term indicated with So is modelled as a smoothing function

Re: [Numpy-discussion] NumPy C API question

2014-05-21 Thread Nathaniel Smith
Hi Stefan, Allocating a new PyArrayObject isn't terribly expensive (compared to all the other allocations that Python programs are constantly doing), but I'm afraid you have a more fundamental problem. The reason there is no supported API to change the storage pointer of a PyArrayObject is that th

Re: [Numpy-discussion] NumPy C API question

2014-05-21 Thread Nathaniel Smith
Hi Stefan, One possibility that comes to mind: you may want in any case some way to temporarily "pin" an object's memory in place (e.g., to prevent one thread trying to migrate it while some other thread is working on it). If so then the Python wrapper could acquire a pin when the ndarray is alloc

Re: [Numpy-discussion] [pydata] Re: ANN: Pandas 0.14.0 released

2014-05-30 Thread Nathaniel Smith
I sometimes do pip install pandas==0.14.0 This requires you know the version number, but is still much easier than the arcane mutterings that are otherwise needed if you want to be fully correct (pull in new dependencies, etc.). -n On 30 May 2014 23:31, "Matthew Brett" wrote: > Hi, > > On Fr

Re: [Numpy-discussion] [pydata] Re: ANN: Pandas 0.14.0 released

2014-05-30 Thread Nathaniel Smith
If you really want to use complicated command line switches I think the correct ones are: pip install -U --no-deps pandas pip install pandas (Yes, you have to run both commands in order to handle all cases correctly.) -n On 30 May 2014 23:54, "Matthew Brett" wrote: > Hi, > > On Fri, May 30, 20

Re: [Numpy-discussion] SciPy 2014 BoF NumPy Participation

2014-06-03 Thread Nathaniel Smith
On Wed, Jun 4, 2014 at 12:33 AM, Charles R Harris wrote: > On Tue, Jun 3, 2014 at 5:08 PM, Kyle Mandli wrote: >> >> Hello everyone, >> >> As one of the co-chairs in charge of organizing the birds-of-a-feather >> sesssions at the SciPy conference this year, I wanted to solicit through the >> NumPy

[Numpy-discussion] big-bangs versus incremental improvements (was: Re: SciPy 2014 BoF NumPy Participation)

2014-06-04 Thread Nathaniel Smith
On Wed, Jun 4, 2014 at 7:18 AM, Travis Oliphant wrote: > Even relatively simple changes can have significant impact at this point. > Nathaniel has laid out a fantastic list of great features. These are the > kind of features I have been eager to see as well. This is why I have been > working to

Re: [Numpy-discussion] big-bangs versus incremental improvements (was: Re: SciPy 2014 BoF NumPy Participation)

2014-06-05 Thread Nathaniel Smith
On Thu, Jun 5, 2014 at 2:29 AM, Travis Oliphant wrote: > At some point there *will* be a NumPy 2.0. What features go into NumPy > 2.0, how much backward compatibility is provided, and how much porting is > needed to move your code from NumPy 1.X to NumPy 2.X is the real user > question --- not w

Re: [Numpy-discussion] big-bangs versus incremental improvements (was: Re: SciPy 2014 BoF NumPy Participation)

2014-06-05 Thread Nathaniel Smith
On Thu, Jun 5, 2014 at 3:36 AM, Charles R Harris wrote: > @nathaniel IIRC, one of the objections to the missing values work was that > it changed the underlying array object by adding a couple of variables to > the structure. I'm willing to do that sort of thing, but it would be good to > have gen

Re: [Numpy-discussion] big-bangs versus incremental improvements (was: Re: SciPy 2014 BoF NumPy Participation)

2014-06-05 Thread Nathaniel Smith
On Thu, Jun 5, 2014 at 3:24 PM, David Cournapeau wrote: > On Thu, Jun 5, 2014 at 2:51 PM, Charles R Harris > wrote: >> On Thu, Jun 5, 2014 at 6:40 AM, David Cournapeau >> wrote: >>> IMO, what is needed the most is refactoring the internal to extract the >>> Python C API low level from the rest o

Re: [Numpy-discussion] ANN: NumPy 1.9.0 beta release

2014-06-10 Thread Nathaniel Smith
On 10 Jun 2014 09:44, "Sebastian Berg" wrote: > The other error looks a bit different because of the nonzero logic, but > probably is the same, i.e. also boolean indexing. The last one is the > change that `arr[[1,2,3,4]] = [1,2]` does not work anymore. A workaround > (maybe also for the rest poss

Re: [Numpy-discussion] ANN: NumPy 1.9.0 beta release

2014-06-10 Thread Nathaniel Smith
On 10 Jun 2014 11:15, "Sebastian Berg" wrote: > > On Di, 2014-06-10 at 10:50 +0100, Nathaniel Smith wrote: > > On 10 Jun 2014 09:44, "Sebastian Berg" > > wrote: > > > The other error looks a bit different because of the nonzero logic, > &

Re: [Numpy-discussion] genfromtxt universal newline support

2014-06-30 Thread Nathaniel Smith
On Mon, Jun 30, 2014 at 12:33 PM, Julian Taylor wrote: > genfromtxt and loadtxt need an almost full rewrite to fix the botched > python3 conversion of these functions. There are a couple threads > about this on this list already. > There are numerous PRs fixing stuff in these functions which I > c

Re: [Numpy-discussion] genfromtxt universal newline support

2014-06-30 Thread Nathaniel Smith
On Mon, Jun 30, 2014 at 3:47 PM, Derek Homeier wrote: > Does it make sense to keep maintaing both functions at all? IIRC the idea that > loadtxt would be the faster version of the two has been discarded long ago, > thus it seems there is very little, if anything, loadtxt can do that cannot > be d

Re: [Numpy-discussion] genfromtxt universal newline support

2014-06-30 Thread Nathaniel Smith
On 30 Jun 2014 17:05, "Chris Barker" wrote: >> >> It's also an interesting >> question whether they've fixed the unicode/binary issues, > > > Which brings up the "how do we handle text/strings in numpy? issue. We had a good thread going here about what the 'S' data type should be , what with py3 a

[Numpy-discussion] Fwd: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments

2014-07-02 Thread Nathaniel Smith
There's some discussion on python-ideas about making it possible for python indexing to accept kwargs, eg arr[1:2, foo=bar] Since numpy is a very heavy user of indexing which might benefit from this, I thought I should forward it here. If we have clear use cases for such a feature then that ma

Re: [Numpy-discussion] Accessing irregular sized array data from C

2014-07-02 Thread Nathaniel Smith
On 2 Jul 2014 20:12, "Sturla Molden" wrote: > > Julian Taylor wrote: > > > another thing, don't use int as the index to the array, use npy_intp > > which is large enough to also index arrays > 4GB if the platform > > supports it. > > With double* a 32-bit int can index 16 GB, a 32-bit unsigned in

Re: [Numpy-discussion] Questions about fixes for 1.9.0rc2

2014-07-04 Thread Nathaniel Smith
On Fri, Jul 4, 2014 at 9:02 PM, Ralf Gommers wrote: > > On Fri, Jul 4, 2014 at 10:00 PM, Charles R Harris > wrote: >> >> On Fri, Jul 4, 2014 at 1:42 PM, Charles R Harris >> wrote: >>> >>> Sebastian Seberg has fixed one class of test failures due to the indexing >>> changes in numpy 1.9.0b1. The

Re: [Numpy-discussion] Questions about fixes for 1.9.0rc2

2014-07-04 Thread Nathaniel Smith
On Fri, Jul 4, 2014 at 9:33 PM, Charles R Harris wrote: > > On Fri, Jul 4, 2014 at 2:09 PM, Nathaniel Smith wrote: >> >> On Fri, Jul 4, 2014 at 9:02 PM, Ralf Gommers >> wrote: >> > >> > On Fri, Jul 4, 2014 at 10:00 PM, Charles R Harris >> >

Re: [Numpy-discussion] Questions about fixes for 1.9.0rc2

2014-07-04 Thread Nathaniel Smith
On Fri, Jul 4, 2014 at 9:48 PM, Charles R Harris wrote: > > On Fri, Jul 4, 2014 at 2:41 PM, Nathaniel Smith wrote: >> >> On Fri, Jul 4, 2014 at 9:33 PM, Charles R Harris >> wrote: >> > >> > On Fri, Jul 4, 2014 at 2:09 PM, Nathaniel Smith wrote: >&g

Re: [Numpy-discussion] Questions about fixes for 1.9.0rc2

2014-07-04 Thread Nathaniel Smith
On Fri, Jul 4, 2014 at 10:31 PM, Charles R Harris wrote: > > On Fri, Jul 4, 2014 at 3:15 PM, Nathaniel Smith wrote: >> >> On Fri, Jul 4, 2014 at 9:48 PM, Charles R Harris >> wrote: >> > >> > On Fri, Jul 4, 2014 at 2:41 PM, Nathaniel Smith wrote:

Re: [Numpy-discussion] Questions about fixes for 1.9.0rc2

2014-07-04 Thread Nathaniel Smith
On 5 Jul 2014 00:07, "Charles R Harris" wrote: > > > > > On Fri, Jul 4, 2014 at 3:33 PM, Nathaniel Smith wrote: >> >> On Fri, Jul 4, 2014 at 10:31 PM, Charles R Harris >> wrote: >> > >> > On Fri, Jul 4, 2014 at 3:15 PM, Nathaniel Smit

Re: [Numpy-discussion] Remove bento from numpy

2014-07-05 Thread Nathaniel Smith
On 5 Jul 2014 09:23, "Ralf Gommers" wrote: > > On Sat, Jul 5, 2014 at 10:13 AM, David Cournapeau wrote: >> >> On Sat, Jul 5, 2014 at 11:25 AM, Charles R Harris < charlesr.har...@gmail.com> wrote: >>> >>> Ralf likes the speed of bento, but it is not currently maintained >> >> >> What exactly is no

Re: [Numpy-discussion] Remove bento from numpy

2014-07-05 Thread Nathaniel Smith
On Sat, Jul 5, 2014 at 2:32 PM, Ralf Gommers wrote: > > On Sat, Jul 5, 2014 at 1:54 PM, Nathaniel Smith wrote: >> >> On 5 Jul 2014 09:23, "Ralf Gommers" wrote: >> > >> > On Sat, Jul 5, 2014 at 10:13 AM, David Cournapeau >> > wrote: >&g

Re: [Numpy-discussion] Remove bento from numpy

2014-07-05 Thread Nathaniel Smith
On Sat, Jul 5, 2014 at 3:21 PM, David Cournapeau wrote: > > On Sat, Jul 5, 2014 at 11:17 PM, Nathaniel Smith wrote: >> >> Maybe bento will revive and take over the new python packaging world! >> Maybe not. Maybe something else will. I don't see how our support for &

Re: [Numpy-discussion] Short-hand array creation in `numpy.mat` style

2014-07-06 Thread Nathaniel Smith
On Sun, Jul 6, 2014 at 9:35 PM, Daniel da Silva wrote: > The idea is that there be a short-hand for creating arrays as there is for > matrices: > > np.mat('.2 .7 .1; .3 .5 .2; .1 .1 .9') > > It was suggested in GitHub issue #4817 in light that it would be beneficial > to beginners and to present

Re: [Numpy-discussion] indexed assignment testcases

2014-07-06 Thread Nathaniel Smith
On Sun, Jul 6, 2014 at 9:14 PM, Benjamin Root wrote: > as for the broadcasting issue, I can see it for the second case, but the > first case still doesn't sit right with me. My understanding of broadcasting > is to effectively *expand* an array to match the shape of another array (or > some target

Re: [Numpy-discussion] 1.10-devel is open

2014-07-07 Thread Nathaniel Smith
On 7 Jul 2014 14:12, "Charles R Harris" wrote:. > > Yes, what I did was like one big cherry-pick. But I think we end up in the same place with two divergent branches. I think git history is just a string of changesets and each changeset has a hash. Same hash, same changeset, and I think that was p

Re: [Numpy-discussion] Short-hand array creation in `numpy.mat` style

2014-07-07 Thread Nathaniel Smith
On Mon, Jul 7, 2014 at 3:28 PM, Sebastian Berg wrote: > On Mo, 2014-07-07 at 09:50 -0400, josef.p...@gmail.com wrote: >> >> On Mon, Jul 7, 2014 at 9:11 AM, Sebastian Berg >> wrote: >> On Mo, 2014-07-07 at 08:25 -0400, Alan G Isaac wrote: >> > On 7/7/2014 7:17 AM, Daπid wrote: >>

Re: [Numpy-discussion] np.zeros of structured array of array of objects

2014-07-12 Thread Nathaniel Smith
On Fri, Jul 11, 2014 at 8:39 AM, Julian Taylor wrote: > Hi, > looking at https://github.com/numpy/numpy/issues/4857 I noticed that > np.zeros of a structured array of array of objects only initializes the > first element of if the embedded array to zero and leaves the rest None: > > In [1]: a = nu

Re: [Numpy-discussion] String type again.

2014-07-13 Thread Nathaniel Smith
On 12 Jul 2014 23:06, "Charles R Harris" wrote: > > As previous posts have pointed out, Numpy's `S` type is currently treated as a byte string, which leads to more complicated code in python3. OTOH, the unicode type is stored as UCS4, which consumes a lot of space, especially for ascii strings. Th

Re: [Numpy-discussion] Short-hand array creation in `numpy.mat` style

2014-07-15 Thread Nathaniel Smith
On Sun, Jul 13, 2014 at 6:31 PM, Alexander Belopolsky wrote: > Also, the use of strings will confuse most syntax highlighters. Compare > the two options in this screenshot: > > [image: Inline image 2] > I guess this is a minor issue for "real" code, but even IPython doesn't (yet?) provide synta

Re: [Numpy-discussion] __numpy_ufunc__

2014-07-16 Thread Nathaniel Smith
Weirdly, I never received Chuck's original email in this thread. Should some list admin be informed? I also am not sure what/where Julian's comments were, so I second the call for context :-). Putting it off until 1.10 doesn't seem like an obviously bad idea to me, but specifics would help... (__

Re: [Numpy-discussion] `allclose` vs `assert_allclose`

2014-07-16 Thread Nathaniel Smith
On 16 Jul 2014 10:26, "Tony Yu" wrote: > > Is there any reason why the defaults for `allclose` and `assert_allclose` differ? This makes debugging a broken test much more difficult. More importantly, using an absolute tolerance of 0 causes failures for some common cases. For example, if two values

[Numpy-discussion] Mailing list slowdown (was Re: __numpy_ufunc__)

2014-07-17 Thread Nathaniel Smith
On 17 Jul 2014 11:51, "Sebastian Berg" wrote: > > On Mi, 2014-07-16 at 09:07 +0100, Nathaniel Smith wrote: > > Weirdly, I never received Chuck's original email in this thread. > > Should some list admin be informed? > > > > I send some mails yesterda

Re: [Numpy-discussion] [SciPy-Dev] __numpy_ufunc__ and 1.9 release

2014-07-17 Thread Nathaniel Smith
On Tue, Jul 15, 2014 at 7:06 PM, Julian Taylor wrote: > hi, > as you may know we want to release numpy 1.9 soon. We should have solved > most indexing regressions the first beta showed. > > The remaining blockers are finishing the new __numpy_ufunc__ feature. > This feature should allow for altern

Re: [Numpy-discussion] String type again.

2014-07-18 Thread Nathaniel Smith
On Tue, Jul 15, 2014 at 7:40 PM, Aldcroft, Thomas wrote: > > On Sat, Jul 12, 2014 at 8:02 PM, Nathaniel Smith wrote: >> >> OTOH, fixed length nul padded latin1 would be useful for various flat file >> reading tasks. > > As one of the original agitators for this, l

Re: [Numpy-discussion] `allclose` vs `assert_allclose`

2014-07-18 Thread Nathaniel Smith
On Wed, Jul 16, 2014 at 7:47 PM, Ralf Gommers wrote: > > On Wed, Jul 16, 2014 at 6:37 AM, Tony Yu wrote: >> It seems like the defaults for `allclose` and `assert_allclose` should >> match, and an absolute tolerance of 0 is probably not ideal. I guess this is >> a pretty big behavioral change, bu

Re: [Numpy-discussion] String type again.

2014-07-18 Thread Nathaniel Smith
On Thu, Jul 17, 2014 at 10:05 PM, Chris Barker wrote: > A bit of a higher-level view of the issues at hand. > > Python has three relevant data types: > > A unicode type (unicode in py2, str in py3) > A one-byte-per-char stringtype (py2 string) > A bytes type > > The big problem is that py2 only ha

Re: [Numpy-discussion] `allclose` vs `assert_allclose`

2014-07-18 Thread Nathaniel Smith
On Thu, Jul 17, 2014 at 9:07 PM, wrote: > On Wed, Jul 16, 2014 at 9:52 AM, Nathaniel Smith wrote: >> What you say makes sense to me, and loosening the default tolerances won't >> break any existing tests. (And I'm not too worried about people who were >> counting

Re: [Numpy-discussion] Short-hand array creation in `numpy.mat` style

2014-07-18 Thread Nathaniel Smith
On Thu, Jul 17, 2014 at 11:10 PM, Charles G. Waldman wrote: > > -1 on the 'arr' name. I think if we're going to support this function at all > (which I'm not convinced is a good idea), it should be np.fromsomething like > the other from* functions. > > Maybe frommatlab? > > I think that 'arr' i

Re: [Numpy-discussion] String type again.

2014-07-18 Thread Nathaniel Smith
On Tue, Jul 15, 2014 at 4:29 PM, Charles R Harris wrote: > Thinking more about it, the easiest thing to do might be to make the S dtype > a UTF-8 encoding. Most of the machinery to deal with that is already in > place. That change might affect some users though, and we might need to do > some work

Re: [Numpy-discussion] proposal: new commit guidelines for backportable bugfixes

2014-07-18 Thread Nathaniel Smith
On 18 Jul 2014 15:36, "Julian Taylor" wrote: > > git rebase --onto $(git merge-base master maintenance/1.9.x) HEAD^ As a potential refinement, this might be simpler if we define a branch that points to this commit. -n ___ NumPy-Discussion mailing list

Re: [Numpy-discussion] `allclose` vs `assert_allclose`

2014-07-18 Thread Nathaniel Smith
On Fri, Jul 18, 2014 at 12:38 PM, wrote: > > On Thu, Jul 17, 2014 at 4:07 PM, wrote: > >> If you mean by this to add atol=1e-8 as default, then I'm against it. >> >> At least it will change the meaning of many of our tests in statsmodels. >> >> I'm using rtol to check for correct 1e-15 or 1e-30,

Re: [Numpy-discussion] String type again.

2014-07-18 Thread Nathaniel Smith
On Fri, Jul 18, 2014 at 5:54 PM, Chris Barker wrote: > > This is why I see no downside to latin-1 -- if you don't use the > 127 code > points, it's the same thing -- if you do, you get some extra handy > characters. The only difference is that a proper ascii type would not let > you store anything

Re: [Numpy-discussion] Short-hand array creation in `numpy.mat` style

2014-07-18 Thread Nathaniel Smith
On Fri, Jul 18, 2014 at 3:02 PM, Charles G. Waldman wrote: > I greatly prefer "np.mat" to "np.arr" for this, FWIW Unfortunately that's already taken... -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___

Re: [Numpy-discussion] `allclose` vs `assert_allclose`

2014-07-18 Thread Nathaniel Smith
On Fri, Jul 18, 2014 at 7:03 PM, wrote: > > On Fri, Jul 18, 2014 at 12:53 PM, Nathaniel Smith wrote: >> >> For cases like this, you need *some* non-zero atol or the thing just >> doesn't work, and one could quibble over the exact value as long as >> it&#x

Re: [Numpy-discussion] `allclose` vs `assert_allclose`

2014-07-18 Thread Nathaniel Smith
On 18 Jul 2014 19:31, wrote: >> >> >> > Making the behavior of assert_allclose depending on whether desired is >> > exactly zero or 1e-20 looks too difficult to remember, and which desired I >> > use would depend on what I get out of R or Stata. >> >> I thought your whole point here was that 1e-20

Re: [Numpy-discussion] Short-hand array creation in `numpy.mat` style

2014-07-18 Thread Nathaniel Smith
On 18 Jul 2014 18:06, "Alan G Isaac" wrote: > > On 7/18/2014 12:45 PM, Mark Miller wrote: > > If the true goal is to just allow quick entry of a 2d array, why not just advocate using > > a = numpy.array(numpy.mat("1 2 3; 4 5 6; 7 8 9")) > > > It's even simpler: > a = np.mat(' 1 2 3;4 5 6;7 8 9').A

Re: [Numpy-discussion] proposal: new commit guidelines for backportable bugfixes

2014-07-18 Thread Nathaniel Smith
On Fri, Jul 18, 2014 at 11:44 PM, Pauli Virtanen wrote: > 18.07.2014 23:53, Julian Taylor kirjoitti: >> On 18.07.2014 19:47, Pauli Virtanen wrote: > [clip] >> > The other well-known alternative to bugfixes is to first commit it in >> > the earliest maintenance branch where you want to have it, and

Re: [Numpy-discussion] change default integer from int32 to int64 on win64?

2014-07-23 Thread Nathaniel Smith
On Wed, Jul 23, 2014 at 9:57 PM, Robert Kern wrote: > That's perhaps what you want, but numpy has never claimed to do this. > The numpy project deliberately chose (and is so documented) to make > its default integer type a C long, not a C size_t, to match Python's > default. This is true, but it'

Re: [Numpy-discussion] numpy.mean still broken for largefloat32arrays

2014-07-25 Thread Nathaniel Smith
On Fri, Jul 25, 2014 at 5:56 PM, RayS wrote: > The important point was that it would be best if all of the methods affected > by summing 32 bit floats with 32 bit accumulators had the same Notes as > numpy.mean(). We went through a lot of code yesterday, assuming that any > numpy or Scipy.stats fu

Re: [Numpy-discussion] numpy.mean still broken for largefloat32arrays

2014-07-27 Thread Nathaniel Smith
On Sun, Jul 27, 2014 at 3:16 PM, RayS wrote: > At 02:04 AM 7/27/2014, you wrote: > >>You won't be able to do it by accident or omission or a lack of >>discipline. It's not a tempting public target like, say, np.seterr(). > > BTW, why not throw an overflow error in the large float32 sum() case? > I

Re: [Numpy-discussion] last call for numpy 1.8.2 bugfixes

2014-08-04 Thread Nathaniel Smith
On Mon, Aug 4, 2014 at 11:12 PM, Julian Taylor wrote: > On 05.08.2014 00:09, Matthew Brett wrote: >> Hi, >> >> On Mon, Aug 4, 2014 at 3:05 PM, Julian Taylor >> wrote: >>> hi, >>> as numpy 1.9 is going to be a relative hard upgrade as indexing changes >>> expose a couple bugs in third party packag

Re: [Numpy-discussion] Array2 subset of array1

2014-08-05 Thread Nathaniel Smith
On Tue, Aug 5, 2014 at 1:58 PM, Jurgens de Bruin wrote: > Hi, > > I am new to numpy so any help would be greatly appreciated. > > I have two arrays: > > array1 = np.arange(1,100+1) > array2 = np.arange(1,50+1) > > How can I calculate/determine if array2 is a subset of array1 (falls within

Re: [Numpy-discussion] Preliminary thoughts on implementing __matmul__

2014-08-06 Thread Nathaniel Smith
On Wed, Aug 6, 2014 at 2:19 AM, Charles R Harris wrote: > Hi All, > > I've been looking to implement the "@" operator from Python 3.5. Looking at > the current implementation of the dot function, it only uses a vector inner > product, which is either that defined in arraytypes.c.src or a version u

Re: [Numpy-discussion] Preliminary thoughts on implementing __matmul__

2014-08-06 Thread Nathaniel Smith
On Wed, Aug 6, 2014 at 4:32 PM, Charles R Harris wrote: > Should also mention that we don't have the ability to operate on stacked > vectors because they can't be identified by dimension info. One workaround > is to add dummy dimensions where needed, another is to add two flags, row > and col, and

Re: [Numpy-discussion] Preliminary thoughts on implementing __matmul__

2014-08-06 Thread Nathaniel Smith
On Thu, Aug 7, 2014 at 12:24 AM, Charles R Harris wrote: > > On Wed, Aug 6, 2014 at 4:57 PM, Nathaniel Smith wrote: >> >> On Wed, Aug 6, 2014 at 4:32 PM, Charles R Harris >> wrote: >> > Should also mention that we don't have the ability to operate on stac

Re: [Numpy-discussion] Preliminary thoughts on implementing __matmul__

2014-08-06 Thread Nathaniel Smith
On 7 Aug 2014 00:41, "Charles R Harris" wrote: > > On Wed, Aug 6, 2014 at 5:33 PM, Nathaniel Smith wrote: >> >> On Thu, Aug 7, 2014 at 12:24 AM, Charles R Harris >> wrote: >> > >> > On Wed, Aug 6, 2014 at 4:57 PM, Nathaniel Smith wrote: &

<    3   4   5   6   7   8   9   10   11   12   >