[Numpy-discussion] Debian/Ubuntu patch help (was: ANN: NumPy 1.6.2 release candidate 1)

> Hi, if there's anyone wants to have a look at the above issue this > week, >that would be great. > If there's a patch by this weekend I can create a second RC, so we can > still have the final release before the end of this month (needed for > Debian freeze). Otherwise a second RC won't be neede

Re: [Numpy-discussion] Debian/Ubuntu patch help

On 05/16/2012 09:01 PM, Ralf Gommers wrote: > > > On Tue, May 15, 2012 at 10:35 PM, Julian Taylor > mailto:jtaylor.deb...@googlemail.com>> > wrote: > > > Hi, if there's anyone wants to have a look at the above issue this > > week, > &

[Numpy-discussion] experiments with SSE vectorization

Hi, I have been experimenting a bit with how applicable SSE vectorization is to NumPy. In principle the core of NumPy mostly deals with memory bound operations, but it turns out on modern machines with large caches you can still get decent speed ups. The experiments are available on this fork: htt

[Numpy-discussion] faster (selection based) median, 2013 edition

could still add median of medians for better worst case performance. If no blockers appear I want to fix this up and file a pull request to have this in numpy 1.8. Guidance on details of implementation in numpys C api is highly appreciated, its the first time I'm dealing with it. Cheers, J

Re: [Numpy-discussion] faster (selection based) median, 2013 edition

On 18.05.2013 08:12, Julian Taylor wrote: > hi, > > once again I want to bring up the median algorithm which is implemented > in terms of sorting in numpy. > median (and percentile and a couple more functions) can be more > efficiently implemented in terms of a select

Re: [Numpy-discussion] quickselect

On 05/29/2013 06:12 AM, josef.p...@gmail.com wrote: > On Tue, May 28, 2013 at 6:31 PM, Charles R Harris > wrote: >> Hi All, >> >> There is a PR adding quickselect to numpy as a function `partition`. >> Comments on name and exposure in the numpy API are welcome. > > I think the name is fine. It's p

Re: [Numpy-discussion] suggested change of behavior for interp

On 05.06.2013 16:33, Nathaniel Smith wrote: > On Wed, Jun 5, 2013 at 3:16 PM, Slavin, Jonathan > wrote: >> The simplest monotonicity test that I've seen is: >> >> dx = np.diff(x) >> monotonic = np.all(dx < 0.) or np.all(dx > 0.) >> >> I expect that this is pretty fast, though I haven't tested it y

Re: [Numpy-discussion] quickselect

On 09.06.2013 12:10, josef.p...@gmail.com wrote: > On Wed, May 29, 2013 at 3:19 PM, wrote: >> On Wed, May 29, 2013 at 12:25 PM, Julian Taylor >> wrote: >>> On 05/29/2013 06:12 AM, josef.p...@gmail.com wrote: >>>> >>>> On Tue, May 28, 2013 at 6:31

Re: [Numpy-discussion] quickselect

On 11.06.2013 14:37, Jonathan J. Helmus wrote: > > > Julian, > > > Since I am the author of the current percentile PR > (https://github.com/numpy/numpy/pull/2970), I'm willing to try > reimplementing percentile with the new partition functionality. I > don't expect to have time to do this until

Re: [Numpy-discussion] svd + multiprocessing hangs

On 12.06.2013 18:07, Uwe Schmitt wrote: > Dear all, > > the following code hangs on my Ubuntu machine. > I use self compiled numpy 1.7.1 and Python > 2.7.3 > > - > > import numpy > import numpy.linalg > import multiprocessing > > def classify(): > X = numpy.random.random((10,3)) > p

[Numpy-discussion] strided copy unroll, benchmarks needed

hi, I posted a pull with a minor change instructing the GCC compiler to unroll the strided copy loops (gcc will almost never do that on its own, not even on O3). https://github.com/numpy/numpy/pull/3429 It improves performance of these copies by 20%-50% depending on the size of the data copied (if

Re: [Numpy-discussion] Seg. fault when running tests

On 15.06.2013 21:12, Charles R Harris wrote: > > > On Sat, Jun 15, 2013 at 9:50 AM, Warren Weckesser > mailto:warren.weckes...@gmail.com>> wrote: > > > On Sat, Jun 15, 2013 at 11:43 AM, Warren Weckesser > mailto:warren.weckes...@gmail.com>> wrote: > > I'm getting a seg. fault i

Re: [Numpy-discussion] Seg. fault when running tests

On 15.06.2013 21:57, Warren Weckesser wrote: > > On Sat, Jun 15, 2013 at 3:15 PM, Julian Taylor > @warren, can you please bisect the commit causing this? > > > > Here's the culprit: > > aef286debfd11a62f1c337dea55624cee7fd4d9e

Re: [Numpy-discussion] Seg. fault when running tests

e: > > > On Sat, Jun 15, 2013 at 4:03 PM, Julian Taylor > <mailto:jtaylor.deb...@googlemail.com>> wrote: > > On 15.06.2013 21:57, Warren Weckesser wrote: > > > > On Sat, Jun 15, 2013 at 3:15 PM,

Re: [Numpy-discussion] Seg. fault when running tests

On 15.06.2013 22:26, Charles R Harris wrote: > > > On Sat, Jun 15, 2013 at 2:23 PM, Warren Weckesser > mailto:warren.weckes...@gmail.com>> wrote: > > > Also on 2.7. The -O3 flag seems to cause the problem. > > Chuck > I was wrong when I changed this flag. The compiler is allowed to assume th

Re: [Numpy-discussion] low level optimization in NumPy and minivect

On 17.06.2013 17:11, Frédéric Bastien wrote: > Hi, > > I saw that recently Julian Taylor is doing many low level optimization > like using SSE instruction. I think it is great. > > Last year, Mark Florisson released the minivect[1] project that he > worked on during is maste

[Numpy-discussion] quickselect via np.partition available in 1.8.dev

Hi, a selection algorithm [0] has now landed in the numpy development branch [1]. The function exposing it is: numpy.partition(data, kth=int/array, axis=-1, kind="introselect", order=None) Please see the docstrings on what it actually does (and report if they are confusing). Thanks to the numpy

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

On 04.09.2013 12:05, Graeme B. Bell wrote: > In my current GIS raster work I often have a situation where I generate code > something like this: > > np.any([A>4, A==2, B==5, ...]) > > However, np.any() is quite slow. > > It's possible to use np.logical_or to solve the problem, but the

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

e https://github.com/gbb/numpy-fast-any-all.git > # cd numpy-fast-any-all > # python test-fast-any-all.py > > Graeme > > > On Sep 4, 2013, at 7:38 PM, Julian Taylor > wrote: > >>> >>> The result is 14 to 17x faster than np.any() for this use case.* &

Re: [Numpy-discussion] ANN: 1.8.0b2 release.

try rebuilding everything from scratch. setup.py dependency handling is a bit dodgy with the generated files. On 09.09.2013 19:09, Frédéric Bastien wrote: > I don't have CFLAGS defined. But I have iothers env variable that point > to other python stuff like CPATH. > > But even in that case, I don

Re: [Numpy-discussion] strange behaviour with numpy arrays

On 11.09.2013 12:33, antlarac wrote: > Hi, I have a numpy array, and I want to create another variable equal to it, > to back it up for later calculations, because the first array will change. > But after the first array changes, the second automatically changes to the > same value. An example of w

Re: [Numpy-discussion] 1.8.0rc1

On 30.09.2013 17:17, Charles R Harris wrote: > Hi All, > > NumPy 1.8.0rc1 is up now on sourceforge > .The > binary builds are included except for Python 3.3 on windows, which will > arrive later. Many thanks to Ralf for the binaries, an

Re: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1

On 01.10.2013 01:30, Charles R Harris wrote: > > > > On Mon, Sep 30, 2013 at 5:12 PM, Christoph Gohlke > wrote: > > On 9/30/2013 3:45 PM, Charles R Harris wrote: > > > > > > > > On Mon, Sep 30, 2013 at 3:51 PM, Christoph Gohlke

Re: [Numpy-discussion] Efficient square distance computation

your computation is symmetric so you only need to compute the upper or lower triangle which will save both memory and time. On Tue, Oct 8, 2013 at 10:06 AM, Ke Sun wrote: > Dear all, > > I have written the following function to compute the square distances of a > large > matrix (each sample a r

Re: [Numpy-discussion] Efficient square distance computation

> Out of interest, how did you do this with matrix multiplication? http://stackoverflow.com/a/4856692 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Efficient square distance computation

On Tue, Oct 8, 2013 at 1:38 PM, Ke Sun wrote: > > > On a machine I had access to it took about 20 minutes. > How? I am using matrix multiplication (the same code as > http://stackoverflow.com/a/4856692) and it runs for around 18 hours. > > make sure you are using an optimized BLAS library. e.g. w

Re: [Numpy-discussion] Segfault with QR Decomposition

yes thats probably openblas fault. Openblas crashes all the time as soon as your matrices get bigger than a couple of megabytes. I'll investigate and report it upstream (as I have already far too often for the exact same reason ...) On Wed, Oct 9, 2013 at 5:05 PM, Charanpal Dhanjal wrote: > Oops

Re: [Numpy-discussion] Segfault with QR Decomposition

https://github.com/xianyi/OpenBLAS/issues/304 On 09.10.2013 17:24, Julian Taylor wrote: > yes thats probably openblas fault. > Openblas crashes all the time as soon as your matrices get bigger than a > couple of megabytes. > I'll investigate and report it upstream (as I have alre

Re: [Numpy-discussion] Reading from binary file with memmap, with offset

you have to use the offset keyword argument of np.memmap, else it will always start from the beginning of the file >>> np.memmap(fd, dtype="float32", mode="r", offset=offset) On Thu, Oct 10, 2013 at 2:43 PM, Andreas Hilboll wrote: > Hi, > > I have a problem using memmap correctly. I need to re

Re: [Numpy-discussion] Bug in numpy.correlate documentation

On 10.10.2013 21:31, Bernhard Spinnler wrote: > > On 10.10.2013, at 19:27, David Goldsmith > wrote: >> >> On Wed, Oct 9, 2013 at 7:48 PM, Bernhard Spinnler >> mailto:bernhard.spinn...@gmx.net>> wrote: >> > Hi Richard, >> > >> > Ah, I searched th

Re: [Numpy-discussion] 1.7.2

On 29.10.2013 21:00, Charles R Harris wrote: > > > > On Tue, Oct 29, 2013 at 1:57 PM, Charles R Harris > mailto:charlesr.har...@gmail.com>> wrote: > > Hi All, > > I'm going to tag 1.7.2 soon. That is, unless someone else would like > the experience of making a release. Any voluntee

[Numpy-discussion] ANN: NumPy 1.7.2rc1 release

will follow soon. Concerning OS X currently only a single person can create the binary installers which is not a good situation. If you have a suitable machine [0] and want to help out please contact us. Cheers, Julian Taylor [0] we currently base the releases on macos 10.6 using python.org p

Re: [Numpy-discussion] Caution about using intrisincs, and other 'advanced' optimizations

On 12.11.2013 03:17, David Cournapeau wrote: > Hi there, > > I have noticed more and more subtle and hard to track serious bugs in > numpy and scipy, due to the use of advanced optimization features > (flags, or gcc intrinsics). > > I am wondering whether those are worth it: they compile wrongly

Re: [Numpy-discussion] Caution about using intrisincs, and other 'advanced' optimizations

On 13.11.2013 18:26, David Cournapeau wrote: > > > Can you narrow it down to a specific intrinsic? they can be enabled and > disabled in set ./numpy/core/setup_common.py > > > valgrind shows quite a few invalid read in BOOL_ functions when running > the scipy or sklearn test suite. BOOL

Re: [Numpy-discussion] Caution about using intrisincs, and other 'advanced' optimizations

> > Will do, but the errors I am seeing only appear in the > simc.inc.src-based implementation of BOOL_logical_or (they disappear if > I disable the simd intrinsics manually in the numpy headers). > that is because the simd code always looks at the stride (as it only can run with unit strides) w

Re: [Numpy-discussion] numpy vbench-marking, compiler comparison

On 25.11.2013 02:32, Yaroslav Halchenko wrote: > > On Tue, 15 Oct 2013, Nathaniel Smith wrote: >> What do you have to lose? > >>> btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ . > >>> I have tuned benchmarking so it now reflects the best performance across >>> multiple

Re: [Numpy-discussion] numpy vbench-marking, compiler comparison

there isn't that much code in numpy that profits from modern x86 instruction sets, even the simple arithmetic loops are strided and thus unvectorizable by the compiler. They have been vectorized manually in 1.8 using sse2 and it is on my todo list to add runtime detected avx support. On 26.11.201

Re: [Numpy-discussion] numpy datetime64 NaT string conversion bug & patch

On 27.11.2013 21:51, Charles G. Waldman wrote: > If you convert an array of strings to datetime64s and 'NaT' (or one of > its variants) appears in the string, all subsequent values are > rendered as NaT: thanks, a little embarrassing I didn't spot that when I fixed a different bug in the function

Re: [Numpy-discussion] -ffast-math

On 29.11.2013 21:15, Dan Goodman wrote: > Hi, > > Is it possible to get access to versions of ufuncs like sin and cos but > compiled with the -ffast-math compiler switch? > > I recently noticed that my weave.inline code was much faster for some fairly > simple operations than my pure numpy code,

Re: [Numpy-discussion] -ffast-math

On 01.12.2013 21:53, Dan Goodman wrote: > Julian Taylor googlemail.com> writes: >> can you show the code that is slow in numpy? >> which version of gcc and libc are you using? >> with gcc 4.8 it uses the glibc 2.17 sin/cos with fast-math, so there >> should be no

Re: [Numpy-discussion] -ffast-math

On 01.12.2013 22:59, Dan Goodman wrote: > Julian Taylor googlemail.com> writes: >> your sin and exp calls are loop invariants, they do not depend on the >> loop iterable. >> This allows to move the expensive functions out of the loop and only >> leave some

Re: [Numpy-discussion] math.fsum() like ufunc for numpy

related this PR attempts to improve the accuracy of summation: https://github.com/numpy/numpy/pull/3685 but math.fsum gives the exact result so it would a valuable ufunc even when that PR is merged. python3.4 will have yet another accurate summation in the statistics module: http://www.python.org/

Re: [Numpy-discussion] nasty bug in 1.8.0??

I opened a ticket for it, though thinking about it, its probably intentional be intentional to find code that assumes it can use the strides to get the itemsize. https://github.com/numpy/numpy/issues/4091 On 02.12.2013 20:35, Neal Becker wrote: > I don't think that behavior is acceptable. > > Fr

Re: [Numpy-discussion] distributing wheels & SSE/superpack options

On 06.12.2013 19:06, Ralf Gommers wrote: > Hi all, > > There are a few discussions on packaging for the scientific Python stack > ongoing, on the NumFOCUS and distutils lists: > https://groups.google.com/forum/#!topic/numfocus/mVNakFqfpZg > https://groups.google.com/forum/#!topic/numfocus/HUcwXTM_

Re: [Numpy-discussion] building 32 bit numpy on 64 bit linux

On 12.12.2013 19:58, David Jones wrote: > I'm trying to compile 32-bit numpy on a 64 bit Centos 6 system, but fails > with the message: > > "Broken toolchain: cannot link a simple C program" > > It get's the compile flags right, but not the linker: > > C compiler: gcc -pthread -fno-strict-alias

Re: [Numpy-discussion] building 32 bit numpy on 64 bit linux

On 12.12.2013 20:40, David Jones wrote: > On 12/12/13 15:54, Julian Taylor wrote: >> On 12.12.2013 19:58, David Jones wrote: >>> I'm trying to compile 32-bit numpy on a 64 bit Centos 6 system, but fails >>> with the message: >>> >>>

Re: [Numpy-discussion] building 32 bit numpy on 64 bit linux

On 13.12.2013 18:46, David Jones wrote: > ... > > Correction. Of course LD_LIBRARY_PATH isn't seen by the compiler. It > only applies at run time. How embarrasing:) This isn't the first time > I've been bitten by that. > > I don't mind doing that with manual builds, but what about with pip? Is

Re: [Numpy-discussion] Need help with np.ma.median and np.apply_along_axis

On 27.12.2013 10:54, alex wrote: > median is faster in version 1.8 > ___ unfortunately that won't help here because masked median uses apply_along_axis again which is very slow, especially if one wants to calculate thousands of medians of 7 elements as i

Re: [Numpy-discussion] Loading large NIfTI file -> MemoryError

On 31.12.2013 14:13, Amira Chekir wrote: > Hello together, > > I try to load a (large) NIfTI file (DMRI from Human Connectome Project, > about 1 GB) with NiBabel. > > import nibabel as nib > img = nib.load("dmri.nii.gz") > data = img.get_data() > > The program crashes during "img.get_data()" wi

[Numpy-discussion] ANN: NumPy 1.7.2 release

ed to the last release candidate four additional minor issues have been fixed and compatibility with python 3.4b1 improved. Source tarballs, installers and release notes can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.7.2 Cheers, Julian T

Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 88, Issue 1

On 01.01.2014 16:50, Amira Chekir wrote: > On 31.12.2013 14:13, Amira Chekir wrote: >> > Hello together, >> > >> > I try to load a (large) NIfTI file (DMRI from Human Connectome Project, >> > about 1 GB) with NiBabel. >> > >> > import nibabel as nib >> > img = nib.load("dmri.nii.gz") >> > data = i

Re: [Numpy-discussion] Speedup by avoiding memory alloc twice in scalar array

On 18.07.2013 15:36, Nathaniel Smith wrote: > On Wed, Jul 17, 2013 at 5:57 PM, Frédéric Bastien wrote: >> On Wed, Jul 17, 2013 at 10:39 AM, Nathaniel Smith wrote: On Tue, Jul 16, 2013 at 11:55 AM, Nathaniel Smith wrote: >>> It's entirely possible I misunderstood, so let's see if we can

Re: [Numpy-discussion] adding fused multiply and add to numpy

On Thu, Jan 9, 2014 at 3:54 PM, Daπid wrote: > > On 8 January 2014 22:39, Julian Taylor wrote: > >> As you can see even without real hardware support it is about 30% faster >> than inplace unblocked numpy due better use of memory bandwidth. Its >> even more than two ti

Re: [Numpy-discussion] adding fused multiply and add to numpy

# require fma, use precise hypot, use exact summation (math.fsum) or at least kahan summation, full overflow/invalid checks etc > > > On Thu, Jan 9, 2014 at 9:43 AM, Freddie Witherden > wrote: > > On 08/01/14 21:39, Julian Taylor wrote: > >> An issue is software emula

Re: [Numpy-discussion] adding fused multiply and add to numpy

On 10.01.2014 01:49, Frédéric Bastien wrote: > > Do you know if those instruction are automatically used by gcc if we > use the good architecture parameter? > > they are used if you enable -ffp-contract=fast. Do not set it to `on` this is an alias to `off` due to the semantics of C. -ffast-math

Re: [Numpy-discussion] Memory allocation cleanup

On Fri, Jan 10, 2014 at 3:48 AM, Nathaniel Smith wrote: > On Thu, Jan 9, 2014 at 11:21 PM, Charles R Harris > wrote: > > [...] > > After a bit more research, some further points to keep in mind: > > Currently, PyDimMem_* and PyArray_* are just aliases for malloc/free, > and PyDataMem_* is an ali

Re: [Numpy-discussion] Memory allocation cleanup

On 10.01.2014 17:03, Nathaniel Smith wrote: > On Fri, Jan 10, 2014 at 9:18 AM, Julian Taylor > wrote: >> On Fri, Jan 10, 2014 at 3:48 AM, Nathaniel Smith wrote: >>> [...] >> >> For this reason and missing calloc I don't think we should use the Python

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 01/15/2014 11:25 AM, Daπid wrote: > On 15 January 2014 11:12, Hedieh Ebrahimi > wrote: > > I try to print my fileContent array after I read it and it looks > like this : > > ["b'C:UsersDocumentsProjectmytextfile1.txt'" > "b'

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 01/15/2014 01:38 PM, Julian Taylor wrote: > On 01/15/2014 11:25 AM, Daπid wrote: >> On 15 January 2014 11:12, Hedieh Ebrahimi for utf 8 data: > > d = np.loadtxt(file, dtype='utf8') > ups this is a very bad example as we can't have utf8 as its variable

Re: [Numpy-discussion] adding more unicode dtypes

On 15.01.2014 18:57, Charles R Harris wrote: > ... > > There was a discussion of this long ago and UCS-4 was chosen as the > numpy standard. There are just too many complications that arise in > supporting both. > my guess is that that discussion was before python3 and you could still simply tre

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 16.01.2014 00:42, Chris Barker wrote: > bump back to the OP: > On Wed, Jan 15, 2014 at 2:12 AM, Hedieh Ebrahimi > mailto:hedieh.ebrah...@amphos21.com>> wrote: > > fileContent=loadtxt(filePath,dtype=str) > > > do either of these work for you? > > fileContent=loadtxt(filePath,dtype='S') t

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

This thread is getting a little out of hand which is my fault for initially mixing different topics in one mail, so let me try to summarize: We have three issues here: - a loadtxt bug when loading strings in python3 this has nothing to do with encodings or dtypes it is a bug that should be fixed.

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin wrote: > On Fri, Jan 17, 2014 at 10:59:27AM +, Pauli Virtanen wrote: > > Julian Taylor googlemail.com> writes: > > [clip] > > > > For backward compatibility we *cannot* change S. > > Do you mean to say

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 2:10 PM, Julian Taylor < jtaylor.deb...@googlemail.com> wrote: > On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin < > oscar.j.benja...@gmail.com> wrote:... > ... > No latin1 de/encoding is required for anything, I don't know why you would >

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On Fri, Jan 17, 2014 at 2:40 PM, Oscar Benjamin wrote: > On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: > > On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin > > wrote: > > > > > On Fri, Jan 17, 2014 at 10:59:27AM +, Pauli Virtanen wrote: > &g

Re: [Numpy-discussion] using loadtxt to load a text file in to a numpy array

On 17.01.2014 15:12, Julian Taylor wrote: > On Fri, Jan 17, 2014 at 2:40 PM, Oscar Benjamin > mailto:oscar.j.benja...@gmail.com>> wrote: > > On Fri, Jan 17, 2014 at 02:10:19PM +0100, Julian Taylor wrote: > > On Fri, Jan 17, 2014 at 1:44 PM, Oscar Benjamin >

Re: [Numpy-discussion] accumulation operation

On 22.01.2014 18:23, Ralf Juengling wrote: > Executing the following code, > > > import numpy as np > a = np.zeros((3,)) > w = np.array([0, 1, 0, 1, 2]) > v = np.array([10.0, 1, 10.0, 2, 9]) > a[w] += v > > > > I was expecting ‘a’ to be array([20., 3., 9.]. Inst

Re: [Numpy-discussion] MKL and OpenBLAS

On 26.01.2014 18:06, Stéfan van der Walt wrote: > On Sun, 26 Jan 2014 16:40:44 +0200, Pauli Virtanen wrote: >> The Numpy Windows binaries distributed in the numpy project at >> sourceforge.net are compiled with ATLAS, which should count as an >> optimized BLAS. I don't recall what's the situation w

Re: [Numpy-discussion] MKL and OpenBLAS

On 26.01.2014 22:33, Sturla Molden wrote: > Julian Taylor wrote: > >> if this issue disqualifies accelerate, it also disqualifies openblas as >> a default. openblas has the same issue, we stuck a big fat warning into >> the docs (site.cfg) for this now as people keep ru

[Numpy-discussion] windows and C99 math

hi, numpys no-C99 fallback keeps turning up issues in corner cases, e.g. hypot https://github.com/numpy/numpy/issues/2385 log1p https://github.com/numpy/numpy/issues/4225 these only seem to happen on windows, on linux and mac it seems to use the C99 math library just fine. Are our binary builds f

Re: [Numpy-discussion] scipy image processing memory leak in python 2.7?

On 28.01.2014 19:44, Joseph McGlinchy wrote: > Hi numpy list! > > > > I am trying to do some image processing on a number of images, 72 to be > specific. I am seeing the python memory usage continually increase. which version of scipy are you using? there is a memory leak in ndimage.label in v

Re: [Numpy-discussion] Memory leak in numpy?

On 29.01.2014 20:44, Nathaniel Smith wrote: > On Wed, Jan 29, 2014 at 7:39 PM, Joseph McGlinchy wrote: >> Upon further investigation, I do believe it is within the scipy code where >> there is a leak. I commented out my call to processBinaryImage(), which is >> all scipy code calls, and my memory

Re: [Numpy-discussion] Memory leak?

which version of numpy are you using? there seems to be a leak in the scalar return due to the PyObject_Malloc usage in git master, but it doesn't affect 1.8.0 On Fri, Jan 31, 2014 at 7:20 AM, Chris Laumann wrote: > Hi all- > > The following snippet appears to leak memory badly (about 10 MB per

Re: [Numpy-discussion] Memory leak?

On 31.01.2014 18:12, Nathaniel Smith wrote: > On Fri, Jan 31, 2014 at 4:29 PM, Benjamin Root wrote: >> Just to chime in here about the SciPy Superpack... this distribution tracks >> the master branch of many projects, and then puts out releases, on the >> assumption that master contains pristine c

Re: [Numpy-discussion] striding through arbitrarily large files

On Tue, Feb 4, 2014 at 4:27 PM, RayS wrote: > At 07:09 AM 2/4/2014, you wrote: > >On 04/02/2014 16:01, RayS wrote: > > > I was struggling with methods of reading large disk files into numpy > > > efficiently (not FITS or .npy, just raw files of IEEE floats from > > > numpy.tostring()). When load

Re: [Numpy-discussion] Memory leak?

eck’s > webpage does say they are dev builds only two sentences into the paragraph. > > Best, Chris > > > > -- > Chris Laumann > Sent with Airmail > > On January 31, 2014 at 9:31:40 AM, Julian Taylor > (jtaylor.deb...@googlemail.com <mailto://jtaylor.de

Re: [Numpy-discussion] MKL and OpenBLAS

On Thu, Feb 6, 2014 at 1:11 PM, Thomas Unterthiner < thomas_unterthi...@web.de> wrote: > On 2014-02-06 11:10, Sturla Molden wrote: > > BTW: The performance of OpenBLAS is far behind Eigen, MKL and ACML, but > > better than ATLAS and Accelerate. > Hi there! > > Sorry for going a bit off-topic, but:

Re: [Numpy-discussion] Geometrically defined masking arrays; how to optimize?

meshgrid also has the sparse keyword argument which archives the same. On 12.02.2014 20:04, Chris Barker wrote: > An extra note here: > > One of the great things about numpy (as opposed, to say, MATLAB), is > array broadcasting Thus you generally don't need meshgrid -- why carry > all that extra

Re: [Numpy-discussion] argsort speed

On 17.02.2014 15:18, Francesc Alted wrote: > On 2/17/14, 1:08 AM, josef.p...@gmail.com wrote: >> On Sun, Feb 16, 2014 at 6:12 PM, Daπid wrote: >>> On 16 February 2014 23:43, wrote: What's the fastest argsort for a 1d array with around 28 Million elements, roughly uniformly distributed,

[Numpy-discussion] allocated memory cache for numpy

hi, I noticed that during some simplistic benchmarks (e.g. https://github.com/numpy/numpy/issues/4310) a lot of time is spent in the kernel zeroing pages. This is because under linux glibc will always allocate large memory blocks with mmap. As these pages can come from other processes the kernel mu

Re: [Numpy-discussion] allocated memory cache for numpy

On 17.02.2014 21:16, Sturla Molden wrote: > Julian Taylor wrote: > >> When an array is created it tries to get its memory from the cache and >> when its deallocated it returns it to the cache. > > Good idea, however there is already a C function that does this. It u

Re: [Numpy-discussion] allocated memory cache for numpy

On 17.02.2014 22:27, Sturla Molden wrote: > Nathaniel Smith wrote: >> Also, I'd be pretty wary of caching large chunks of unused memory. People >> already have a lot of trouble understanding their program's memory usage, >> and getting rid of 'greedy free' will make this even worse. > > A cache w

Re: [Numpy-discussion] allocated memory cache for numpy

On Tue, Feb 18, 2014 at 1:47 AM, David Cournapeau wrote: > > On Mon, Feb 17, 2014 at 7:31 PM, Julian Taylor > wrote: >> >> hi, >> I noticed that during some simplistic benchmarks (e.g. >> https://github.com/numpy/numpy/issues/4310) a lot of time is spent in >

Re: [Numpy-discussion] allocated memory cache for numpy

On Mon, Feb 17, 2014 at 9:42 PM, Nathaniel Smith wrote: > On 17 Feb 2014 15:17, "Sturla Molden" wrote: >> >> Julian Taylor wrote: >> >> > When an array is created it tries to get its memory from the cache and >> > when its deallocated it returns

Re: [Numpy-discussion] allocated memory cache for numpy

On 18.02.2014 16:21, Julian Taylor wrote: > On Mon, Feb 17, 2014 at 9:42 PM, Nathaniel Smith wrote: >> On 17 Feb 2014 15:17, "Sturla Molden" wrote: >>> >>> Julian Taylor wrote: >>> >>>> When an array is created it tries to get its memo

Re: [Numpy-discussion] Default builds of OpenBLAS development branch are now fork safe

On Thu, Feb 20, 2014 at 1:25 AM, Nathaniel Smith wrote: > Hey all, > > Just a heads up: thanks to the tireless work of Olivier Grisel, the OpenBLAS > development branch is now fork-safe when built with its default threading > support. (It is still not thread-safe when built using OMP for threading

Re: [Numpy-discussion] Default builds of OpenBLAS development branch are now fork safe

On Thu, Feb 20, 2014 at 3:50 PM, Olivier Grisel wrote: > Thanks for sharing, this is all very interesting. > > Have you tried to have a look at the memory usage and import time of > numpy when linked against libopenblas.dll? > > -- this is probably caused by the memory warmup it can be disabled w

Re: [Numpy-discussion] cPickle.loads and Numeric

On Tue, Feb 25, 2014 at 5:41 PM, Alexander Belopolsky wrote: > > On Tue, Feb 25, 2014 at 11:29 AM, Benjamin Root wrote: >> >> I seem to recall reading somewhere that pickles are not intended to be >> long-term archives as there is no guarantee that a pickle made in one >> version of python would

Re: [Numpy-discussion] Help Understanding Indexing Behavior

On 26.02.2014 00:04, JB wrote: > At the risk of igniting a flame war...can someone please help me understand > the indexing behavior of NumPy? I will readily I admit I come from a Matlab > background, but I appreciate the power of Python and am trying to learn more. > >>From a Matlab user's persp

[Numpy-discussion] last call for fixes for numpy 1.8.1rc1

hi, We want to start preparing the release candidate for the bugfix release 1.8.1rc1 this weekend, I'll start preparing the changelog tomorrow. So if you want a certain issue fixed please scream now or better create a pull request/patch on the maintenance/1.8.x branch. Please only consider bugfix

Re: [Numpy-discussion] last call for fixes for numpy 1.8.1rc1

s/286 > > Thanks, > Francesc > > On 2/27/14, 9:05 PM, Julian Taylor wrote: >> hi, >> >> We want to start preparing the release candidate for the bugfix release >> 1.8.1rc1 this weekend, I'll start preparing the changelog tomorrow. >> >> S

Re: [Numpy-discussion] last call for fixes for numpy 1.8.1rc1

id. > > On the other hand, increasing the temporary arrays in nditer from 32kb > to 128kb is a bit worrying, but probably we should do some benchmarks > and see how much performance would be compromised (if any). > > Francesc > > On 2/28/14, 1:09 PM, Julian Taylor wrote:

Re: [Numpy-discussion] 2D array indexing

On 01.03.2014 00:32, Gökhan Sever wrote: > > Hello, > > Given this simple 2D array: > > In [1]: np.arange(9).reshape((3,3)) > Out[1]: > array([[0, 1, 2], >[3, 4, 5], >[6, 7, 8]]) > > In [2]: a = np.arange(9).reshape((3,3)) > > In [3]: a[:1:] > Out[3]: array([[0, 1, 2]]) > > In

[Numpy-discussion] numpy gsoc topic idea: configurable algorithm precision and vector math library integration

hi, as the numpy gsoc topic page is a little short on options I was thinking about adding two topics for interested students. But as I have no experience with gsoc or mentoring and the ideas are not very fleshed out yet I'd like to ask if it might make sense at all: 1. configurable algorithm prec

Re: [Numpy-discussion] 1.8.1rc1 on sourceforge.

On 04.03.2014 18:08, Christoph Gohlke wrote: > On 3/4/2014 4:49 AM, Thomas Unterthiner wrote: >> Hi there! >> >> I just tried setting up a new installation using numpy 1.8.1rc1 (+scipy >> 0.13.3 and matplotlib 1.3.1). I ran into problems when installing >> matplotlib 1.3.1. The attached logfile sho

Re: [Numpy-discussion] 1.8.1 release

On 06.03.2014 19:46, Skipper Seabold wrote: > Hi, > > Should [1] be considered a release blocker for 1.8.1? > > Skipper > > [1] https://github.com/numpy/numpy/issues/4442 as far as I can tell its a regression of the 1.8.0 release but not the 1.8.1 release so I wouldn't consider it a blocker. Bu

Re: [Numpy-discussion] 1.8.1rc1 on sourceforge.

lled cleanly) when I re-did > the whole shebang using numpy 1.8.0, so I suspect this was caused by > something in the rc. > > Cheers > > Thomas > > > > On 2014-03-03 17:23, Charles R Harris wrote: >> Hi All, >> >> Julian Taylor has put windows bi

Re: [Numpy-discussion] GSoC project: draft of proposal

On 14.03.2014 17:40, Robert Kern wrote: > On Fri, Mar 14, 2014 at 4:33 PM, Leo Mao wrote: > >> Yeppp is bsd 3 clauses so I think Yeppp is really a good choice. >> Is there a list of licenses which can be added into numpy without pain? (how >> about LGPL3 ?) > > No, just BSD and its rough equival

Re: [Numpy-discussion] GSoC project: draft of proposal

lean stuff) The runtime feature detection vector libraries provide can be very useful for this. Regards, Julian Taylor ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Drop support for Python 3.2?

On 25.03.2014 00:28, Charles R Harris wrote: > Hi All, > > The suggestion has been made the we drop Python 3.2 support in numpy 1.9 > and scipy 0.15. The advantage, from my point of view, to supporting > Python >= 3.3 is that the u'unicode' syntax is supported in 3.3 and this > makes it easier to

  1   2   3   4   >