Re: [Numpy-discussion] ndarray.T2 for 2D transpose

[Stéfan van der Walt]

On 7 April 2016 at 15:03, Stéfan van der Walt <stef...@berkeley.edu> wrote:
> 4) x @ colvec(x)  -- gives an error, but perhaps this should work and
> be equivalent to np.dot(colvec(x), rowvec(x)) ?

Sorry, that should have been

4) colvec(x) @ x

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray.T2 for 2D transpose

[Stéfan van der Walt]

On 7 April 2016 at 11:17, Chris Barker  wrote:
> np.col_vector(arr)
>
> which would be a synonym for np.reshape(arr, (-1,1))
>
> would that make anyone happy?

I'm curious to see use cases where this doesn't solve the problem.

The most common operations that I run into:

colvec = lambda x: np.c_[x]

x = np.array([1, 2, 3])
A = np.arange(9).reshape((3, 3))


1) x @ x   (equivalent to x @ colvec(x))
2) A @ x  (equivalent to A @ colvec(x), apart from the shape)
3) x @ A
4) x @ colvec(x)  -- gives an error, but perhaps this should work and
be equivalent to np.dot(colvec(x), rowvec(x)) ?

If (4) were changed, 1D arrays would mostly* be interpreted as row
vectors, and there would be no need for a rowvec function.  And we
already do that kind of magic for (2).

Stéfan

* not for special case (1)
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy FFT.FFT slow with certain samples

[Stéfan van der Walt]

On Aug 28, 2015 5:17 PM, Pierre-Andre Noel noel.pierre.an...@gmail.com
wrote:

 I had in mind the use of FFT to do convolutions (
 https://en.wikipedia.org/wiki/Convolution_theorem ). If you do not
 zero-pad properly, then the end of the signal may bleed on the
 beginning, and vice versa.

Ah, gotcha! All these things should also be handled nicely in
scipy.signal.fftconvolve.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Changed behavior of np.gradient

[Stéfan van der Walt]

On Oct 4, 2014 10:14 PM, Derek Homeier 
de...@astro.physik.uni-goettingen.de wrote:

 +1 for an order=2 or maxorder=2 flag

If you parameterize that flag, users will want to change its value (above
two). Perhaps rather use a boolean flag such as second_order or
high_order, unless it seems feasible to include additional orders in the
future.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Fwd: We need help working on code coverage for Cython code

[Stéfan van der Walt]

Hi Matthew

On Tue, Aug 12, 2014 at 9:24 PM, Matthew Brett matthew.br...@gmail.com wrote:
 The Cython developers have done some work on this [1] but it is
 currently stalled for lack of developer time to work on it.

It looks like we can help them with the rest of the work once the
lnotab PR is merged; is that correct?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Fwd: We need help working on code coverage for Cython code

[Stéfan van der Walt]

Hi Matthew

On Tue, Aug 12, 2014 at 9:49 PM, Matthew Brett matthew.br...@gmail.com wrote:
 My very vague impression is that Stefan B thinks of the lnotab PR as
 part of the process of getting the work done, so that merging would
 only be worthwhile if it was pretty clear that the rest of the work
 would happen as well.  We could ask again on the Cython list...

I think a clear roadmap with small targets would help--few of us know
Cython in too much depth, so it would help if we could avoid any
duplicate effort/research.

Thank you for bringing this issue up on the radar.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Fwd: We need help working on code coverage for Cython code

[Stéfan van der Walt]

On Tue, Aug 12, 2014 at 10:15 PM, Matthew Brett matthew.br...@gmail.com wrote:
 The first step we thought of was having a group live conversation of
 some sort with the Cython developers to get an idea of what work needs
 doing.  So, I think the first question is - who would be up for
 joining that?

I'd be up for that.  Also, perhaps some key Cython players would be at
EuroSciPy, then we can discuss it in person?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Inverted indices

[Stéfan van der Walt]

Hi Nicolas

On Thu, Aug 7, 2014 at 1:16 PM, Nicolas P. Rougier
nicolas.roug...@inria.fr wrote:
 Here is a small example:

 Z = [(0,0), (1,1), (2,2), (3,3), (4,4))
 I  = [0, 20, 23, 24, 37]

 S = [ 20,20,0,24]
 - Result should be [(1,1), (1,1), (0,0),(3,3)]

 S = [15,15]
 - Wrong (15 not in I) but ideally, I would like this to be converted to 
 [(0,0), (0,0)]

First try:

Z = np.array([(0,0), (1,1), (2,2), (3,3), (4,4)])
I = np.array([0, 20, 23, 24, 37])
S = np.array([ 20,20,0,24,15])

out = np.zeros((len(S), len(Z[0])))
mask = (S[:, np.newaxis] == I)
item, coord = np.where(mask)
out[item, :] = Z[coord]

Perhaps there's a neater way of doing it!

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Remove bento from numpy

[Stéfan van der Walt]

On Sat, Jul 5, 2014 at 6:40 PM, David Cournapeau courn...@gmail.com wrote:
 The efforts are on average less demanding than this discussion. We are
 talking about adding entries to a list in most cases...

In scikit-image we use the following script to check for the most
basic discrepancies:

https://github.com/scikit-image/scikit-image/blob/master/check_bento_build.py

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy 1.9b1 bug in pad function?

[Stéfan van der Walt]

Hi Nadav

On Sat, Jun 14, 2014 at 8:11 AM, Nadav Horesh nad...@visionsense.com wrote:
 In [4]: np.__version__
 Out[4]: '1.9.0b1'

 The documentation specify that the mode parameter is optional

I don't see the optional specification in the docstring.  Perhaps
because mode=None in the signature?

The reason is that then, if you do not specify the signature (as in
your case), you get the following helpful message:

ValueError: Keyword mode must be a function or one of ['reflect',
'linear_ramp', 'edge', 'constant', 'minimum', 'wrap', 'symmetric',
'median', 'maximum', 'mean'].

Instead of

pad() takes exactly 3 arguments (2 given)

Regards
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] SciPy 2014 BoF NumPy Participation

[Stéfan van der Walt]

Hi Kyle

Kyle Mandli writes:

 The BoF format would be up to those who would lead
 the discussion, a couple of ideas used in the past include picking out a
 few of the lead devs to be on a panel and have a QA type of session or an
 open QA with perhaps audience guided list of topics.

Unfortunately I won't be at the conference this year, but if I were I'd
have enjoyed seeing a couple of short presentations, drawn from, e.g.,
some of the people involved in this discussion (Nathan can perhaps join
in via Google Hangout), about possible future directions.  That way one
can sketch out the playing field to seed the discussion.  In addition, I
those sketches would provide a useful update to all those watching the
conference remotely via video.

Regards
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Suggestions for GSoC Projects

[Stéfan van der Walt]

On 8 Feb 2014 04:51, Ralf Gommers ralf.gomm...@gmail.com wrote:

  Members of the dipy team would also be interested.

 That's specifically for the spherical harmonics topic right?

Right. Spherical harmonics are used as bases in many of DiPy's
reconstruction algorithms.

You are right, though, that gsoc would also require an expert in special
functions.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Suggestions for GSoC Projects

[Stéfan van der Walt]

On Tue, 04 Feb 2014 12:21:58 +0100, Ralf Gommers wrote:
 Finding a suitable mentor for whatever project Jennifer chooses is an
 important factor in the choice of project, so I have to ask: do you have
 the bandwidth to be a mentor or help out this summer?

I completely agree.  I have time to be co-mentor, and I have ideas for
other potential mentors (of course, Ralf would be on top of that list
:).  Members of the dipy team would also be interested.

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Suggestions for GSoC Projects

[Stéfan van der Walt]

On Fri, 31 Jan 2014 04:31:01 +0530, jennifer stone wrote:
 3. As stated earlier, we have spherical harmonic functions (with much scope
 for dev) we are yet to have elliptical and cylindrical harmonic function,
 which may be developed.

As stated before, I am personally interested in seeing the spherical harmonics
in SciPy improve.

 5. Further reading the road-map given by Mr.Ralf, I would like to develop
 the Bluestein's FFT algorithm.

https://gist.github.com/endolith/2783807

Regards
Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy Enhancement Proposal: group_by functionality

[Stéfan van der Walt]

Hi Eelco

On Sun, 26 Jan 2014 12:20:04 +0100, Eelco Hoogendoorn wrote:
 key1 = list('abaabb')
 key2 = np.random.randint(0,2,(6,2))
 values = np.random.rand(6,3)
 print group_by((key1, key2)).median(values)

I agree that group_by functionality could be handy in numpy.
In the above example, what would the output of

``group_by((key1, key2))``

be?

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] MKL and OpenBLAS

[Stéfan van der Walt]

On Sun, 26 Jan 2014 16:40:44 +0200, Pauli Virtanen wrote:
 The Numpy Windows binaries distributed in the numpy project at
 sourceforge.net are compiled with ATLAS, which should count as an
 optimized BLAS. I don't recall what's the situation with OSX binaries,
 but I'd believe they're with Atlas too.

Was a switch made away from Accelerate after this?

http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Comparison changes

[Stéfan van der Walt]

On Sat, 25 Jan 2014 01:05:15 +0100, Sebastian Berg wrote:
 1. Comparison with None will broadcast in the future, so that `arr ==
 None` will actually compare all elements to None. (A FutureWarning for
 now)

This is a very useful change in behavior--thanks!

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] np.array creation: unexpected behaviour

[Stéfan van der Walt]

On Fri, 24 Jan 2014 17:30:33 +0100, Emanuele Olivetti wrote:
 I just came across this unexpected behaviour when creating
 a np.array() from two other np.arrays of different shape.

The tuple parsing for the construction of new numpy arrays is pretty
tricky/hairy, and doesn't always do exactly what you'd expect.

The easiest workaround is probably to pre-allocate the array:

In [24]: data = [a, c]
In [25]: x = np.empty(len(data), dtype=object)
In [26]: x[:] = data
In [27]: x.shape
Out[27]: (2,)

Regards
Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] (no subject)

[Stéfan van der Walt]

On Tue, 21 Jan 2014 21:56:17 +0530, jennifer stone wrote:
  I am an undergraduate student with CS as major and have interest in Math
 and Physics. This has led me to use NumPy and SciPy to work on innumerable
 cases involving special polynomial functions and polynomials like Legendre
 polynomials, Bessel Functions and so on. So, The packages are closer known
 to me from this point of view. I have a* few proposals* in mind. But I
 don't have any idea if they are acceptable within the scope of GSoC
 1. Many special functions and polynomials are neither included in NumPy nor
 on SciPy.. These include Ellipsoidal Harmonic Functions (lames function),
 Cylindrical Harmonic function. Scipy at present supports only spherical
 Harmonic function.

SciPy's spherical harmonics are very inefficient if one is only interested in
computing one specific order.  I'd be so happy if someone would work on that!

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] surprising behavior of np.asarray on masked arrays

[Stéfan van der Walt]

Hi Faraz

On Thu, 05 Dec 2013 19:14:01 -0800, Faraz Mirzaei wrote:
 If I pass a masked array through np.asarray, I get original unmasked array.

`asarray` disregards any information attached to the underlying ndarray by the
subclass.  To preserve the subclass, you'd need to use `asanyarray`.

The only functions that are aware of masked arrays live inside of `np.ma`, so
you can also use `np.ma.asarray`.

Which behavior in particular would you like to see, since I presume you can
already get hold of the filled array, should you want to?

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] strange runtimes of numpy fft

[Stéfan van der Walt]

On Tue, Nov 19, 2013 at 6:03 PM, Henry Gomersall h...@cantab.net wrote:
 However, FFTW is dual licensed GPL/commercial and so the wrappers are
 also GPL by necessity.

I'm not sure if that is true, strictly speaking--you may license your
wrapper code under any license you wish.  It's just that it becomes
confusing when the combined work then has to be released under GPL.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ANN: NumPy 1.7.2rc1 release

[Stéfan van der Walt]

Hi Christian

On Sun, 10 Nov 2013 21:06:00 -0300, Christian K. wrote:
 On OSX compilation succeeds (with some errors though) but test() fails.

Do you have the build log available as well?

Thanks
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy 1.9 release date

[Stéfan van der Walt]

On 9 Nov 2013 03:22, Charles R Harris charlesr.har...@gmail.com wrote:

 that the main thing missing at this point is fixing the datetime problems.

What needs to be done, and what is the plan forward? Is there perhaps an
issue one can follow?

Thanks
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] ANN: scikit-image 0.9 release

[Stéfan van der Walt]

We're happy to announce the release of scikit-image v0.9.0!

scikit-image is an image processing toolbox for SciPy that includes algorithms
for segmentation, geometric transformations, color space manipulation,
analysis, filtering, morphology, feature detection, and more.

For more information, examples, and documentation, please visit our website:

http://scikit-image.org


New Features


`scikit-image` now runs without translation under both Python 2 and 3.

In addition to several bug fixes, speed improvements and examples, the 204 pull
requests merged for this release include the following new features (PR number
in brackets):

Segmentation:

- 3D support in SLIC segmentation (#546)
- SLIC voxel spacing (#719)
- Generalized anisotropic spacing support for random_walker (#775)
- Yen threshold method (#686)

Transforms and filters:

- SART algorithm for tomography reconstruction (#584)
- Gabor filters (#371)
- Hough transform for ellipses (#597)
- Fast resampling of nD arrays (#511)
- Rotation axis center for Radon transforms with inverses. (#654)
- Reconstruction circle in inverse Radon transform (#567)
- Pixelwise image adjustment curves and methods (#505)

Feature detection:

- [experimental API] BRIEF feature descriptor (#591)
- [experimental API] Censure (STAR) Feature Detector (#668)
- Octagon structural element (#669)
- Add non rotation invariant uniform LBPs (#704)

Color and noise:

- Add deltaE color comparison and lab2lch conversion (#665)
- Isotropic denoising (#653)
- Generator to add various types of random noise to images (#625)
- Color deconvolution for immunohistochemical images (#441)
- Color label visualization (#485)

Drawing and visualization:

- Wu's anti-aliased circle, line, bezier curve (#709)
- Linked image viewers and docked plugins (#575)
- Rotated ellipse + bezier curve drawing (#510)
- PySide  PyQt4 compatibility in skimage-viewer (#551)

Other:

- Python 3 support without 2to3. (#620)
- 3D Marching Cubes (#469)
- Line, Circle, Ellipse total least squares fitting and RANSAC algorithm (#440)
- N-dimensional array padding (#577)
- Add a wrapper around `scipy.ndimage.gaussian_filter` with useful
default behaviors. (#712)
- Predefined structuring elements for 3D morphology (#484)


API changes
---

The following backward-incompatible API changes were made between 0.8 and 0.9:

- No longer wrap ``imread`` output in an ``Image`` class
- Change default value of `sigma` parameter in ``skimage.segmentation.slic``
  to 0
- ``hough_circle`` now returns a stack of arrays that are the same size as the
  input image. Set the ``full_output`` flag to True for the old behavior.
- The following functions were deprecated over two releases:
  `skimage.filter.denoise_tv_chambolle`,
  `skimage.morphology.is_local_maximum`, `skimage.transform.hough`,
  `skimage.transform.probabilistic_hough`,`skimage.transform.hough_peaks`.
  Their functionality still exists, but under different names.


Contributors to this release


This release was made possible by the collaborative efforts of many
contributors, both new and old.  They are listed in alphabetical order by
surname:

- Ankit Agrawal
- K.-Michael Aye
- Chris Beaumont
- François Boulogne
- Luis Pedro Coelho
- Marianne Corvellec
- Olivier Debeir
- Ferdinand Deger
- Kemal Eren
- Jostein Bø Fløystad
- Christoph Gohlke
- Emmanuelle Gouillart
- Christian Horea
- Thouis (Ray) Jones
- Almar Klein
- Xavier Moles Lopez
- Alexis Mignon
- Juan Nunez-Iglesias
- Zachary Pincus
- Nicolas Pinto
- Davin Potts
- Malcolm Reynolds
- Umesh Sharma
- Johannes Schönberger
- Chintak Sheth
- Kirill Shklovsky
- Steven Silvester
- Matt Terry
- Riaan van den Dool
- Stéfan van der Walt
- Josh Warner
- Adam Wisniewski
- Yang Zetian
- Tony S Yu
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Python function for line intersection??

[Stéfan van der Walt]

On Tue, Oct 8, 2013 at 3:15 PM, Happyman bahtiyor_zohi...@mail.ru wrote:
 # slope of each line
 m = (B[:, 1] - A[:, 1]) / (B[:, 0] - A[:, 0])
 b = A[:, 1] - m * A[:, 0]

Calculating the slope of a line leads to difficulties, e.g. when
dealing with vertical lines.  Rather, parameterize lines as shown
here:

http://paulbourke.net/geometry/pointlineplane/

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [SciPy-Dev] 1.8.0rc1

[Stéfan van der Walt]

Hi Chuck

On Tue, Oct 1, 2013 at 1:07 AM, Charles R Harris
charlesr.har...@gmail.com wrote:
 I'll bet the skimage problems come from
 https://github.com/numpy/numpy/pull/3811. They may be doing something
 naughty...


 Reverting that commit fixes those skimage failures. However, there are a
 number of python2.7 failures that look pretty strange.

What is the exact change in behavior with that PR?  I'm trying to
figure out what skimage does wrong in this case.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max, min} for all-nan slices.

[Stéfan van der Walt]

On 2 Oct 2013 18:04, Charles R Harris charlesr.har...@gmail.com wrote:

 The question is what to do when all-nan slices are encountered in the
nan{max,min} and nanarg{max, min} functions. Currently in 1.8.0, the first
returns nan and raises a warning, the second returns intp.min and raises a
warning. It is proposed that the nanarg{max, min} functions, and possibly
the nan{max, min} also, raise an error instead.

I agree with Nathan; this sounds like more reasonable behaviour to me.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max, min} for all-nan slices.

[Stéfan van der Walt]

On 2 Oct 2013 19:14, Benjamin Root ben.r...@ou.edu wrote:

 And it is logically consistent, I think.  a[nanargmax(a)] == nanmax(a)
(ignoring the silly detail that you can't do an equality on nans).

Why do you call this a silly detail? It seems to me a fundamental flaw to
this approach.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Behavior of nan{max, min} and nanarg{max, min} for all-nan slices.

[Stéfan van der Walt]

On 2 Oct 2013 21:19, Charles R Harris charlesr.har...@gmail.com wrote:


 The main problem I had was deciding what arg{max, min} should return as
the return value is an integer. I like your suggestion of returning 0.

This doesn't allow the user to know the difference between valid and
invalid output, does it?

 One further possibility is to add a keyword 'raise' to make the behavior
selectable.

Preferably just pick a sensible default.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy newbie question: matrix creation

[Stéfan van der Walt]

On Wed, Sep 25, 2013 at 9:06 AM, Edmondo Porcu edmondo.po...@gmail.com wrote:
 I am a Newbie with Numpy and I would need some advice on how to create a
 matrix with certain characteristics :

  - Every entry should be minimum 0 maximum 1 with a step of 0.1 (legal
 values are 0,0.1,0.2,0.3 etc)

You can generate random integers between 0 and 10, and divide them by 10:

np.random.random_integers(0, 10, (5, 5)) / 10.

 - Only the rows where the sum is 1 must be kept

I'm not sure I understand the exact structure you describe.  Perhaps
an example would help.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Generating a (uniformly distributed) random bit list of length N

[Stéfan van der Walt]

On 22 Sep 2013 23:04, David Goldsmith d.l.goldsm...@gmail.com wrote:

 Is np.random.randint(2, size=N) the fastest way to do this?  Thanks!

Are you concerned about speed or memory use? The operation you show should
already be quite fast. A more memory efficient approach would be to
generate integers and use their binary representation.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] multiprocessing, numpy and 32-64 bit cohabitation

[Stéfan van der Walt]

On Wed, Sep 18, 2013 at 2:51 AM, Antony Lee antony@berkeley.edu wrote:
 While I realize that this is certainly tweaking multiprocessing beyond its
 specifications, I would like to use it on Windows to start a 32-bit Python
 process from a 64-bit Python process (use case: I need to interface with a
 64-bit DLL and use an extension (pyFFTW) for which I can only find a 32-bit
 compiled version (yes, I could try to install MSVC and compile it myself but
 I'm trying to avoid that...))

In the end, this may be the less painful solution, especially if speed
is a requirement.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] find appropriate dtype based on a set of values

[Stéfan van der Walt]

On Tue, Sep 3, 2013 at 2:47 PM, Robert Kern robert.k...@gmail.com wrote:
 Here's one way of doing it: https://gist.github.com/stefanv/6413742

 You can probably reduce the amount of work by only comparing a.min() and
 a.max() instead of the whole array.

Thanks, fixed.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Faster np.triu_indices

[Stéfan van der Walt]

On Mon, Sep 2, 2013 at 12:38 AM, Daniel Smith malor...@me.com wrote:
 This is just a first run at the function, and unfortunately does not work for 
 k0. However, it does return the correct results for k=0 and is between 2-8 
 faster on my machine then `np.triu_indices`. Any thoughts on this?

That looks like a good start to me.  Let's get a pull request up that
fixes it for k  0 as well, and then continue the discussion there.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] find appropriate dtype based on a set of values

[Stéfan van der Walt]

On Mon, Sep 2, 2013 at 4:21 PM, Gregorio Bastardo
gregorio.basta...@gmail.com wrote:
 np.min_scalar_type([-1,256]) # int16 expected
 dtype('int32')

 Am I missing something? Anyone knows how to achieve the desired operation?

The docstring states explicitly that this use case is not supported.

Here's one way of doing it: https://gist.github.com/stefanv/6413742

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Relative speed

[Stéfan van der Walt]

On Fri, Aug 30, 2013 at 6:20 AM, Anubhab Baksi anubha...@gmail.com wrote:
 I need to deal with nearly 2**19 or 2**20 arrays of length about 250 each.

As mentioned elsewhere in this thread: what does deal mean.  You may
be better off with something like:

http://kwant-project.org/tinyarray/

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] _PyADt

[Stéfan van der Walt]

On Fri, Aug 30, 2013 at 1:27 AM, Charles R Harris
charlesr.har...@gmail.com wrote:
 Anyone know what _PyADt is? It turns up in ndarraytypes.h

 #define PyDataType_ISBOOL(obj) PyTypeNum_ISBOOL(_PyADt(obj))

That code looks broken--can't we just remove it?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Structured array dtype

[Stéfan van der Walt]

Hi Nicolas

On Fri, 30 Aug 2013 17:26:51 +0200, Nicolas Rougier wrote:
  Z = np.zeros(10, [('a', np.float32, 3), ('b', np.float32, 4)])
 
  Z['a'].dtype
 dtype('float32')
 
  Z.dtype['a']
 dtype(('f4', (3,)))
 
 
 Does that mean that dtype['a'] is the dtype of field 'a' when in Z, while 
 Z['a'].dtype is the dtype of field 'a' when extracted or my way of thinking 
 is totally wrong ?

Apologies if this is a duplicate response; I'm sending it offline.

In case 1, you are indexing into the array, and querying its dtype.  In case
two, you are indexing into a dtype.

I.e., in case two, you are doing this:

In [18]: dtype = np.dtype([('a', float, 3), ('b', int)])

In [19]: dtype['a']
Out[19]: dtype(('f8', (3,)))

 What bothers me the most is that I cannot do:
 
  Z['a'].view(Z.dtype['a'])
 ValueError: new type not compatible with array.

That's quite a tricky operation to perform, since it has to take into account
the underlying strides of the old array as well as calculate a shape for the
new array.  It should be possible to make it work using something similar to
`np.lib.stride_tricks.as_strided`, but my quick attempt failed because of the
following:

In [13]: class Foo:
__array_interface__ = Z.__array_interface__
   :

In [14]: f = Foo()

In [15]: np.asarray(f)
Out[15]:
array([, ,
   , ,
   , ,
   , ,
   , ],
  dtype='|V28')

This does not seem right.

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Adding an axis argument to numpy.unique

[Stéfan van der Walt]

On Tue, Aug 20, 2013 at 2:39 AM, Joe Kington joferking...@gmail.com wrote:
 That's certainly a potential source of confusion.  However, I can't seem to
 come up with a better name for the kwarg. Matlab's unique function has a
 rows option, which is probably a more intuitive name, but doesn't imply
 the expansion to N-dimensions.

To me, `unique_rows` sounds perfect.  To go along columns: unique_rows(A.T)

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Deprecation of financial routines

[Stéfan van der Walt]

On Mon, Aug 19, 2013 at 3:47 PM, Skipper Seabold jsseab...@gmail.com wrote:

 +1 on scipy.finance / scipy.financial (or even numpy.finance /
 numpy.financial)

Are there no external libraries that deal with these things?  If they
exist, we can deprecate with two releases pointing to that external
source.

Factoring the code out to a package only makes sense if someone is
willing to take ownership, otherwise we're essentially deprecating it
out to die.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Upcoming 1.8 release.

[Stéfan van der Walt]

On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris
charlesr.har...@gmail.com wrote:

 I'm thinking of making the 1.8.x branch next Sunday. Any complaints, thoughts?

Thanks, Chuck.  Are there any specific PRs up for review that should
be incorporated into 1.8?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Using as_strided to avoid copy on repeat (with 2-dimensional array)

[Stéfan van der Walt]

On Mon, Aug 12, 2013 at 11:23 PM, Robert Kern robert.k...@gmail.com wrote:
 Is it possible to do the same using 'as_strided' to avoid copy (and still
 get the same output shape for B) ?

 No, this would not be uniformly strided in the 0 axis.

Now, if only we supported simple fraction strides...

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Reading footer lines

[Stéfan van der Walt]

Hi Resmi

On Tue, Aug 13, 2013 at 2:20 PM, Resmi l.re...@gmail.com wrote:
 I've a list of long files of numerical data ending with footer lines
 (beginning with #). I am using numpy.loadtxt to read the numbers, and
 loadtxt ignores these footer lines. I want the numpy code to read one of the
 footer lines and extract words from it. Is there a way to use loadtxt for
 this? If there weren't many files I could have used the line number (which
 keep varying between the files) of the footer line along with linecache.
 Nevertheless there should be a generic way to do this in numpy?

If the data-set is not too big, you can read it into memory with
f.readlines() and then do a bit of preparsing on the resulting array
before sending the rest of it to np.loadtxt (which allows an array of
strings as input).

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] quickselect via np.partition available in 1.8.dev

[Stéfan van der Walt]

Hi Julian

On Mon, Aug 12, 2013 at 4:23 PM, Julian Taylor
jtaylor.deb...@googlemail.com wrote:
 The function exposing it is:
 numpy.partition(data, kth=int/array, axis=-1, kind=introselect,
 order=None)

This looks great, thanks very much!

A minor bug was introduced into the Bento build:

https://github.com/numpy/numpy/pull/3606
https://github.com/numpy/numpy/pull/3607

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] unit tests / developing numpy

[Stéfan van der Walt]

On Thu, 25 Jul 2013 08:47:03 +, Graeme B. Bell wrote:
 
 Does anyone know how to get the unit tests to run on a local fork, without 
 doing a complete install of numpy? 
 

I usually do an in-place build with either

bentomaker build -i -j

or

python setup.py build_ext -i

Then

export PYTHONPATH=$PYTHONPATH:/path/to/numpy

and

nosetests numpy

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] add .H attribute?

[Stéfan van der Walt]

On Wed, Jul 24, 2013 at 2:15 AM, Chris Barker - NOAA Federal
chris.bar...@noaa.gov wrote:

 On Tue, Jul 23, 2013 at 6:09 AM, Pauli Virtanen p...@iki.fi wrote:

  The .H property has been implemented in Numpy matrices and Scipy's
  sparse matrices for many years.

 Then we're done. Numpy  is an array package, NOT a matrix package, and
 while you can implement matrix math with arrays (and we do), having
 quick and easy mnemonics for common matrix math operations (but
 uncommon general purpose array operations) is not eh job of numpy.
 That's what the matrix object is for.

I would argue that the ship sailed when we added .T already.  Most
users see no difference between the addition of .T and .H.

The matrix class should probably be deprecated and removed from NumPy
in the long run--being a second class citizen not used by the
developers themselves is not sustainable.  And, now that we have dot
as a method, there's very little advantage to it.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] add .H attribute?

[Stéfan van der Walt]

On Wed, Jul 24, 2013 at 9:15 AM, Sebastian Haase seb.ha...@gmail.com wrote:
 I feel that adding a method
 .H()
 would be the compromise !

Thinking about this more, I think it would just confuse most users...
why .T and not .H; then you have to start explaining the underlying
implementation detail.  For users who already understand the
implementation detail, finding .T.conj() would not be too hard.

 (( I would not open the discussion about ndarray vs. matrix -- it gets
 far to involving and we would be talking about far-future directions
 instead of a single letter addition, which abvious already has big
 enough support and had so years ago))

I am willing to write up a NEP if there's any interest.  The plan
would be to remove the Matrix class from numpy over two or three
releases, and publish it as a separate package on PyPi.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question regarding documentation of structured arrays

[Stéfan van der Walt]

Hi Hanno

On Wed, Jul 24, 2013 at 11:46 AM, Hanno Klemm kl...@phys.ethz.ch wrote:
 I found the following inconsistency between the advertised and the
 actual behviour of structured arrays:

 on http://docs.scipy.org/doc/numpy/user/basics.rec.html it says in the
 section

 Accessing multiple fields at once
 Notice that the fields are always returned in the same order regardless
 of the sequence they are asked for.

I can confirm the behavior you see under the latest development
version.  Would you mind filing a pull request against the docs?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Question regarding documentation of structured arrays

[Stéfan van der Walt]

Hallo Hanno

On Wed, Jul 24, 2013 at 1:29 PM, Hanno Klemm kl...@phys.ethz.ch wrote:
 I would be happy to file a pull request against the docs if you (or
 somebody) could point me to a document on how and where to do that.

The file you want to edit is here:
https://github.com/numpy/numpy/blob/master/numpy/doc/structured_arrays.py#L194

You can click on the edit button, then GitHub will help you to make
a pull request.

Thanks!
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] add .H attribute?

[Stéfan van der Walt]

On Wed, Jul 24, 2013 at 12:54 PM, Nathaniel Smith n...@pobox.com wrote:
 The point isn't that there's code that relies specifically on .T
 returning a view. It's that to be a good programmer, you need to *know
 whether* it returns a view -- exactly as you say in the second
 paragraph. And a library should not hide these kinds of details.

After listening to the arguments by yourself and Dag, I think I buy
into the idea that we should hold off on this until we have ufunc
views or something similar implemented.

Also, if we split off the matrix package, we can give other people who
really care about that (perhaps Alan is interested?) ownership, and
let them run with it (I mainly use ndarrays myself).

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Splitting numpydoc to a separate repo

[Stéfan van der Walt]

On Wed, Jul 24, 2013 at 5:33 PM, Pauli Virtanen p...@iki.fi wrote:
 How about splitting doc/sphinxext out from the main Numpy repository to
 a separate `numpydoc` repo under Numpy project?

That would be great, also for scikits that rely on these extensions.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] add .H attribute?

[Stéfan van der Walt]

On Tue, Jul 23, 2013 at 10:35 AM, Dag Sverre Seljebotn
d.s.seljeb...@astro.uio.no wrote:
 So -1 on using A.H for anything but a proper view, and A.conjt() or
 something similar for a method that does a copy.

A.T.conj() is just as clear, so my feeling is that we should either
add A.H / A.H() or leave it be.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] .flat (was: add .H attribute?)

[Stéfan van der Walt]

On Tue, Jul 23, 2013 at 3:39 PM, Alan G Isaac alan.is...@gmail.com wrote:
 On 7/23/2013 9:09 AM, Pauli Virtanen wrote:
 .flat which I think
 is rarely used

 Until ``diagonal`` completes its transition,
 use of ``flat`` seems the best way to reset
 the diagonal on an array.  Am I wrong?
 I use it that way all the time.

I usually write

x[np.diag_indices_from(x)] = [1,2,3]

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] add .H attribute?

[Stéfan van der Walt]

On Tue, Jul 23, 2013 at 4:51 PM, Nathaniel Smith n...@pobox.com wrote:
 Don't know if we want to actually do this, but it's doable.

Would we need a matching conjugate data-type for each complex
data-type then, or can the data-type be parameterized?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Bringing order to higher dimensional operations

[Stéfan van der Walt]

On Sat, Jul 20, 2013 at 7:44 AM,  josef.p...@gmail.com wrote:
 related: is there any advantage to np.add.reduce?
 I find it more difficult to read than sum() and still see it used sometimes.

I think ``np.add.reduce`` just falls out of the ufunc
implementation--there's no per ufunc choice to remove certain parts
of the API, if I recall correctly.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] User Guide

[Stéfan van der Walt]

On Fri, Jul 19, 2013 at 12:21 AM, Rob Clewley rob.clew...@gmail.com wrote:
 The exception: one can have arrays of python objects, including numpy
 objects, which allows arrays to contain different sized elements.

What are numpy objects?  numpy objects - numpy ndarrays or numpy
ndarray objects?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Bringing order to higher dimensional operations

[Stéfan van der Walt]

On Thu, Jul 18, 2013 at 2:52 PM, Nathaniel Smith n...@pobox.com wrote:
 Compare:
   gu_dot_leftwards(ones((10, 11, 4)), ones((11, 12, 3, 4))) - (10, 12, 3, 4)
 versus
   gu_dot_rightwards(ones((4, 10, 11)), ones((3, 4, 11, 12))) - (3, 4, 10, 12)

The second makes quite a bit more sense to me, and fits with the
current way we match broadcasting dimensions (align to the right,
match right to left).

The np.dot outer example you gave and other exceptions like that will
probably give us headaches in the future, so I'd opt for moving away
from them.  The way ellipses are broadcast, well that's a battle for
another day.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Bringing order to higher dimensional operations

[Stéfan van der Walt]

On Fri, Jul 19, 2013 at 5:31 PM, Nathaniel Smith n...@pobox.com wrote:
 3) Extend the gufunc machinery to understand the idea that some core
 dimensions are allowed to take on a special nonexistent size. So the
 signature for dot would be:
   (m*,k) x (k, n*) - (m*, n*)
 where '*' denotes dimensions who are allowed to take on the
 nonexistent size if necessary. So dot(ones((2, 3)), ones((3, 4)))
 would have
   m = 2
   k = 3
   n = 4
 and produce an output with shape (m, n) = (2, 4). But dot(ones((2,
 3)), ones((3,))) would have
   m = 2
   k = 3
   n = nothing
 and produce an output with shape (m, n) = (2, nothing) = (2,). And
 dot(ones((3,)), ones((3,))) would have
   m = nothing
   k = 3
   n = nothing
 and produce an output with shape (m, n) = (nothing, nothing) = (),
 i.e., dot(vector, vector) would return a scalar.

This looks like a fairly clean solution; it could be implemented in a
shape pre- and post-processing step, where we pad the array dimensions
to match the full signature, and remove it again afterwards.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] add .H attribute?

[Stéfan van der Walt]

On Sat, Jul 13, 2013 at 7:46 PM, Nathaniel Smith n...@pobox.com wrote:
 Why not just write

 def H(a):
 return a.conj().T

It's hard to convince students that this is the Best Way of doing
things in NumPy.  Why, they ask, can you do it using a' in MATLAB,
then?

I've tripped over this one before, since it's not the kind of thing
you imagine would be unimplemented, and then spend some time trying to
find it.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] azip

[Stéfan van der Walt]

On Thu, Jul 18, 2013 at 7:06 PM, Alan G Isaac alan.is...@gmail.com wrote:
 On 7/18/2013 1:03 PM, Robert Kern wrote:
 np.column_stack([x, b]) does everything you need.

 So it does.

 It's not referenced from the hstack or concatenate documentation.

A pull request would fix all of that in seconds!  GitHub now allows
online editing, and provides a one-click option for creating the PR.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] azip

[Stéfan van der Walt]

Hi Ben

On Thu, Jul 18, 2013 at 7:18 PM, Benjamin Root ben.r...@ou.edu wrote:
 Forgive my ignorance, but has numpy and scipy stopped doing that weird doc
 editing thing that existed back in the days of Trac? I have actually held
 back on submitting doc edits because I hated using that thing so much.

That thing helps people without hacking experience to contribute, but
you are welcome to issue pull-requests instead.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] PIL and NumPy

[Stéfan van der Walt]

Dear Brady

On Fri, 12 Jul 2013 22:00:08 -0500, Brady McCary wrote:

 I want to load images with PIL and then operate on them with NumPy.
 According to the PIL and NumPy documentation, I would expect the
 following to work, but it is not.

Reading images as PIL is a little bit trickier than one would hope.  You can
find an example of how to do it (taken scikit-image) here:

https://github.com/scikit-image/scikit-image/blob/master/skimage/io/_plugins/pil_plugin.py#L15

Stéfan

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] What should be the result in some statistics corner cases?

[Stéfan van der Walt]

On Mon, 15 Jul 2013 08:33:47 -0600, Charles R Harris wrote:
 On Mon, Jul 15, 2013 at 8:25 AM, Benjamin Root ben.r...@ou.edu wrote:
 
  This is going to need to be heavily documented with doctests. Also, just
  to clarify, are we talking about a ValueError for doing a nansum on an
  empty array as well, or will that now return a zero?
 
 
 I was going to leave nansum as is, as it seems that the result was by
 choice rather than by accident.

That makes sense--I like Sebastian's explanation whereby operations that
define an identity yields that upon empty input.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] What should be the result in some statistics corner cases?

[Stéfan van der Walt]

On Mon, 15 Jul 2013 18:46:33 -0600, Charles R Harris wrote:
 
 So nansum should return zeros rather than the current NaNs?

Yes, my feeling is that nansum([]) should be 0.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] read-only or immutable masked array

[Stéfan van der Walt]

On Fri, Jul 12, 2013 at 4:41 PM, Gregorio Bastardo
gregorio.basta...@gmail.com wrote:
 array.flags.writeable = False

 is perfectly fine, but it does not work on ma-s. Moreover, mask
 hardening only protects masked elements, and does not raise error (as
 I'd expect).

You probably have to modify the underlying array and mask:

x = np.ma.array(...)
x.mask.flags.writeable = False
x.data.flags.writeable = False

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] np.ma.argmax not respecting the mask?

[Stéfan van der Walt]

On Tue, Jul 9, 2013 at 2:55 PM, Chao YUE chaoyue...@gmail.com wrote:
 I am using 1.7.1 version of numpy and np.ma.argmax is not repecting the
 mask?

 In [96]: d3
 Out[96]:
 masked_array(data =
  [[-- -- -- -- 4]
  [5 -- 7 8 9]],
  mask =
  [[ True  True  True  True False]
  [False  True False False False]],
fill_value = 6)


 In [97]: np.ma.argmax(d3,axis=0)
 Out[97]: array([1, 0, 1, 1, 1])

This is the result I would expect.  If both values are masked, the
fill value is used, so there is always an argmin value.

The following workaround should have done the trick, but it exposes a
different bug:

x = np.ma.array([[0,1,2,3,4],[5,6,7,8, 9]], mask=[[1, 1, 1, 1, 0], [0,
1, 0, 0 ,0]], dtype=float)
np.nanargmax(x.filled(np.nan), axis=0)

This breaks with ValueError: cannot convert float NaN to integer

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] FWIW: regressions of dependees of nukmpy 1.7.0b1

[Stéfan van der Walt]

On Wed, Sep 5, 2012 at 1:38 PM, Yaroslav Halchenko li...@onerussian.com wrote:
 skimage_0.6.1-1.dscok   FAILED

This breakage is due to

https://github.com/numpy/numpy/issues/392

Thanks for checking!

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Buildbot status

[Stéfan van der Walt]

On Sun, Jul 8, 2012 at 3:44 PM, Chris Ball s0454...@sms.ed.ac.uk wrote:
 I'm happy to help connect slaves to the EC2 buildbot if someone sends
 connection details to me.

Thanks very much, Chris.  Matthew gave us access to the NiPy build
slaves, so catch me online then we can hook those up.

 Another thing to consider is where to send test results. Travis-CI
 already comments on pull requests, which is great; the other solutions
 could be set up to email a list (or even the individuals potentially
 responsible for causing test failures).

If we notify the individuals, for now, that's fine.  Others can keep
track by looking at the buildbot homepage.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Buildbot status

[Stéfan van der Walt]

On Thu, Jul 5, 2012 at 4:36 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote:
 So feel free to go ahead with what you think is the best and I will join you
 in a few days.

I propose that we following a simple migration path for now: move the
current buildbot onto the EC2 instance, redirect buildbot.scipy.org,
and then connect the nipy build slaves.  This should require minimal
effort, but provide us with fairly wide coverage until you can invest
more time in Jenkins etc.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Buildbot status

[Stéfan van der Walt]

On Mon, Jul 2, 2012 at 6:31 PM, Travis Oliphant tra...@continuum.io wrote:
 Ondrej should have time to work on this full time in the coming days.

That's great; having Ondrej on this full time will help a great deal.

 NumFocus can provide some funding needed for maintaining servers, etc, but 
 keeping build bots active requires the efforts of multiple volunteers.
 If anyone has build machines to offer, please let Ondrej know so he can 
 coordinate getting Jenkins slaves onto them and hooking them up to the master.

I'd be glad if we could discuss it mainly on list, just to keep
everyone in the loop.

For now, I think we need to answer the two questions mentioned above:

1) What happens to the current installation on buildbot.scipy.org?
2) If we're not keeping buildbot, or if we want additional systems,
which ones should we use? Jenkins?

and then also

3) Which build slaves should we employ?  We have the current build
slaves, the nipy ones have been volunteered, and then there's the GCC
build farm mentioned by Fernando.

Ondrej, perhaps you can comment on what you had in mind?  If we have a
clear plan of action before you start off, we can all help out in
putting the pieces together.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Buildbot status

[Stéfan van der Walt]

Hi all,

I'd like to find out what the current status of continuous integration
is for numpy.  I'm aware of:

a) http://buildbot.scipy.org -- used by Ralf for testing releases?
b) http://travis-ci.org -- connected via GitHub
c) http://184.73.247.160:8111 -- dedicated Amazon EC2 with TeamCity
d) http://build.pydata.org:8111/ -- dedicated Rackspace instance with TeamCity
e) https://jenkins.shiningpanda.com/numpy/ -- python 2.4 on Debian

Is there interest in maintaining the buildbot setup?  If so, I suggest
we move (a) onto (c), and then connect in several of the NiPy
buildbots [including a Raspberry Pi!] (offered by Matthew Brett).

It'd be nice to have a semi-official set up, so that we don't
duplicate too much effort.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Buildbot status

[Stéfan van der Walt]

On Mon, Jul 2, 2012 at 5:31 PM, Ondřej Čertík ondrej.cer...@gmail.com wrote:
 Yes, definitely. I will have time to work on the tests in about 2 weeks.
 Could you coordinate with Travis? He can make it official.

I'd gladly coordinate with everyone, but I'd like to do it here on the
mailing list so that we're on the same page.  Numerous parties have
spoken about this informally, but now there are multiple efforts that
I'd like to consolidate.

Here are some questions we need to answer:

1) Do we keep the current buildbot.scipy.org?

I suggest that we move it to the EC2 machine, if we do.  Both Chris
Ball and Matthew Brett has set up fairly sophisticated buildbots
before, so we can leverage their knowledge.

2) Do we switch to another system, such as Jenkins?  It seems as
though you've investigated some of those alternatives.  Did you also
look at TeamCity?

If anyone needs access to the EC2 machine, just let me know.

Regards
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] SciPy2012 conference: Last week for early birds, poster submissions

[Stéfan van der Walt]

Hi everyone

We're rapidly approaching SciPy2012 http://conference.scipy.org/scipy2012,
which takes place in Austin, Texas from July 16th to 21st.  This is a
reminder that the *discounted early bird registration* closes on the 18th
of this month.

Also, we decided to keep the queue for *poster submissions* open until all
slots are filled. So, whether you have a neat side project, a lightning
talk gone rogue, or simply want to get the community talking about your
latest and greatest idea--send in a poster abstract to
2012submissi...@scipy.org.

See you in Austin!
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [enhancement] sum_angle() and sum_polar()

[Stéfan van der Walt]

On Mon, May 28, 2012 at 11:53 AM, Travis Oliphant tra...@continuum.io wrote:
 I could see these functions going into scipy.ndimage but again because they
 are not necessarily just image processing functions, and the fact that they
 are so simple, perhaps they are best put into NumPy itself.

I'm wondering about the general applicability of these functions.  Can
anyone suggest some use cases?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] tracing numpy data allocation with python callbacks

[Stéfan van der Walt]

On Wed, May 16, 2012 at 12:34 PM, Thouis Jones thouis.jo...@curie.fr wrote:
 I wondered, however, if there were a better way to accomplish the same
 goal, preferably in pure python.

Fabien recently posted this; not sure if it addresses your use case:

http://fseoane.net/blog/2012/line-by-line-report-of-memory-usage/

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Fancy-indexing reorders output in corner cases?

[Stéfan van der Walt]

Hi Zach

On Mon, May 14, 2012 at 4:33 PM, Zachary Pincus zachary.pin...@yale.edu wrote:
 The below seems to be a bug, but perhaps it's unavoidably part of the 
 indexing mechanism?

 It's easiest to show via example... note that using [0,1] to pull two 
 columns out of the array gives the same shape as using :2 in the simple 
 case, but when there's additional slicing happening, the shapes get 
 transposed or something.

When fancy indexing and slicing is mixed, the resulting shape is
essentially unpredictable.  The correct way to do it is to only use
fancy indexing, i.e. generate the indices of the sliced dimension as
well.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] copying array to itself

[Stéfan van der Walt]

On Thu, May 3, 2012 at 1:51 AM, Henry Gomersall h...@cantab.net wrote:
 Right, so this is expected behaviour then. Is this documented somewhere?
 It strikes me that this is pretty unexpected behaviour.

Imagine the way you would code this in a for-loop.  You want

a = np.arange(10)
a[2:] = a[:-2]

Now you write:

for i in range(2, len(a)):
a[i] = a[i - 2]

which yields

[0, 1, 0, 1, 0, 1, 0, 1, 0, 1]

One of the great things about NumPy is that it only copies data when
it really has to, and in this case it would need to be very clever to
figure out what you are trying to do.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] record arrays and vectorizing

[Stéfan van der Walt]

On Wed, May 2, 2012 at 11:06 AM, Moroney, Catherine M (388D)
catherine.m.moro...@jpl.nasa.gov wrote:
 I will want to compare a 7-element vector (called element) to a large list 
 of similarly-dimensioned
 vectors (called target, and pick out the vector in target that is the 
 closest to element
 (determined by minimizing the Euclidean distance).

It's not entirely clear what you mean from the description above.  In
the code example, you return a single index, but from the description
it sounds like you want to pick out a vector?

If you need multiple answers, one for each element, then you probably
need to do broadcasting as shown in the NumPy medkit:

http://mentat.za.net/numpy/numpy_advanced_slides/

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] copying array to itself

[Stéfan van der Walt]

On Wed, May 2, 2012 at 9:03 AM, Henry Gomersall h...@cantab.net wrote:
 Is this some nuance of the way numpy does things? Or am I missing some
 stupid bug in my code?

Try playing with the parameters of the following code:

sz = 1
N = 10

import numpy as np

x = np.arange(sz)
y = x.copy()
x[:-N] = x[N:]

np.testing.assert_equal(x[:-N], y[N:])


For small values of sz this typically works, but as soon as numpy
needs to buffer strange things happen because you are reading from
memory locations that you've already written to.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sparse array data

[Stéfan van der Walt]

Hi Francesc

On Wed, May 2, 2012 at 1:53 PM, Francesc Alted franc...@continuum.io wrote:
 and add another one for the actual values of the array.  For a 3-D
 sparse array, this looks like:

 dim0 | dim1 | dim2 | value
 ==
    0 |   0  |   0  | val0
    0 |  10  | 100  | val1
   20 |   5  | 202  | val2

What's the distinction between this and a coo_matrix?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] record arrays initialization

[Stéfan van der Walt]

On Wed, May 2, 2012 at 2:45 PM, Moroney, Catherine M (388D)
catherine.m.moro...@jpl.nasa.gov wrote:
 Find the best match in an array t(5000, 7) for a single vector e(7).  Now 
 scale
 it up so e is (128, 512, 7) and I want to return a (128, 512) array of the 
 t-identifiers
 that are the best match for e.  Best match is defined as the minimum 
 Euclidean distance.

 I'm going to try three ways: (a) brute force and lots of looping in python,
 (b) constructing a function to find the match for a single instance of e and
 vectorizing it, and (c) coding it in Fortran.  I'll be curious to see the
 performance figures.

I'd use a mixture of (a) and (b):  break the t(N, 7) up into blocks
of, say, (1000, 7), compute the best match in each using broadcasting,
and then combine your results to find the best of the best.  This
strategy should be best for very large N.  For moderate N, where
broadcasting easily fits into memory, the answer given by the OP to
your original email would do the trick.

 A)  How do I most efficiently construct a record array from a single array?
 I want to do the following, but it segfaults on me when i try to print b.

 vtype = [(x, numpy.ndarray)]
 a = numpy.arange(0, 16).reshape(4,4)
 b = numpy.recarray((4), dtype=vtype, buf=a)

I prefer not to use record arrays, and stick to structured arrays:

In [11]: vtype = np.dtype([('x', (np.float, 4))])

In [12]: a = np.arange(16.).reshape((4,4))

In [13]: a.view(vtype)
Out[13]:
array([[([0.0, 1.0, 2.0, 3.0],)],
   [([4.0, 5.0, 6.0, 7.0],)],
   [([8.0, 9.0, 10.0, 11.0],)],
   [([12.0, 13.0, 14.0, 15.0],)]],
  dtype=[('x', 'f8', (4,))])

 B)  If I'm vectorizing a function (single) to find the best match for
 a single element of e within t, how do I pass the entire array t into
 the function without having it parcelled down to its individual elements?

I think the new dtype just makes your life more difficult here.  Simply do:

In [49]: np.sum(a - elements.T, axis=1)
Out[49]: array([  0.,  16.,  32.,  48.])

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sparse array data

[Stéfan van der Walt]

On Wed, May 2, 2012 at 3:20 PM, Francesc Alted franc...@continuum.io wrote:
 On 5/2/12 4:07 PM, Stéfan van der Walt wrote:
 Well, as the OP said, coo_matrix does not support dimensions larger than
 2, right?

That's just an implementation detail, I would imagine--I'm trying to
figure out if there is a new principle behind synthetic dimensions?
By the way, David Cournapeau mentioned using b-trees for sparse ops a
while ago; did you ever talk to him about those ideas?

BTW, this coo-type storage is used in Stanford's probabilistic
graphical models course, but for dense data (like we have in the
course) it's a pain.  Writing code in both Octave and Python, I again
came to realize what a very elegant N-dimensional structure the numpy
array exposes!

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] record arrays initialization

[Stéfan van der Walt]

On Wed, May 2, 2012 at 4:26 PM, Moroney, Catherine M (388D)
catherine.m.moro...@jpl.nasa.gov wrote:
 Using structured arrays is making my code complex when I try to call the
 vectorized function.  If I stick to the original record arrays, what's the
 best way of initializing b from a without doing an row-by-row copy?

What does your original data look like?  It seems like `a` is already
what you need after the reshape?

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] record arrays initialization

[Stéfan van der Walt]

On Wed, May 2, 2012 at 4:46 PM, Kevin Jacobs jac...@bioinformed.com
bioinfor...@gmail.com wrote:
 A FLANN implementation should be even faster--perhaps by as much as another
 factor of two.

I guess it depends on whether you care about the Approximate in
Fast Library for Approximate Nearest Neighbors.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sparse array data

[Stéfan van der Walt]

On Wed, May 2, 2012 at 6:25 PM, Travis Oliphant tra...@continuum.io wrote:
 The only new principle (which is not strictly new --- but new to NumPy's 
 world-view) is using one (or more) fields of a structured array as synthetic 
 dimensions which replace 1 or more of the raw table dimensions.

Ah, thanks--that's the detail I was missing.  I wonder if the
contiguity requirement will hamper us here, though.  E.g., I could
imagine that some tree structure might be more suitable to storing and
organizing indices, and for large arrays we wouldn't like to make a
copy for each operation.  I guess we can't wait for discontiguous
arrays to come along, though :)

 More to come  If you are interested in this sort of thing please let me 
 know

Definitely--if we can optimize this machinery it will be beneficial to
scipy.sparse as well.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] What is consensus anyway

[Stéfan van der Walt]

On Tue, Apr 24, 2012 at 11:12 AM, Charles R Harris
charlesr.har...@gmail.com wrote:
 The advantage of nans, I suppose, is that they are in the hardware and so

Why are we having a discussion on NAN's in a thread on consensus?
This is a strong indicator of the problem we're facing.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] What is consensus anyway

[Stéfan van der Walt]

On Tue, Apr 24, 2012 at 2:25 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
 Why are we having a discussion on NAN's in a thread on consensus?
 This is a strong indicator of the problem we're facing.

 We seem to have a consensus regarding interest in the topic.

For the benefit of those of us interested in both discussions, would
you kindly start a new thread on the MA topic?

In response to Travis's suggestion of writing up a short summary of
community principles, as well as Matthew's initial formulation, I
agree that this would be helpful in enshrining the values we cherish
here, as well as in communicating those values to the next generation
of developers.

From observing the community, I would guess that these values include:

- That any party with an interest in NumPy is given the opportunity to
speak and to be heard on the list.
- That discussions that influence the course of the project take place
openly, for anyone to observe.
- That decisions are made once consensus is reached, i.e., if everyone
agrees that they can live with the outcome.

To summarize: NumPy development that is free  fair, open and unified.

We'll sometimes mess up and not follow our own guidelines, but with
them in place at least we'll have something to refer back to as a
reminder.

Regards
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] What is consensus anyway

[Stéfan van der Walt]

On Tue, Apr 24, 2012 at 4:49 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
 But a right to veto doesn't automatically extend to everyone who happens to 
 have
 an interest in a topic.

The time has long gone when we simply hacked on NumPy for our own
benefit; if you will, NumPy users are our customers, and they have a
stake in its development (or, to phrase it differently, I think we
have a commitment to them).

If we strongly encourage people to discuss, but still give them an
avenue to object, we keep ourselves honest (both w.r.t. expectations
on numpy and our own insight into problems and their solutions).

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] What is consensus anyway

[Stéfan van der Walt]

On Mon, Apr 23, 2012 at 4:39 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
 I'm starting to think that a fork might be the best solution to the present
 problem.

If you are referring to the traditional concept of a fork, and not to
the type we frequently make on GitHub, then I'm surprised that no one
has objected already.  What would a fork solve? To paraphrase the
regexp saying: after forking, we'll simply have two problems.

It's really not that hard to focus our attention on technical issues
and to reach consensus.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] draft enum NEP

[Stéfan van der Walt]

On Thu, Mar 15, 2012 at 4:02 PM, Nathaniel Smith n...@pobox.com wrote:
 I'm not sure what it would even mean to treat this kind of data as
 flags, since you can't take the bitwise-or of two strings...

This makes a more sense outside of ndarrays, where you would do something like:

enum FLAG0 = 1, FLAG1 = 2, FLAG2 = 4
do_something(data, mode=FLAG0  FLAG2)

The enum is therefore just a handle for its numerical value.  While it
may not be that useful in an array, I think Mark was just pointing out
that there may be other similar use cases, such as enumerating from 0
to N-1, or in reverse from N-1 down to 0, or in steps of 2, or in
powers of 2, etc.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] np.longlong casts to int

[Stéfan van der Walt]

On Wed, Feb 22, 2012 at 2:47 PM, Matthew Brett matthew.br...@gmail.com wrote:
 In [4]: np.array([2.1], dtype=np.longlong)
 Out[4]: array([2], dtype=int64)

Maybe just a typo:

In [3]: np.array([2.1], dtype=np.longfloat)
Out[3]: array([ 2.1], dtype=float128)

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Stéfan van der Walt]

On Mon, Feb 20, 2012 at 1:54 AM, Pauli Virtanen p...@iki.fi wrote:
 20.02.2012 08:35, Paul Anton Letnes kirjoitti:
 In the language wars, I have one question.
 Why is Fortran not being considered?

 Fortran is OK for simple numerical algorithms, but starts to suck
 heavily if you need to do any string handling, I/O, complicated logic,
 or data structures.

Out of curiosity, is this still true for the latest Fortran versions?
I guess there the problem may be compiler support over various
platforms.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How a transition to C++ could work

[Stéfan van der Walt]

Hey, Mark

On Feb 18, 2012 11:18 PM, Mark Wiebe mwwi...@gmail.com wrote:
 My experience has been that providing a C API from a C++ library is no
harder than providing a C API from a C library.

Interfacing to compiled C++ libs have been tricky, so can this concern be
dismissed so easily? (Some examples that came to mind were
_import_array--easy to fix because it is ours, I guess--or Cython generated
code).

 A really important point to emphasize is that C++ allows for a strategy
where we gradually evolve the codebase to better incorporate its language
features. This is what I'm advocating. No massive rewrite, no disruptive
changes. Gradual code evolution, with ABI and API compatibility comparable
to what we've delivered in 1.6 and the upcoming 1.7 releases.

If we're to switch to C++ (a language that can very easily be wielded in
terrible ways), then this certainly seems like a sound approach.

Regards
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Stéfan van der Walt]

On Feb 19, 2012 12:09 AM, Mark Wiebe mwwi...@gmail.com wrote:

 These standard library issues were definitely valid 10 years ago, but all
the major C++ compilers have great C++98 support now. Is there a specific
target platform/compiler combination you're thinking of where we can do
tests on this? I don't believe the compile times are as bad as many people
suspect, can you give some simple examples of things we might do in NumPy
you expect to compile slower in C++ vs C?

The concern may be more that this will be an issue once we start templating
(scipy.sparse as an example). Compiling templates requires a lot of memory
(more than with the current Heath Robbinson solution).

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] How a transition to C++ could work

[Stéfan van der Walt]

On Feb 19, 2012 2:41 AM, Mark Wiebe mwwi...@gmail.com wrote:

 This is the role I see good coding standards and consistent code review
playing. Programmers who don't know how to write good C++ code can be
taught. There are also good books to read, like C++ Coding Standards,
Effective C++, and others that can help people learn proper technique.

I recommended this book (one in the list avove) to anyone who is not afraid
of C++ yet:

http://search.barnesandnoble.com/Effective-C/Scott-Meyers/e/9780321334879

With great power comes great responsibility.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Proposed Roadmap Overview

[Stéfan van der Walt]

On Feb 19, 2012 4:14 PM, Sturla Molden stu...@molden.no wrote:

 Den 20.02.2012 00:39, skrev Nathaniel Smith:
  But there's an order-of-magnitude difference in compile times between
  most real-world C projects and most real-world C++ projects. It might
  not be a deal-breaker and it might not apply for subset of C++ you're
  planning to use, but AFAICT that's the facts.

 This is mainly a complaint about the build-process.

This has nothing to do with the build process. More complex languages take
longer to compile. The benchmark shown is also entirely independent of
build system.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Buildbot/continuous integration (was Re: Issue Tracking)

[Stéfan van der Walt]

On Thu, Feb 16, 2012 at 3:52 PM, Chris Ball ceb...@gmail.com wrote:
 After getting to this initial stage, I'll discuss about adding more
 features (such as testing pull requests, performance testing, building
 binaries on the different operating systems, etc). Also, if it's working
 well, this Buildbot setup could replace/be merged with the one at
 buildbot.scipy.org (I don't know who is currently running that).

That machine is up and running at Stellenbosch University; the main
problem is that it is behind a firewall, and we've had some issues
working around that.  I can gladly share the current configuration,
and the instructions I send out to the maintainers of slave machines
 (I agree--putting it in a Git repo--minus passwords!--is a good
idea).

At the moment, there are three active build slaves: one Windows XP and
two different setups of Linux Sparc.  To trigger a build, click on the
machine name and then Force Build.

Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] test errors on deprecation/runtime warnings

[Stéfan van der Walt]

Hi Ralf

On Thu, Feb 16, 2012 at 11:05 AM, Ralf Gommers
ralf.gomm...@googlemail.com wrote:
 Last week we merged https://github.com/numpy/numpy/pull/201, which causes
 DeprecationWarning's and RuntimeWarning's to be converted to errors if they
 occur when running the test suite.

It looks like this change affects other packages, too, which may
legitimately raise RuntimeWarnings while running their test suites
(unless I read the patch wrong).  Would it be an option to rather add
a flag (False by default) to enable this behaviour, and enable it
inside of numpy.test() ?

Regards
Stéfan
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Change in scalar upcasting rules for 1.6.x?

[Stéfan van der Walt]

On Tue, Feb 14, 2012 at 12:05 AM, Eric Firing efir...@hawaii.edu wrote:
 On 02/13/2012 08:07 PM, Charles R Harris wrote:



 Let it go, Travis. It's a waste of time.

 (Off-list) Chuck, I really appreciate your consistent good sense; this
 is just one of many examples.  Thank you for all your numpy work.

Not off list.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] trapezoidal grids

[Stéfan van der Walt]

On Fri, Feb 10, 2012 at 9:26 AM, Hugo Gagnon
sourceforge.nu...@user.fastmail.fm wrote:
 Hello,

 Say I have four corner points a = (X0, Y0), b = (X1, Y1), c = (X2, Y2)
 and d = (X3, Y3):

 a--b
  \        /
  \      /
   cd

 Is there a function like meshgrid that would return me a grid of points
 linearly interpolating those four corner points?

It depends on what you mean by linearly interpolating.  For example,
you can construct a regular grid and simply discard points outside the
trapesium, or assume that the trapesium is a warped perspective on a
regular grid.  For the latter case, you can construct a grid like
this:

http://mentat.za.net/refer/np_warped_grid.png

(code attached)

Stéfan
import numpy as np
import matplotlib.pyplot as plt

rx, ry = np.array([[0, 0],
   [1, 0],
   [0, 1],
   [1, 1]]).T

tx, ty = np.array([[7, 10],
   [20, 10],
   [4, 20],
   [25, 20]]).T

N = len(rx)
U = np.zeros((2 * N, 8))

U[:N, 0] = rx
U[:N, 1] = ry
U[:N, 2] = 1
U[:N, 6] = -rx * tx
U[:N, 7] = -ry * tx

U[N:, 3] = rx
U[N:, 4] = ry
U[N:, 5] = 1
U[N:, 6] = -rx * ty
U[N:, 7] = -ry * ty

b = np.concatenate((tx, ty))[:, np.newaxis]

M, res, rank, s = np.linalg.lstsq(U, b)
M = np.append(M, 1).reshape((3, 3))

y, x = np.mgrid[:1:10j, :1:10j]
I = np.ones(x.size)
square_grid = np.column_stack((x.ravel(), y.ravel(), I))
warp_grid = M.dot(square_grid.T).T
warp_grid /= warp_grid[:, 2][:, None]

plt.scatter(tx, ty, c='b', s=50)
plt.scatter(warp_grid[:, 0], warp_grid[:, 1], c='r', marker='x')
plt.show()

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion