Re: [Numpy-discussion] Willing to contribute to SciPy NumPy ...
On 31 March 2011 07:27, Sylvain Bellemare sbel...@gmail.com wrote: I would like to seriously start contributing to NumPy and/or SciPy, as much as I possibly can. I'm sure that your help will be welcomed! A good place to get started is helping out with documentation (see http://docs.scipy.org/numpy/Front%20Page/). SciPy has plenty of work required - you'll probably learn your way into the code that way. Another place to look is http://www.scipy.org/Developer_Zone. It's worthwhile learning how to work with Git and Github if you want to get patches accepted more easily. Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: Numpy 1.6.0 beta 1
On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen ro...@uw.edu wrote: In article AANLkTi=eeg8kl7639imrtl-ihg1ncqyolddsid5tf...@mail.gmail.com, Ralf Gommers ralf.gomm...@googlemail.com wrote: Hi, I am pleased to announce the availability of the first beta of NumPy 1.6.0. Due to the extensive changes in the Numpy core for this release, the beta testing phase will last at least one month. Please test this beta and report any problems on the Numpy mailing list. Sources and binaries can be found at: http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/ For (preliminary) release notes see below. I see a segfault on Ubuntu 64 bits for the test TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py. Am I the only one seeing it ? David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] np.histogram on arrays.
Hi, On Wed, Mar 30, 2011 at 1:44 PM, Éric Depagne e...@depagne.org wrote: Well I guess, for a slight performance improvement, you could create your own streamlined histogrammer. But, in order to better grasp your situation it would be beneficial to know how the counts and bounds are used later on. Just wondering if this kind massive histogramming could be somehow avoided totally. Indeed. Here's what I do. My images come from CCD, and as such, the zero level in the image is not the true zero level, but is the true zero + the background noise of each pixels. By doing the histogram, I plan on detecting what is the most common value per row. Once I have the most common value, I can derive the interval where most of the values are (the index of the largest occurence is easily obtained by sorting the counts, and I take a slice [index_max_count,index_max_count+1] in the second array given by the histogram). Then, I take the mean value of this interval and I assume it is the value of the bias for my row. I do this procedure both on the row and columns as a sanity check. And I know this procedure will not work if on any row/column there is a lot of signal and very little bias. I'll fix that afterwards ;-) Perhaps something along these lines will help you: from numpy import histogram, linspace, r_ def hist2(a, nob): bins= linspace(a.min(), a.max(), nob+ 1) d= linspace(0, bins[-1]* a.shape[0], a.shape[0])[:, None] b= (a+ d).ravel() bbins= (bins[:-1]+ d).ravel() bbins= r_[bbins, bbins[-1]+ 1] counts, _= histogram(b, bbins) return counts.reshape(-1, nob), bins It has two disadvantages 1) needs more memory and 2) global bins (which although should be quite straightforward to enhance if needed). Regards, eat Éric. Regards, eat Un clavier azerty en vaut deux -- Éric Depagnee...@depagne.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] random number genration
thanks Sturl, your second code works well although I had to divide the vector elements through 128 to get just the binaries ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1
On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau courn...@gmail.comwrote: On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen ro...@uw.edu wrote: In article AANLkTi=eeg8kl7639imrtl-ihg1ncqyolddsid5tf...@mail.gmail.com, Ralf Gommers ralf.gomm...@googlemail.com wrote: Hi, I am pleased to announce the availability of the first beta of NumPy 1.6.0. Due to the extensive changes in the Numpy core for this release, the beta testing phase will last at least one month. Please test this beta and report any problems on the Numpy mailing list. Sources and binaries can be found at: http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/ For (preliminary) release notes see below. I see a segfault on Ubuntu 64 bits for the test TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py. Am I the only one seeing it ? The test work here ok on Ubuntu 64 with numpy master. Could you try the maintenance/1.6.x branch where the related bugs are fixed. Pearu ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1
On 31 March 2011 11:37, Pearu Peterson pearu.peter...@gmail.com wrote: On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau courn...@gmail.com wrote: On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen ro...@uw.edu wrote: In article AANLkTi=eeg8kl7639imrtl-ihg1ncqyolddsid5tf...@mail.gmail.com, Ralf Gommers ralf.gomm...@googlemail.com wrote: Hi, I am pleased to announce the availability of the first beta of NumPy 1.6.0. Due to the extensive changes in the Numpy core for this release, the beta testing phase will last at least one month. Please test this beta and report any problems on the Numpy mailing list. Sources and binaries can be found at: http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/ For (preliminary) release notes see below. I see a segfault on Ubuntu 64 bits for the test TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py. Am I the only one seeing it ? The test work here ok on Ubuntu 64 with numpy master. Could you try the maintenance/1.6.x branch where the related bugs are fixed. For what it's worth, the maintenance/1.6.x branch works for me on 64-bit Ubuntu: (numpy-1.6.x)scott@godzilla:~$ python -c import numpy; numpy.test() Running unit tests for numpy NumPy version 1.6.0b2.dev-a172fd6 NumPy is installed in /home/scott/.virtualenvs/numpy-1.6.x/lib/python2.6/site-packages/numpy Python version 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5] nose version 1.0.0 snip... -- Ran 3406 tests in 16.889s OK (KNOWNFAIL=3, SKIP=4) Cheers, Scott ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1
On Thu, Mar 31, 2011 at 1:00 PM, Scott Sinclair scott.sinclair...@gmail.com
wrote:
On 31 March 2011 11:37, Pearu Peterson pearu.peter...@gmail.com wrote:
On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau courn...@gmail.com
wrote:
On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen ro...@uw.edu wrote:
In article
AANLkTi=eeg8kl7639imrtl-ihg1ncqyolddsid5tf...@mail.gmail.com,
Ralf Gommers ralf.gomm...@googlemail.com wrote:
Hi,
I am pleased to announce the availability of the first beta of NumPy
1.6.0. Due to the extensive changes in the Numpy core for this
release, the beta testing phase will last at least one month. Please
test this beta and report any problems on the Numpy mailing list.
Sources and binaries can be found at:
http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/
For (preliminary) release notes see below.
I see a segfault on Ubuntu 64 bits for the test
TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py.
Am I the only one seeing it ?
The test work here ok on Ubuntu 64 with numpy master. Could you try the
maintenance/1.6.x branch where the related bugs are fixed.
For what it's worth, the maintenance/1.6.x branch works for me on 64-bit
Ubuntu:
(numpy-1.6.x)scott@godzilla:~$ python -c import numpy; numpy.test()
You might want to run
python -c import numpy; numpy.test('full')
as the corresponding test is decorated as slow.
Pearu
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] np.histogramdd of empty data
Hi Ralf, I cloned numpy/master and played around a little. when giving the bins explicitely, now histogram2d and histogramdd work as expected in all tests i tried. However, some of the cases with missing bin specification appear somewhat inconsistent. The first question is if creating arbitrary bins for empty data and empty bin specification is better than raising an Exception: Specifically: numpy.histogram2d([],[],bins=[0,0]) (array([ 0., 0.]), array([ 0.]), array([ 0.])) numpy.histogram([],bins=0) ValueError: zero-size array to minimum.reduce without identity so 1-d and 2-d behave not quite the same. also, these work (although with arbitrary bin edges): numpy.histogram2d([],[],bins=[1,1]) (array([ 0., 0.]), array([ 0., 1.]), array([ 0., 1.])) numpy.histogram2d([],[],bins=[0,1]) (array([ 0., 0.]), array([ 0.]), array([ 0., 1.])) while this raises an error: numpy.histogram([],bins=1) ValueError: zero-size array to minimum.reduce without identity another thing with non-empty data: numpy.histogram([1],bins=1) (array([1]), array([ 0.5, 1.5])) numpy.histogram([1],bins=0) (array([], dtype=int64), array([ 0.5])) while numpy.histogram2d([1],[1],bins=A) ValueError: zero-size array to minimum.reduce without identity (here A==[0,0] or A==[0,1] but not A==[1,1] which gives a result) Nils ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1
On 31 March 2011 12:18, Pearu Peterson pearu.peter...@gmail.com wrote:
On Thu, Mar 31, 2011 at 1:00 PM, Scott Sinclair
scott.sinclair...@gmail.com wrote:
For what it's worth, the maintenance/1.6.x branch works for me on 64-bit
Ubuntu:
(numpy-1.6.x)scott@godzilla:~$ python -c import numpy; numpy.test()
You might want to run
python -c import numpy; numpy.test('full')
as the corresponding test is decorated as slow.
python -c import numpy; numpy.test('full')
Running unit tests for numpy
NumPy version 1.6.0b2.dev-a172fd6
NumPy is installed in
/home/scott/.virtualenvs/numpy-1.6.x/lib/python2.6/site-packages/numpy
Python version 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5]
nose version 1.0.0
snip...
Ran 3423 tests in 28.713s
OK (KNOWNFAIL=3, SKIP=4)
Cheers,
Scott
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] EuroPython 2011: call for paper is ending - Please spread the word
Hi all, I'm Francesco and I am writing on behalf of Python Italia APS, a no-profit association promoting EuroPython conference. (www.europython.eu) Europython End of Call for Presentations is April 6th. I'd like to ask to you to forward this mail to anyone that you feel may be interested. We're looking for proposals on every aspects of Python: programming from novice to advanced levels, applications and frameworks, or how you have been involved in introducing Python into your organisation. **First-time speakers are especially welcome**; EuroPython is a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! Presenting at EuroPython We will accept a broad range of presentations, from reports on academic and commercial projects to tutorials and case studies. As long as the presentation is interesting and potentially useful to the Python community, it will be considered for inclusion in the programme. Can you show the conference-goers something new and useful? Can you show attendees how to: use a module? Explore a Python language feature? Package an application? If so, consider submitting a talk. Talks and hands-on trainings There are two different kind of presentations that you can give as a speaker at EuroPython: * **Regular talk**. These are standard talk with slides, allocated in slots of 45, 60 or 90 minutes, depending on your preference and scheduling constraints. A QA session is held at the end of the talk. * **Hands-on training**. These are advanced training sessions for a smaller audience (10-20 people), to dive into the subject with all details. These sessions are 4-hours long, and audience will be strongly encouraged to bring a laptop to experiment. They should be prepared with less slides and more source code. If possible, trainers will also give a short teaser talk of 30 minutes the day before the training, to tease delegates into attending the training. In the talk submission form, we assume that you intend to give a regular talk on the subject, but you will be asked if you are available for also doing a hands-on training on the very same subject. Speakers that will give a hands-on training are rewarded with a **free entrance** to EuroPython to compensate for the longer preparation required, and might also be eligible for a speaking fee (which we cannot confirm at the moment). Topics and goals Specific topics for EuroPython presentations include, but are not limited to: - Core Python - Other implementations: Jython, IronPython, PyPy, and Stackless - Python libraries and extensions - Python 3.x migration - Databases - Documentation - GUI Programming - Game Programming - Network Programming - Open Source Python projects - Packaging Issues - Programming Tools - Project Best Practices - Embedding and Extending - Science and Math - Web-based Systems Presentation goals usually are some of the following: - Introduce audience to a new topic they are unaware of - Introduce audience to new developments on a well-known topic - Show audience real-world usage scenarios for a specific topic (case study) - Dig into advanced and relatively-unknown details on a topic - Compare different options in the market on a topic Community-based talk voting --- This year, for the first time in EuroPython history, the talk voting process is fully public. Every partecipant gains the right to vote for talks submitted during the Call For Papers, as soon as they commit to their presence at the conference by buying a ticket. See all the details in the talk voting[1] page. Contacts For any further question, feel free to contact the organizers at i...@pycon.it. Thank you! [1]: http://ep2011.europython.eu/talk-voting -- -PALLA ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt/savetxt tickets
On Thu, Mar 31, 2011 at 4:53 AM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
paul.anton.let...@gmail.com wrote:
On 26. mars 2011, at 21.44, Derek Homeier wrote:
Hi Paul,
having had a look at the other tickets you dug up,
My opinions are my own, and in detail, they are:
1752:
I attach a possible patch. FWIW, I agree with the request. The
patch is written to be compatible with the fix in ticket #1562, but
I did not test that yet.
Tested, see also my comments on Trac.
Great!
1731:
This seems like a rather trivial feature enhancement. I attach a
possible patch.
Agreed. Haven't tested it though.
Great!
1616:
The suggested patch seems reasonable to me, but I do not have a
full list of what objects loadtxt supports today as opposed to what
this patch will support.
Looks like you got this one. Just remember to make it compatible with
#1752. Should be easy.
1562:
I attach a possible patch. This could also be the default
behavior to my mind, since the function caller can simply call
numpy.squeeze if needed. Changing default behavior would probably
break old code, however.
See comments on Trac as well.
Your patch is better, but there is one thing I disagree with.
808 if X.ndim ndmin:
809 if ndmin == 1:
810 X.shape = (X.size, )
811 elif ndmin == 2:
812 X.shape = (X.size, 1)
The last line should be:
812 X.shape = (1, X.size)
If someone wants a 2D array out, they would most likely expect a one-row
file to come out as a one-row array, not the other way around. IMHO.
1458:
The fix suggested in the ticket seems reasonable, but I have
never used record arrays, so I am not sure of this.
There were some issues with Python3, and I also had some general
reservations
as noted on Trac - basically, it makes 'unpack' equivalent to
transposing for 2D-arrays,
but to splitting into fields for 1D-recarrays. My question was, what's
going to happen
when you get to 2D-recarrays? Currently this is not an issue since
loadtxt can only
read 2D regular or 1D structured arrays. But this might change if the
data block
functionality (see below) were to be implemented - data could then be
returned as
3D arrays or 2D structured arrays... Still, it would probably make
most sense (or at
least give the widest functionality) to have 'unpack=True' always
return a list or iterator
over columns.
OK, I don't know recarrays, as I said.
1445:
Adding this functionality could break old code, as some old
datafiles may have empty lines which are now simply ignored. I do
not think the feature is a good idea. It could rather be implemented
as a separate function.
1107:
I do not see the need for this enhancement. In my eyes, the
usecols kwarg does this and more. Perhaps I am misunderstanding
something here.
Agree about #1445, and the bit about 'usecols' - 'numcols' would just
provide a
shorter call to e.g. read the first 20 columns of a file (well, not
even that much
over 'usecols=range(20)'...), don't think that justifies an extra
argument.
But the 'datablocks' provides something new, that a number of people
seem
to miss from e.g. gnuplot (including me, actually ;-). And it would
also satisfy the
request from #1445 without breaking backwards compatibility.
I've been wondering if could instead specify the separator lines
through the
parameter, e.g. blocksep=['None', 'blank','invalid'], not sure if
that would make
it more useful...
What about writing a separate function, e.g. loadblocktxt, and have it
separate the chunks and call loadtxt for each chunk? Just a thought. Another
possibility would be to write a function that would let you load a set of
text files in a directory, and return a dict of datasets, one per file. One
could write a similar save-function, too. They would just need to call
loadtxt/savetxt on a per-file basis.
1071:
It is not clear to me whether loadtxt is supposed to support
missing values in the fashion indicated in the ticket.
In principle it should at least allow you to, by the use of converters
as described there.
The problem is, the default delimiter is described as 'any
whitespace', which in the
present implementation obviously includes any number of blanks or
tabs. These
are therefore treated differently from delimiters like ',' or ''. I'd
reckon there are
too many people actually relying on this behaviour to silently change it
(e.g. I know plenty of tables with columns separated by either one or
several
tabs depending on the length of the previous entry). But the tab is
apparently also
treated differently if explicitly specified with delimiter='\t' -
and in that case using
a converter à la {2: lambda s: float(s or 'Nan')} is working for
fields in the middle of
the line, but not at
Re: [Numpy-discussion] Old tickets
On Wed, Mar 30, 2011 at 11:52 PM, Derek Homeier
de...@astro.physik.uni-goettingen.de wrote:
On 30 Mar 2011, at 23:26, Benjamin Root wrote:
Ticket 301: 'Make power and divide return floats from int inputs (like
true_divide)'
http://projects.scipy.org/numpy/ticket/301
Invalid because the output dtype is the same as the input dtype unless
you override using the dtype argument:
np.power(3, 1, dtype=np.float128).dtype
dtype('float128')
Alternatively return a float and indicate in the docstring that the
output dtype can be changed.
FWIW,
Just thought I'd note (on a python 2.6 system):
import numpy as np
a = np.array([1, 2, 3, 4])
a.dtype
dtype('int32')
2 / a
array([2, 1, 0, 0])
from __future__ import division
2 / a
array([ 2. , 1. , 0.6667, 0.5 ])
So, numpy already does this when true division is imported (and
therefore consistent with whatever the python environment does), and
python currently also returns integers for exponentials when both
inputs are integers.
I'd agree, and in my view power(3, -1) is well defined as 1 / 3 -
also, in future (or Python3)
a/2
array([ 0.5, 1. , 1.5, 2. ])
a//2
array([0, 1, 1, 2], dtype=int32)
The ticket is about the functions np.divide and np.power, not / and
**. This currently does not work the same, unlike what's said above:
from __future__ import division
x = np.arange(4) + 1
2 / x
array([ 2., 1., 0.6667, 0.5 ])
np.divide(2, x)
array([2, 1, 0, 0])
The power and divide functions should not check the from __future__
import division, just return floats IMHO. This is what's expected by
most users, and much more useful than integer math.
so I think at least a**n should follow integer math rules; depends on
whether we want
np.power to behave differently from ** (if they are internally handled
separately at all)...
Not sure if I understand the overload suggestion in the ticket
I don't understand that either, it'll just lead to more confusion.
Cheers,
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt/savetxt tickets
On Wed, Mar 30, 2011 at 9:53 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
paul.anton.let...@gmail.com wrote:
On 26. mars 2011, at 21.44, Derek Homeier wrote:
Hi Paul,
having had a look at the other tickets you dug up,
[snip]
1071:
It is not clear to me whether loadtxt is supposed to support
missing values in the fashion indicated in the ticket.
In principle it should at least allow you to, by the use of converters
as described there.
The problem is, the default delimiter is described as 'any
whitespace', which in the
present implementation obviously includes any number of blanks or
tabs. These
are therefore treated differently from delimiters like ',' or ''. I'd
reckon there are
too many people actually relying on this behaviour to silently change it
(e.g. I know plenty of tables with columns separated by either one or
several
tabs depending on the length of the previous entry). But the tab is
apparently also
treated differently if explicitly specified with delimiter='\t' -
and in that case using
a converter à la {2: lambda s: float(s or 'Nan')} is working for
fields in the middle of
the line, but not at the end - clearly warrants improvement. I've
prepared a patch
working for Python3 as well.
Great!
This is an invalid ticket because the docstring clearly states that in
3 different, yet critical places, that missing values are not handled
here:
Each row in the text file must have the same number of values.
genfromtxt : Load data with missing values handled as specified.
This function aims to be a fast reader for simply formatted files. The
`genfromtxt` function provides more sophisticated handling of, e.g.,
lines with missing values.
Really I am trying to separate the usage of loadtxt and genfromtxt to
avoid unnecessary duplication and confusion. Part of this is
historical because loadtxt was added in 2007 and genfromtxt was added
in 2009. So really certain features of loadtxt have been 'kept' for
backwards compatibility purposes yet these features can be 'abused' to
handle missing data. But I really consider that any missing values
should cause loadtxt to fail.
The patch is incorrect because it should not include a space in the
split() as indicated in the comment by the original reporter. Of
course a corrected patch alone still is not sufficient to address the
problem without the user providing the correct converter. Also you
start to run into problems with multiple delimiters (such as one space
versus two spaces) so you start down the path to add all the features
that duplicate genfromtxt.
Bruce
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt/savetxt tickets
On Thu, Mar 31, 2011 at 5:03 PM, Bruce Southey bsout...@gmail.com wrote:
On Wed, Mar 30, 2011 at 9:53 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
paul.anton.let...@gmail.com wrote:
On 26. mars 2011, at 21.44, Derek Homeier wrote:
Hi Paul,
having had a look at the other tickets you dug up,
[snip]
1071:
It is not clear to me whether loadtxt is supposed to support
missing values in the fashion indicated in the ticket.
In principle it should at least allow you to, by the use of converters
as described there.
The problem is, the default delimiter is described as 'any
whitespace', which in the
present implementation obviously includes any number of blanks or
tabs. These
are therefore treated differently from delimiters like ',' or ''. I'd
reckon there are
too many people actually relying on this behaviour to silently change it
(e.g. I know plenty of tables with columns separated by either one or
several
tabs depending on the length of the previous entry). But the tab is
apparently also
treated differently if explicitly specified with delimiter='\t' -
and in that case using
a converter à la {2: lambda s: float(s or 'Nan')} is working for
fields in the middle of
the line, but not at the end - clearly warrants improvement. I've
prepared a patch
working for Python3 as well.
Great!
This is an invalid ticket because the docstring clearly states that in
3 different, yet critical places, that missing values are not handled
here:
Each row in the text file must have the same number of values.
genfromtxt : Load data with missing values handled as specified.
This function aims to be a fast reader for simply formatted files. The
`genfromtxt` function provides more sophisticated handling of, e.g.,
lines with missing values.
Really I am trying to separate the usage of loadtxt and genfromtxt to
avoid unnecessary duplication and confusion. Part of this is
historical because loadtxt was added in 2007 and genfromtxt was added
in 2009. So really certain features of loadtxt have been 'kept' for
backwards compatibility purposes yet these features can be 'abused' to
handle missing data. But I really consider that any missing values
should cause loadtxt to fail.
I agree with you Bruce, but it would be easier to discuss this on the
tickets instead of here. Could you add your comments there please?
Ralf
The patch is incorrect because it should not include a space in the
split() as indicated in the comment by the original reporter. Of
course a corrected patch alone still is not sufficient to address the
problem without the user providing the correct converter. Also you
start to run into problems with multiple delimiters (such as one space
versus two spaces) so you start down the path to add all the features
that duplicate genfromtxt.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt/savetxt tickets
On 03/31/2011 10:08 AM, Ralf Gommers wrote:
On Thu, Mar 31, 2011 at 5:03 PM, Bruce Southeybsout...@gmail.com wrote:
On Wed, Mar 30, 2011 at 9:53 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes
paul.anton.let...@gmail.com wrote:
On 26. mars 2011, at 21.44, Derek Homeier wrote:
Hi Paul,
having had a look at the other tickets you dug up,
[snip]
1071:
It is not clear to me whether loadtxt is supposed to support
missing values in the fashion indicated in the ticket.
In principle it should at least allow you to, by the use of converters
as described there.
The problem is, the default delimiter is described as 'any
whitespace', which in the
present implementation obviously includes any number of blanks or
tabs. These
are therefore treated differently from delimiters like ',' or ''. I'd
reckon there are
too many people actually relying on this behaviour to silently change it
(e.g. I know plenty of tables with columns separated by either one or
several
tabs depending on the length of the previous entry). But the tab is
apparently also
treated differently if explicitly specified with delimiter='\t' -
and in that case using
a converter à la {2: lambda s: float(s or 'Nan')} is working for
fields in the middle of
the line, but not at the end - clearly warrants improvement. I've
prepared a patch
working for Python3 as well.
Great!
This is an invalid ticket because the docstring clearly states that in
3 different, yet critical places, that missing values are not handled
here:
Each row in the text file must have the same number of values.
genfromtxt : Load data with missing values handled as specified.
This function aims to be a fast reader for simply formatted files. The
`genfromtxt` function provides more sophisticated handling of, e.g.,
lines with missing values.
Really I am trying to separate the usage of loadtxt and genfromtxt to
avoid unnecessary duplication and confusion. Part of this is
historical because loadtxt was added in 2007 and genfromtxt was added
in 2009. So really certain features of loadtxt have been 'kept' for
backwards compatibility purposes yet these features can be 'abused' to
handle missing data. But I really consider that any missing values
should cause loadtxt to fail.
I agree with you Bruce, but it would be easier to discuss this on the
tickets instead of here. Could you add your comments there please?
Ralf
'Easier' seems a contradiction when you have use captcha...
Sure I will add more comments there.
Bruce
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Old tickets
On Thu, Mar 31, 2011 at 10:00 AM, Ralf Gommers
ralf.gomm...@googlemail.comwrote:
On Wed, Mar 30, 2011 at 11:52 PM, Derek Homeier
de...@astro.physik.uni-goettingen.de wrote:
On 30 Mar 2011, at 23:26, Benjamin Root wrote:
Ticket 301: 'Make power and divide return floats from int inputs (like
true_divide)'
http://projects.scipy.org/numpy/ticket/301
Invalid because the output dtype is the same as the input dtype unless
you override using the dtype argument:
np.power(3, 1, dtype=np.float128).dtype
dtype('float128')
Alternatively return a float and indicate in the docstring that the
output dtype can be changed.
FWIW,
Just thought I'd note (on a python 2.6 system):
import numpy as np
a = np.array([1, 2, 3, 4])
a.dtype
dtype('int32')
2 / a
array([2, 1, 0, 0])
from __future__ import division
2 / a
array([ 2., 1., 0.6667, 0.5 ])
So, numpy already does this when true division is imported (and
therefore consistent with whatever the python environment does), and
python currently also returns integers for exponentials when both
inputs are integers.
I'd agree, and in my view power(3, -1) is well defined as 1 / 3 -
also, in future (or Python3)
a/2
array([ 0.5, 1. , 1.5, 2. ])
a//2
array([0, 1, 1, 2], dtype=int32)
The ticket is about the functions np.divide and np.power, not / and
**. This currently does not work the same, unlike what's said above:
from __future__ import division
x = np.arange(4) + 1
2 / x
array([ 2., 1., 0.6667, 0.5 ])
np.divide(2, x)
array([2, 1, 0, 0])
The power and divide functions should not check the from __future__
import division, just return floats IMHO. This is what's expected by
most users, and much more useful than integer math.
Hmm, I didn't notice that distinction. So long as there is a consistent
difference between the function and the operator, I am fine with that.
so I think at least a**n should follow integer math rules; depends on
whether we want
np.power to behave differently from ** (if they are internally handled
separately at all)...
Not sure if I understand the overload suggestion in the ticket
I don't understand that either, it'll just lead to more confusion.
Did a little further investigation of this (I hadn't thought of negative
powers), and now I am getting thoroughly confused...
import numpy as np
a = np.array([1, 2])
a.dtype
dtype('int32')
1**2
1
2**2
4
1**-2
1.0
2**-2
0.25
a**2
array([1, 4])
a**-2
array([1, 0])
np.power(a, 2)
array([1, 4])
np.power(a, -2)
array([1, 0])
So, in this case, the ** operator and np.power are identical, but diverges
from the behavior of python's operator.
FWIW,
Ben Root
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Old tickets
On Thu, Mar 31, 2011 at 9:00 AM, Ralf Gommers
ralf.gomm...@googlemail.comwrote:
On Wed, Mar 30, 2011 at 11:52 PM, Derek Homeier
de...@astro.physik.uni-goettingen.de wrote:
On 30 Mar 2011, at 23:26, Benjamin Root wrote:
Ticket 301: 'Make power and divide return floats from int inputs (like
true_divide)'
http://projects.scipy.org/numpy/ticket/301
Invalid because the output dtype is the same as the input dtype unless
you override using the dtype argument:
np.power(3, 1, dtype=np.float128).dtype
dtype('float128')
Alternatively return a float and indicate in the docstring that the
output dtype can be changed.
FWIW,
Just thought I'd note (on a python 2.6 system):
import numpy as np
a = np.array([1, 2, 3, 4])
a.dtype
dtype('int32')
2 / a
array([2, 1, 0, 0])
from __future__ import division
2 / a
array([ 2., 1., 0.6667, 0.5 ])
So, numpy already does this when true division is imported (and
therefore consistent with whatever the python environment does), and
python currently also returns integers for exponentials when both
inputs are integers.
I'd agree, and in my view power(3, -1) is well defined as 1 / 3 -
also, in future (or Python3)
a/2
array([ 0.5, 1. , 1.5, 2. ])
a//2
array([0, 1, 1, 2], dtype=int32)
The ticket is about the functions np.divide and np.power, not / and
**. This currently does not work the same, unlike what's said above:
from __future__ import division
x = np.arange(4) + 1
2 / x
array([ 2., 1., 0.6667, 0.5 ])
np.divide(2, x)
array([2, 1, 0, 0])
The power and divide functions should not check the from __future__
import division, just return floats IMHO. This is what's expected by
most users, and much more useful than integer math
There is true_divide for that.
In [4]: np.true_divide(2, x)
Out[4]: array([ 2., 1., 0.6667, 0.5 ])
true_divide is what is called by the '/' operator in Python3.
snip
Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Old tickets
On 3/31/11 8:41 AM, Benjamin Root wrote: The ticket is about the functions np.divide and np.power, not / and **. This currently does not work the same, unlike what's said above: Hmm, I didn't notice that distinction. So long as there is a consistent difference between the function and the operator, I am fine with that. whether we want np.power to behave differently from ** (if they are internally handled separately at all)... Please, no! the operator and functions should behave the same. I certainly always expect them to. For instance, I often write code with operators, then replace that with the functions so that i can add an out parameter for performance reasons -- I would be very surprised if the function were defined differently -- it would be a source of surprising bugs. So, in this case, the ** operator and np.power are identical, but diverges from the behavior of python's operator. I think consistency within numpy is more important that consistency with python -- I expect differences like this from python (long integers, different data types, etc) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1. It appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc (see below for examples). This seems wrong. Thoughts? /usr/lib64/python2.7/site-packages/doc/cython/MANIFEST /usr/lib64/python2.7/site-packages/doc/cython/Makefile /usr/lib64/python2.7/site-packages/doc/cython/README.txt /usr/lib64/python2.7/site-packages/doc/cython/c_numpy.pxd /usr/lib64/python2.7/site-packages/doc/cython/c_python.pxd /usr/lib64/python2.7/site-packages/doc/cython/numpyx.pyx /usr/lib64/python2.7/site-packages/doc/cython/run_test.py /usr/lib64/python2.7/site-packages/doc/cython/run_test.pyc /usr/lib64/python2.7/site-packages/doc/cython/run_test.pyo /usr/lib64/python2.7/site-packages/doc/cython/setup.py /usr/lib64/python2.7/site-packages/doc/cython/setup.pyc /usr/lib64/python2.7/site-packages/doc/cython/setup.pyo /usr/lib64/python2.7/site-packages/doc/pyrex/MANIFEST /usr/lib64/python2.7/site-packages/doc/pyrex/Makefile /usr/lib64/python2.7/site-packages/doc/pyrex/README.txt /usr/lib64/python2.7/site-packages/doc/pyrex/c_numpy.pxd /usr/lib64/python2.7/site-packages/doc/pyrex/c_python.pxd /usr/lib64/python2.7/site-packages/doc/pyrex/notes /usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.c /usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.pyx /usr/lib64/python2.7/site-packages/doc/pyrex/run_test.py /usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyc /usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyo /usr/lib64/python2.7/site-packages/doc/pyrex/setup.py /usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyc /usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyo /usr/lib64/python2.7/site-packages/doc/swig/Array.i /usr/lib64/python2.7/site-packages/doc/swig/Array1.cxx /usr/lib64/python2.7/site-packages/doc/swig/Array1.h -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA DivisionFAX: 303-415-9702 3380 Mitchell Lane or...@cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski or...@cora.nwra.com wrote:
I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1. It
appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc
(see below for examples). This seems wrong. Thoughts?
That is indeed wrong. It's caused by the following in setup.py:
# we want these files also in binaries/installed files, so it belongs here
# instead of in Manifest.in
config.add_data_files(('doc/cython/'),
('doc/pyrex/'),
('doc/swig/'))
Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
source tree?
Ralf
/usr/lib64/python2.7/site-packages/doc/cython/MANIFEST
/usr/lib64/python2.7/site-packages/doc/cython/Makefile
/usr/lib64/python2.7/site-packages/doc/cython/README.txt
/usr/lib64/python2.7/site-packages/doc/cython/c_numpy.pxd
/usr/lib64/python2.7/site-packages/doc/cython/c_python.pxd
/usr/lib64/python2.7/site-packages/doc/cython/numpyx.pyx
/usr/lib64/python2.7/site-packages/doc/cython/run_test.py
/usr/lib64/python2.7/site-packages/doc/cython/run_test.pyc
/usr/lib64/python2.7/site-packages/doc/cython/run_test.pyo
/usr/lib64/python2.7/site-packages/doc/cython/setup.py
/usr/lib64/python2.7/site-packages/doc/cython/setup.pyc
/usr/lib64/python2.7/site-packages/doc/cython/setup.pyo
/usr/lib64/python2.7/site-packages/doc/pyrex/MANIFEST
/usr/lib64/python2.7/site-packages/doc/pyrex/Makefile
/usr/lib64/python2.7/site-packages/doc/pyrex/README.txt
/usr/lib64/python2.7/site-packages/doc/pyrex/c_numpy.pxd
/usr/lib64/python2.7/site-packages/doc/pyrex/c_python.pxd
/usr/lib64/python2.7/site-packages/doc/pyrex/notes
/usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.c
/usr/lib64/python2.7/site-packages/doc/pyrex/numpyx.pyx
/usr/lib64/python2.7/site-packages/doc/pyrex/run_test.py
/usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyc
/usr/lib64/python2.7/site-packages/doc/pyrex/run_test.pyo
/usr/lib64/python2.7/site-packages/doc/pyrex/setup.py
/usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyc
/usr/lib64/python2.7/site-packages/doc/pyrex/setup.pyo
/usr/lib64/python2.7/site-packages/doc/swig/Array.i
/usr/lib64/python2.7/site-packages/doc/swig/Array1.cxx
/usr/lib64/python2.7/site-packages/doc/swig/Array1.h
--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA Division FAX: 303-415-9702
3380 Mitchell Lane or...@cora.nwra.com
Boulder, CO 80301 http://www.cora.nwra.com
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt/savetxt tickets
On 31 Mar 2011, at 17:03, Bruce Southey wrote:
This is an invalid ticket because the docstring clearly states that in
3 different, yet critical places, that missing values are not handled
here:
Each row in the text file must have the same number of values.
genfromtxt : Load data with missing values handled as specified.
This function aims to be a fast reader for simply formatted
files. The
`genfromtxt` function provides more sophisticated handling of,
e.g.,
lines with missing values.
Really I am trying to separate the usage of loadtxt and genfromtxt to
avoid unnecessary duplication and confusion. Part of this is
historical because loadtxt was added in 2007 and genfromtxt was added
in 2009. So really certain features of loadtxt have been 'kept' for
backwards compatibility purposes yet these features can be 'abused' to
handle missing data. But I really consider that any missing values
should cause loadtxt to fail.
OK, I was not aware of the design issues of loadtxt vs. genfromtxt -
you could probably say also for historical reasons since I have not
used genfromtxt much so far.
Anyway the docstring statement Converters can also be used to
provide a default value for missing data:
then appears quite misleading, or an invitation to abuse, if you will.
This should better be removed from the documentation then, or users
explicitly discouraged from using converters instead of genfromtxt
(I don't see how you could completely prevent using converters in
this way).
The patch is incorrect because it should not include a space in the
split() as indicated in the comment by the original reporter. Of
The split('\r\n') alone caused test_dtype_with_object(self) to fail,
probably
because it relies on stripping the blanks. But maybe the test is ill-
formed?
course a corrected patch alone still is not sufficient to address the
problem without the user providing the correct converter. Also you
start to run into problems with multiple delimiters (such as one space
versus two spaces) so you start down the path to add all the features
that duplicate genfromtxt.
Given that genfromtxt provides that functionality more conveniently,
I agree again users should be encouraged to use this instead of
converters.
But the actual tab-problem causes in fact an issue not related to
missing
values at all (well, depending on what you call a missing value).
I am describing an example on the ticket.
Cheers,
Derek
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?
On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail) wickedg...@gmail.comwrote: On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail) wickedg...@gmail.com wrote: Can anyone please give me some suggestions on how to go about writing a unit test for this? Or should I just submit a pull request? I've gotten a bit of positive feedback to adding the 'e' type to the struct module on the python-ideas list (per my understanding, not before python 3.3, but I don't think that should hinder adoption in other libraries), so I'd like to ask again about unit testing a change like this. Can anyone offer some advice for where to start? The tests for the buffer protocol are in numpy/core/tests/test_multiarray.py, starting at line 1847. It does some round-trip testing of all the types through the memoryview object. You'll also need to change the _dtype_from_pep3118 in numpy/core/_internal.py, called from the _descriptor_from_pep3118_format function, so that NumPy can accept the half type from a buffer as well. Also, what kind of timeframe / cutoff am I looking at to get this into 1.6.0 or 1.6.x? Since the changes to NumPy are pretty small, I think it should be ok to get into 1.6.0. How to handle it is up to Ralf, however, and it would probably be best to get it into beta 2 which I believe he would like to release over the weekend (please chime in to correct me). -Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] np.histogramdd of empty data
-- Forwarded message -- From: Ralf Gommers ralf.gomm...@googlemail.com Date: Thu, Mar 31, 2011 at 7:31 PM Subject: Re: [Numpy-discussion] np.histogramdd of empty data To: Nils Becker n.bec...@amolf.nl On Thu, Mar 31, 2011 at 12:33 PM, Nils Becker n.bec...@amolf.nl wrote: Hi Ralf, I cloned numpy/master and played around a little. when giving the bins explicitely, now histogram2d and histogramdd work as expected in all tests i tried. However, some of the cases with missing bin specification appear somewhat inconsistent. The first question is if creating arbitrary bins for empty data and empty bin specification is better than raising an Exception: Specifically: Bins of size 0 should give a meaningful error, I was just fixing that as part of #1788 in https://github.com/rgommers/numpy/tree/ticket-1788-histogramdd numpy.histogram2d([],[],bins=[0,0]) (array([ 0., 0.]), array([ 0.]), array([ 0.])) Now gives: ValueError: Element at index 0 in `bins` should be a positive integer. numpy.histogram([],bins=0) ValueError: zero-size array to minimum.reduce without identity Now gives: ValueError: `bins` should be a positive integer. so 1-d and 2-d behave not quite the same. also, these work (although with arbitrary bin edges): numpy.histogram2d([],[],bins=[1,1]) (array([ 0., 0.]), array([ 0., 1.]), array([ 0., 1.])) numpy.histogram2d([],[],bins=[0,1]) (array([ 0., 0.]), array([ 0.]), array([ 0., 1.])) while this raises an error: numpy.histogram([],bins=1) ValueError: zero-size array to minimum.reduce without identity Now gives: (array([0]), array([ 0., 1.])) another thing with non-empty data: numpy.histogram([1],bins=1) (array([1]), array([ 0.5, 1.5])) That is the correct answer. numpy.histogram([1],bins=0) (array([], dtype=int64), array([ 0.5])) Now gives: ValueError: `bins` should be a positive integer. while numpy.histogram2d([1],[1],bins=A) ValueError: zero-size array to minimum.reduce without identity (here A==[0,0] or A==[0,1] but not A==[1,1] which gives a result) Same sensible errors now, telling you bins elements shouldn't be 0. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?
On Thu, Mar 31, 2011 at 7:06 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail) wickedg...@gmail.com wrote: On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail) wickedg...@gmail.com wrote: Can anyone please give me some suggestions on how to go about writing a unit test for this? Or should I just submit a pull request? I've gotten a bit of positive feedback to adding the 'e' type to the struct module on the python-ideas list (per my understanding, not before python 3.3, but I don't think that should hinder adoption in other libraries), so I'd like to ask again about unit testing a change like this. Can anyone offer some advice for where to start? The tests for the buffer protocol are in numpy/core/tests/test_multiarray.py, starting at line 1847. It does some round-trip testing of all the types through the memoryview object. You'll also need to change the _dtype_from_pep3118 in numpy/core/_internal.py, called from the _descriptor_from_pep3118_format function, so that NumPy can accept the half type from a buffer as well. Also, what kind of timeframe / cutoff am I looking at to get this into 1.6.0 or 1.6.x? Since the changes to NumPy are pretty small, I think it should be ok to get into 1.6.0. How to handle it is up to Ralf, however, and it would probably be best to get it into beta 2 which I believe he would like to release over the weekend (please chime in to correct me). It would definitely need to get into beta 2, and even then I'm a little hesitant to push in such a change at the last moment. It would need some testing with other libraries that use the numpy buffer protocol. Also, reading back the thread, Pauli seemed to disagree with this. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] np.histogramdd of empty data
On Thu, Mar 31, 2011 at 12:32 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: -- Forwarded message -- From: Ralf Gommers ralf.gomm...@googlemail.com Date: Thu, Mar 31, 2011 at 7:31 PM Subject: Re: [Numpy-discussion] np.histogramdd of empty data To: Nils Becker n.bec...@amolf.nl On Thu, Mar 31, 2011 at 12:33 PM, Nils Becker n.bec...@amolf.nl wrote: Hi Ralf, I cloned numpy/master and played around a little. when giving the bins explicitely, now histogram2d and histogramdd work as expected in all tests i tried. However, some of the cases with missing bin specification appear somewhat inconsistent. The first question is if creating arbitrary bins for empty data and empty bin specification is better than raising an Exception: Specifically: Bins of size 0 should give a meaningful error, I was just fixing that as part of #1788 in https://github.com/rgommers/numpy/tree/ticket-1788-histogramdd numpy.histogram2d([],[],bins=[0,0]) (array([ 0., 0.]), array([ 0.]), array([ 0.])) Now gives: ValueError: Element at index 0 in `bins` should be a positive integer. numpy.histogram([],bins=0) ValueError: zero-size array to minimum.reduce without identity Now gives: ValueError: `bins` should be a positive integer. so 1-d and 2-d behave not quite the same. also, these work (although with arbitrary bin edges): numpy.histogram2d([],[],bins=[1,1]) (array([ 0., 0.]), array([ 0., 1.]), array([ 0., 1.])) numpy.histogram2d([],[],bins=[0,1]) (array([ 0., 0.]), array([ 0.]), array([ 0., 1.])) while this raises an error: numpy.histogram([],bins=1) ValueError: zero-size array to minimum.reduce without identity Now gives: (array([0]), array([ 0., 1.])) Just for consistency's sake, maybe the same should be done for np.bincount() and np.digitize() for empty data (but known bins)? I don't know if your fix to histogram does this or not, but the latest pull from numpy master doesn't do this. Thanks, Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?
On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Thu, Mar 31, 2011 at 7:06 PM, Mark Wiebe mwwi...@gmail.com wrote: On Wed, Mar 30, 2011 at 4:42 PM, Eli Stevens (Gmail) wickedg...@gmail.com wrote: On Fri, Mar 25, 2011 at 10:00 AM, Eli Stevens (Gmail) wickedg...@gmail.com wrote: Can anyone please give me some suggestions on how to go about writing a unit test for this? Or should I just submit a pull request? I've gotten a bit of positive feedback to adding the 'e' type to the struct module on the python-ideas list (per my understanding, not before python 3.3, but I don't think that should hinder adoption in other libraries), so I'd like to ask again about unit testing a change like this. Can anyone offer some advice for where to start? The tests for the buffer protocol are in numpy/core/tests/test_multiarray.py, starting at line 1847. It does some round-trip testing of all the types through the memoryview object. You'll also need to change the _dtype_from_pep3118 in numpy/core/_internal.py, called from the _descriptor_from_pep3118_format function, so that NumPy can accept the half type from a buffer as well. Also, what kind of timeframe / cutoff am I looking at to get this into 1.6.0 or 1.6.x? Since the changes to NumPy are pretty small, I think it should be ok to get into 1.6.0. How to handle it is up to Ralf, however, and it would probably be best to get it into beta 2 which I believe he would like to release over the weekend (please chime in to correct me). It would definitely need to get into beta 2, and even then I'm a little hesitant to push in such a change at the last moment. It would need some testing with other libraries that use the numpy buffer protocol. Also, reading back the thread, Pauli seemed to disagree with this. My reading of Pauli's thoughts was that putting it in unilaterally is undesirable, something I definitely agree with. I think with Eli doing the legwork of getting input and acceptance from the relevant parties, we should help him out as much as possible. Not getting this change into 1.6 makes the Cython support much more difficult because of their design based around the buffer protocol. Looking at the thread on python-users, I see mostly concern that the type be standard or have precedent elsewhere, which as an IEEE standard it definitely satisfies. The other question is on the chosen letter - we picked 'e' as 'h' was taken and it is close to 'f' and 'd', the other float types. If a letter change is deemed necessary, it would be good to get that into 1.6 as well, since this kind of change is easy now, but much more difficult later. For testing, Eli implementing the Cython support seems like a good start to me. Another nice test would be to get PIL's OpenEXR support to export a half buffer to NumPy. -Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On Thu, Mar 31, 2011 at 12:00, Ralf Gommers ralf.gomm...@googlemail.com wrote:
On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski or...@cora.nwra.com wrote:
I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1. It
appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc
(see below for examples). This seems wrong. Thoughts?
That is indeed wrong. It's caused by the following in setup.py:
# we want these files also in binaries/installed files, so it belongs here
# instead of in Manifest.in
config.add_data_files(('doc/cython/'),
('doc/pyrex/'),
('doc/swig/'))
Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
source tree?
It's not clear to me how they were included in the Windows .exe
installer originally (the original ticket that led to this
change[1088]). I don't think it was through the generic setup.py. The
proper fix would probably be something specific to each binary
installer.
[1088] http://projects.scipy.org/numpy/ticket/1088
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?
On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: It would definitely need to get into beta 2, and even then I'm a little hesitant to push in such a change at the last moment. If I miss the 1.6.0b2 cutoff, would a change like this be appropriate for 1.6.1, or will it need to wait until 1.7.0? I wish I'd known that half floats were going to be in 1.6.0 (the bug tracker says 2.0); I'd have started working on this a few weeks earlier. :) It would need some testing with other libraries that use the numpy buffer protocol. I am currently working on a patch for cython to accept half-floats. ATM, it is able to accept the data from numpy (though it does not yet handle the bit-fiddling to expand the values into a float32 on read, nor collapse it from a float32 back into a float16 on write). The code change to numpy was literally a one-line change to add NPY_HALF to a switch statement (though I haven't tried to import half floats yet). Is one other library enough, or will I need to find another library to patch as well? I think that OpenCL would probably be my next choice, since I think that we'll eventually want float16 support there too. Also, reading back the thread, Pauli seemed to disagree with this. On Fri, Mar 25, 2011 at 10:21 AM, Pauli Virtanen p...@iki.fi wrote: The buffer interface cannot be used to export the half-float types, since the type is not specified in PEP 3118. Numpy cannot unilaterally add nonstandard format codes to the spec. I'm currently in the process of submitting a patch to CPython to update the struct module to handle half-floats. So far, the response on python-ideas has been generally positive (a couple +1s from people with commit bits, GVR has chimed in on the thread with an aside (but not an objection), etc.). Unless I'm misunderstanding the objection, making half-floats an official part of the spec (the discussion on python-ideas explicitly notes that numpy driving force behind the change) would satisfy Pauli's concern. Next up for me is to get a patch onto the CPython issue tracker, but as soon as I have that done, I'll start working on adding unit tests to my numpy fork. I anticipate being able to get that done today. Thanks! Eli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On Thu, Mar 31, 2011 at 8:32 PM, Robert Kern robert.k...@gmail.com wrote:
On Thu, Mar 31, 2011 at 12:00, Ralf Gommers ralf.gomm...@googlemail.com
wrote:
On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski or...@cora.nwra.com wrote:
I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1. It
appears to install a bunch of suff in /usr/lib64/python2.7/site-packages/doc
(see below for examples). This seems wrong. Thoughts?
That is indeed wrong. It's caused by the following in setup.py:
# we want these files also in binaries/installed files, so it belongs here
# instead of in Manifest.in
config.add_data_files(('doc/cython/'),
('doc/pyrex/'),
('doc/swig/'))
Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
source tree?
It's not clear to me how they were included in the Windows .exe
installer originally (the original ticket that led to this
change[1088]). I don't think it was through the generic setup.py. The
proper fix would probably be something specific to each binary
installer.
[1088] http://projects.scipy.org/numpy/ticket/1088
I'm not sure how this was done at first (maybe someone remembers?), it
kind of hard to search for that in the history. But something specific
for each way of installing sounds fragile. The first question to be
answered is where these files should be installed to. The current
location is obviously wrong.
If it is under numpy/doc/swig/ etc. then moving them there in the
source tree itself seems logical. Those dirs are not used to build the
user and reference guides, so there's no need for them to stay where
they are.
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt/savetxt tickets
On 03/31/2011 12:02 PM, Derek Homeier wrote:
On 31 Mar 2011, at 17:03, Bruce Southey wrote:
This is an invalid ticket because the docstring clearly states that in
3 different, yet critical places, that missing values are not handled
here:
Each row in the text file must have the same number of values.
genfromtxt : Load data with missing values handled as specified.
This function aims to be a fast reader for simply formatted
files. The
`genfromtxt` function provides more sophisticated handling of,
e.g.,
lines with missing values.
Really I am trying to separate the usage of loadtxt and genfromtxt to
avoid unnecessary duplication and confusion. Part of this is
historical because loadtxt was added in 2007 and genfromtxt was added
in 2009. So really certain features of loadtxt have been 'kept' for
backwards compatibility purposes yet these features can be 'abused' to
handle missing data. But I really consider that any missing values
should cause loadtxt to fail.
OK, I was not aware of the design issues of loadtxt vs. genfromtxt -
you could probably say also for historical reasons since I have not
used genfromtxt much so far.
Anyway the docstring statement Converters can also be used to
provide a default value for missing data:
then appears quite misleading, or an invitation to abuse, if you will.
This should better be removed from the documentation then, or users
explicitly discouraged from using converters instead of genfromtxt
(I don't see how you could completely prevent using converters in
this way).
The patch is incorrect because it should not include a space in the
split() as indicated in the comment by the original reporter. Of
The split('\r\n') alone caused test_dtype_with_object(self) to fail,
probably
because it relies on stripping the blanks. But maybe the test is ill-
formed?
course a corrected patch alone still is not sufficient to address the
problem without the user providing the correct converter. Also you
start to run into problems with multiple delimiters (such as one space
versus two spaces) so you start down the path to add all the features
that duplicate genfromtxt.
Given that genfromtxt provides that functionality more conveniently,
I agree again users should be encouraged to use this instead of
converters.
But the actual tab-problem causes in fact an issue not related to
missing
values at all (well, depending on what you call a missing value).
I am describing an example on the ticket.
Cheers,
Derek
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
I am really not disagreeing that much with you. Rather that, as you have
shown, it is very easy to increase the complexity of examples that
loadtxt does not handle. By missing value I mean when no data value is
stored for the variable in the current observation (via Wikipedia)
since encoded missing values (such as '.', 'NA' and 'NaN') can be
recovered.
Bruce
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?
On Thu, Mar 31, 2011 at 8:37 PM, Eli Stevens (Gmail) wickedg...@gmail.com wrote: On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: It would definitely need to get into beta 2, and even then I'm a little hesitant to push in such a change at the last moment. If I miss the 1.6.0b2 cutoff, would a change like this be appropriate for 1.6.1, or will it need to wait until 1.7.0? I wish I'd known that half floats were going to be in 1.6.0 (the bug tracker says 2.0); I'd have started working on this a few weeks earlier. :) There likely will not be a 1.7.0, if things with 2.0 go according to plan. It probably wouldn't be a very appropriate change for 1.6.1. It would need some testing with other libraries that use the numpy buffer protocol. I am currently working on a patch for cython to accept half-floats. ATM, it is able to accept the data from numpy (though it does not yet handle the bit-fiddling to expand the values into a float32 on read, nor collapse it from a float32 back into a float16 on write). The code change to numpy was literally a one-line change to add NPY_HALF to a switch statement (though I haven't tried to import half floats yet). Is one other library enough, or will I need to find another library to patch as well? I think that OpenCL would probably be my next choice, since I think that we'll eventually want float16 support there too. If it works with Cython I think that's enough. With other libraries I did not mean that you should write patches for those, just some testing to make sure the change is 100% backwards compatible. Also, reading back the thread, Pauli seemed to disagree with this. On Fri, Mar 25, 2011 at 10:21 AM, Pauli Virtanen p...@iki.fi wrote: The buffer interface cannot be used to export the half-float types, since the type is not specified in PEP 3118. Numpy cannot unilaterally add nonstandard format codes to the spec. I'm currently in the process of submitting a patch to CPython to update the struct module to handle half-floats. So far, the response on python-ideas has been generally positive (a couple +1s from people with commit bits, GVR has chimed in on the thread with an aside (but not an objection), etc.). Unless I'm misunderstanding the objection, making half-floats an official part of the spec (the discussion on python-ideas explicitly notes that numpy driving force behind the change) would satisfy Pauli's concern. Next up for me is to get a patch onto the CPython issue tracker, but as soon as I have that done, I'll start working on adding unit tests to my numpy fork. I anticipate being able to get that done today. Sounds good. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On Thu, Mar 31, 2011 at 13:39, Ralf Gommers ralf.gomm...@googlemail.com wrote:
On Thu, Mar 31, 2011 at 8:32 PM, Robert Kern robert.k...@gmail.com wrote:
On Thu, Mar 31, 2011 at 12:00, Ralf Gommers ralf.gomm...@googlemail.com
wrote:
On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski or...@cora.nwra.com
wrote:
I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1. It
appears to install a bunch of suff in
/usr/lib64/python2.7/site-packages/doc
(see below for examples). This seems wrong. Thoughts?
That is indeed wrong. It's caused by the following in setup.py:
# we want these files also in binaries/installed files, so it belongs
here
# instead of in Manifest.in
config.add_data_files(('doc/cython/'),
('doc/pyrex/'),
('doc/swig/'))
Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
source tree?
It's not clear to me how they were included in the Windows .exe
installer originally (the original ticket that led to this
change[1088]). I don't think it was through the generic setup.py. The
proper fix would probably be something specific to each binary
installer.
[1088] http://projects.scipy.org/numpy/ticket/1088
I'm not sure how this was done at first (maybe someone remembers?), it
kind of hard to search for that in the history. But something specific
for each way of installing sounds fragile. The first question to be
answered is where these files should be installed to. The current
location is obviously wrong.
Well, the answer is different for each platform. That's why it ought
to be up to each binary installer.
If it is under numpy/doc/swig/ etc. then moving them there in the
source tree itself seems logical. Those dirs are not used to build the
user and reference guides, so there's no need for them to stay where
they are.
Linux distributions start to complain when data files (especially
documentation data files not used at runtime) are placed into the
Python packages. I would argue that it's the wrong place for them on
*any* platform.
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
-- Umberto Eco
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On Thu, Mar 31, 2011 at 8:52 PM, Robert Kern robert.k...@gmail.com wrote:
On Thu, Mar 31, 2011 at 13:39, Ralf Gommers ralf.gomm...@googlemail.com
wrote:
On Thu, Mar 31, 2011 at 8:32 PM, Robert Kern robert.k...@gmail.com wrote:
On Thu, Mar 31, 2011 at 12:00, Ralf Gommers ralf.gomm...@googlemail.com
wrote:
On Thu, Mar 31, 2011 at 6:33 PM, Orion Poplawski or...@cora.nwra.com
wrote:
I'm looking at updating the numpy package in Fedora rawhide to 1.6.0b1.
It
appears to install a bunch of suff in
/usr/lib64/python2.7/site-packages/doc
(see below for examples). This seems wrong. Thoughts?
That is indeed wrong. It's caused by the following in setup.py:
# we want these files also in binaries/installed files, so it belongs
here
# instead of in Manifest.in
config.add_data_files(('doc/cython/'),
('doc/pyrex/'),
('doc/swig/'))
Maybe those three dirs should be moved from doc/ to numpy/doc/ in the
source tree?
It's not clear to me how they were included in the Windows .exe
installer originally (the original ticket that led to this
change[1088]). I don't think it was through the generic setup.py. The
proper fix would probably be something specific to each binary
installer.
[1088] http://projects.scipy.org/numpy/ticket/1088
I'm not sure how this was done at first (maybe someone remembers?), it
kind of hard to search for that in the history. But something specific
for each way of installing sounds fragile. The first question to be
answered is where these files should be installed to. The current
location is obviously wrong.
Well, the answer is different for each platform. That's why it ought
to be up to each binary installer.
If it is under numpy/doc/swig/ etc. then moving them there in the
source tree itself seems logical. Those dirs are not used to build the
user and reference guides, so there's no need for them to stay where
they are.
Linux distributions start to complain when data files (especially
documentation data files not used at runtime) are placed into the
Python packages. I would argue that it's the wrong place for them on
*any* platform.
But they're not documentation files. The line is a bit blurry, but
something like the numpy.i SWIG interface face is meant for using
numpy, not documenting usage of SWIG and numpy (maybe most similar to
what's in core/include/ ?). The actual documentation of how to use
numpy.i is included in the reference guide, and is not present in that
dir.
Perhaps to make clear this is not documentation, put it in something
like numpy/interfacing/swig/, etc.?
Ralf
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On Thu, Mar 31, 2011 at 14:08, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Mar 31, 2011 at 8:52 PM, Robert Kern robert.k...@gmail.com wrote: Linux distributions start to complain when data files (especially documentation data files not used at runtime) are placed into the Python packages. I would argue that it's the wrong place for them on *any* platform. But they're not documentation files. The line is a bit blurry, but something like the numpy.i SWIG interface face is meant for using numpy, not documenting usage of SWIG and numpy (maybe most similar to what's in core/include/ ?). Yeah, and they hate that, too. As the originator of that idea, I'm loathe to push it any further. :-) The actual documentation of how to use numpy.i is included in the reference guide, and is not present in that dir. Perhaps to make clear this is not documentation, put it in something like numpy/interfacing/swig/, etc.? Well, they're meant to be copied into your own code, which is why they end up under a doc/ directory. Lots of things like this tend to end up in doc/ directories. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On 03/31/2011 01:08 PM, Ralf Gommers wrote: But they're not documentation files. The line is a bit blurry, but something like the numpy.i SWIG interface face is meant for using numpy, not documenting usage of SWIG and numpy (maybe most similar to what's in core/include/ ?). The actual documentation of how to use numpy.i is included in the reference guide, and is not present in that dir. Perhaps to make clear this is not documentation, put it in something like numpy/interfacing/swig/, etc.? How are these files used? Are they used by the python code directly? Just used by people wanting to use swig/cython/pyrex to interface with numpy? If the latter, they don't belong in python/site-packages but something lib /usr/share/numpy. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA DivisionFAX: 303-415-9702 3380 Mitchell Lane or...@cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On 03/31/2011 01:12 PM, Robert Kern wrote: Well, they're meant to be copied into your own code, which is why they end up under a doc/ directory. Lots of things like this tend to end up in doc/ directories. perhaps /usr/share/doc/numpy then. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA/CoRA DivisionFAX: 303-415-9702 3380 Mitchell Lane or...@cora.nwra.com Boulder, CO 80301 http://www.cora.nwra.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On Thu, Mar 31, 2011 at 14:16, Orion Poplawski or...@cora.nwra.com wrote: On 03/31/2011 01:12 PM, Robert Kern wrote: Well, they're meant to be copied into your own code, which is why they end up under a doc/ directory. Lots of things like this tend to end up in doc/ directories. perhaps /usr/share/doc/numpy then. This is up to the various Linux packagers of numpy. They have their own tools to do this according to their own policies. Doing *anything* in our setup.py to install these files will just make their jobs harder. That's why I think that this needs to be handled in the binary installers, not the setup.py. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 installs stuff in /usr/lib/python2.7/site-packages/doc
On 03/31/2011 01:21 PM, Robert Kern wrote:
On Thu, Mar 31, 2011 at 14:16, Orion Poplawskior...@cora.nwra.com wrote:
On 03/31/2011 01:12 PM, Robert Kern wrote:
Well, they're meant to be copied into your own code, which is why they
end up under a doc/ directory. Lots of things like this tend to end up
in doc/ directories.
perhaps /usr/share/doc/numpy then.
This is up to the various Linux packagers of numpy. They have their
own tools to do this according to their own policies. Doing *anything*
in our setup.py to install these files will just make their jobs
harder. That's why I think that this needs to be handled in the binary
installers, not the setup.py.
Fedora already copies everything in doc/* to /usr/share/doc/numpy-%{version},
so yeah, no need to put it anywhere.
--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA/CoRA DivisionFAX: 303-415-9702
3380 Mitchell Lane or...@cora.nwra.com
Boulder, CO 80301 http://www.cora.nwra.com
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Willing to contribute to SciPy NumPy ...
On 3/31/11 12:04 PM, Anthony Scopatz wrote: A good place to get started is helping out with documentation (see http://docs.scipy.org/numpy/Front%20Page/). Actually, according to the Jacob Kaplan-Moss's talk at PyCon this year (http://pycon.blip.tv/file/4881071/) Nice talk -- thanks for the link. (with which I am inclined to agree), documentation is probably not the best way to get started. I agree as well. However, I think newbies can be a great help in identifying holes in the documentation -- so when documentation is missing or unclear, ask here, and then you'll have something to contribute. -Chris Do check out the developer zone, maybe search the the open tickets and see if there are any bugs you want to tackle. Maybe write some tests for untested code you find. Good ideas as well, of course. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Willing to contribute to SciPy NumPy ...
On Thu, Mar 31, 2011 at 3:29 PM, Christopher Barker chris.bar...@noaa.govwrote: On 3/31/11 12:04 PM, Anthony Scopatz wrote: (with which I am inclined to agree), documentation is probably not the best way to get started. I agree as well. However, I think newbies can be a great help in identifying holes in the documentation -- so when documentation is missing or unclear, ask here, and then you'll have something to contribute. +1 on this. The biggest problem with developers writing documentation for tools like NumPy is that we have already become accustom to the quirks and assumptions of the tool. Fresh minds are perfect for identifying problem areas in the docs. The problem I have seen, though, is that it becomes so easy to just answer the user's question and then just move on. We really need to better remind ourselves that each time someone has to ask an easy question, it is a failing with our documentation. What if it was already in the docs, you ask? Then the question is, why couldn't they find it (or recognize it)? What if they didn't read the documentation, you say? Then the question becomes how did our documentation fail to be read? Just food for thought... Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-Dev] ANN: Numpy 1.6.0 beta 1
On 03/31/2011 06:37 PM, Pearu Peterson wrote: On Thu, Mar 31, 2011 at 12:19 PM, David Cournapeau courn...@gmail.com mailto:courn...@gmail.com wrote: On Wed, Mar 30, 2011 at 7:22 AM, Russell E. Owen ro...@uw.edu mailto:ro...@uw.edu wrote: In article AANLkTi=eeg8kl7639imrtl-ihg1ncqyolddsid5tf...@mail.gmail.com mailto:eeg8kl7639imrtl-ihg1ncqyolddsid5tf...@mail.gmail.com, Ralf Gommers ralf.gomm...@googlemail.com mailto:ralf.gomm...@googlemail.com wrote: Hi, I am pleased to announce the availability of the first beta of NumPy 1.6.0. Due to the extensive changes in the Numpy core for this release, the beta testing phase will last at least one month. Please test this beta and report any problems on the Numpy mailing list. Sources and binaries can be found at: http://sourceforge.net/projects/numpy/files/NumPy/1.6.0b1/ For (preliminary) release notes see below. I see a segfault on Ubuntu 64 bits for the test TestAssumedShapeSumExample in numpy/f2py/tests/test_assumed_shape.py. Am I the only one seeing it ? The test work here ok on Ubuntu 64 with numpy master. Could you try the maintenance/1.6.x branch where the related bugs are fixed. I did test that as well, and got the same issue, but could not reproduce it on another machine. I do get the error every time on my main work machine, though. I will look more into it, but it is most likely something due to my machine, cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: Numpy 1.6.0 beta 1
Ralf Gommers, on 2011-03-23 17:07, wrote: Please test this beta and report any problems on the Numpy mailing list. Hi, I reopened http://projects.scipy.org/numpy/ticket/1768, as 'test_chisquare' still fails for me on a 32 bit machine. Lowering the precision from 14 to 13 decimals makes the test pass, however. Other than that: NumPy version 1.6.0b1 NumPy is installed in /home/pi/.local/lib/python2.6/site-packages/numpy Python version 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] nose version 0.11.1 --snip-- Ran 3399 tests in 23.052s FAILED (KNOWNFAIL=3, SKIP=4, failures=1) best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 signature.asc Description: Digital signature ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] loadtxt/savetxt tickets
On Thu, Mar 31, 2011 at 8:42 AM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Thu, Mar 31, 2011 at 4:53 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Mar 27, 2011 at 4:09 AM, Paul Anton Letnes paul.anton.let...@gmail.com wrote: snip If you look in Trac under All Tickets by Milestone you'll find all nine tickets together under 1.6.0. Five are bug fixes, four are enhancements. There are some missing tests, but all tickets have proposed patches. OK. I changed 1562 to enhancement because it adds a keyword. With that change the current status looks like this. Bug Fixes: 1163 -- convert int64 correctly 1458 -- make loadtxt unpack structured arrays 1071 -- loadtxt fails if the last column contains empty value, under discussion 1565 -- duplicate of 1163 Enhancements: 1107 -- support for blocks of data, adds two keywords. 1562 -- add ndmin keyword to aid in getting correct dimensions, doesn't apply on top of previous. 1616 -- remove use of readline so input isn't restricted to files. 1731 -- allow loadtxt to read given number of rows, adds keyword. 1752 -- return empty array when empty file encountered, conflicts with 1616. Some of this might should go into genfromtxt. None of the patches have tests. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.6.0b1 half float buffer bug?
On Thu, Mar 31, 2011 at 11:45 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: On Thu, Mar 31, 2011 at 8:37 PM, Eli Stevens (Gmail) wickedg...@gmail.com wrote: Next up for me is to get a patch onto the CPython issue tracker, but as soon as I have that done, I'll start working on adding unit tests to my numpy fork. I anticipate being able to get that done today. Sounds good. CPython issue here: http://bugs.python.org/issue11734 My numpy changes so far: https://github.com/wickedgrey/numpy/compare/23965b6b665985390d7a...maintenance%2F1.6.x From my last commit message: Note that my changes to _internal were mostly of the make it work variety; I'm not really understanding the wider scope. Very much needs review by wiser eyes. I'd also appreciate some feedback on the unit test coverage; the test_roundtrip_half function uncovered a problem that the other two didn't (which surprised me), and I'd like to make sure that there aren't more gaps in the tests that I've written. Thanks for all the help! Eli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
