Re: [Numpy-discussion] More loadtxt() changes
Pierre GM wrote:
On Nov 26, 2008, at 5:55 PM, Ryan May wrote:
Manuel Metz wrote:
Ryan May wrote:
3) Better support for missing values. The docstring mentions a
way of
handling missing values by passing in a converter. The problem
with this is
that you have to pass in a converter for *every column* that will
contain
missing values. If you have a text file with 50 columns, writing
this
dictionary of converters seems like ugly and needless
boilerplate. I'm
unsure of how best to pass in both what values indicate missing
values and
what values to fill in their place. I'd love suggestions
Hi Ryan,
this would be a great feature to have !!!
About missing values:
* I don't think missing values should be supported in np.loadtxt. That
should go into a specific np.ma.io.loadtxt function, a preview of
which I posted earlier. I'll modify it taking Ryan's new function into
account, and Chrisopher's suggestion (defining a dictionary {column
name : missing values}.
* StringConverter already defines some default filling values for each
dtype. In np.ma.io.loadtxt, these values can be overwritten. Note
that you should also be able to define a filling value by specifying a
converter (think float(x or 0) for example)
* Missing values on space-separated fields are very tricky to handle:
take a line like a,,,d. With a comma as separator, it's clear that
the 2nd and 3rd fields are missing.
Now, imagine that commas are actually spaces ( a d): 'd' is now
seen as the 2nd field of a 2-field record, not as the 4th field of a 4-
field record with 2 missing values. I thought about it, and kicked in
touch
* That said, there should be a way to deal with fixed-length fields,
probably by taking consecutive slices of the initial string. That way,
we should be able to keep track of missing data...
Certainly, yes! Dealing with fixed-length fields would be necessary. The
case I had in mind had both -- a separator (|) __and__ fixed-length
fields -- and is probably very special in that sense. But such
data-files exists out there...
mm
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] More loadtxt() changes
On Nov 27, 2008, at 3:08 AM, Manuel Metz wrote: Certainly, yes! Dealing with fixed-length fields would be necessary. The case I had in mind had both -- a separator (|) __and__ fixed-length fields -- and is probably very special in that sense. But such data-files exists out there... Well, if you have a non-space delimiter, it doesn't matter if the fields have a fixed length or not, does it? Each field is stripped anyway. The real issue is when the delimiter is ' '... I should be able to take care of that over the week-end (which started earlier today over here :) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] More loadtxt() changes
On Thu, 27 Nov 2008 09:08:41 +0100
Manuel Metz [EMAIL PROTECTED] wrote:
Pierre GM wrote:
On Nov 26, 2008, at 5:55 PM, Ryan May wrote:
Manuel Metz wrote:
Ryan May wrote:
3) Better support for missing values. The docstring
mentions a
way of
handling missing values by passing in a converter. The
problem
with this is
that you have to pass in a converter for *every column*
that will
contain
missing values. If you have a text file with 50
columns, writing
this
dictionary of converters seems like ugly and needless
boilerplate. I'm
unsure of how best to pass in both what values indicate
missing
values and
what values to fill in their place. I'd love
suggestions
Hi Ryan,
this would be a great feature to have !!!
About missing values:
* I don't think missing values should be supported in
np.loadtxt. That
should go into a specific np.ma.io.loadtxt function, a
preview of
which I posted earlier. I'll modify it taking Ryan's new
function into
account, and Chrisopher's suggestion (defining a
dictionary {column
name : missing values}.
* StringConverter already defines some default filling
values for each
dtype. In np.ma.io.loadtxt, these values can be
overwritten. Note
that you should also be able to define a filling value
by specifying a
converter (think float(x or 0) for example)
* Missing values on space-separated fields are very
tricky to handle:
take a line like a,,,d. With a comma as separator,
it's clear that
the 2nd and 3rd fields are missing.
Now, imagine that commas are actually spaces ( a
d): 'd' is now
seen as the 2nd field of a 2-field record, not as the
4th field of a 4-
field record with 2 missing values. I thought about it,
and kicked in
touch
* That said, there should be a way to deal with
fixed-length fields,
probably by taking consecutive slices of the initial
string. That way,
we should be able to keep track of missing data...
Certainly, yes! Dealing with fixed-length fields would
be necessary. The
case I had in mind had both -- a separator (|) __and__
fixed-length
fields -- and is probably very special in that sense.
But such
data-files exists out there...
See page 9, 10 (Bulk data input deck)
http://www.zonatech.com/Documentation/zndalusersmanual2.0.pdf
Nils
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Masked array usage
I have a question about assigning to masked arrays. a is a len ==3 masked array, with 2 unmasked elements. b is a len == 2 array. I want to put the elements of b into the unmasked elements of a. How do I do that? In [598]: a Out[598]: masked_array(data = [1 -- 3], mask = [False True False], fill_value=99) In [599]: b Out[599]: array([7, 8]) I'd like an operation that gives me: masked_array(data = [7 -- 8], mask = [False True False], fill_value=99) Seems like it shouldn't be that hard, but I can't figure it out. Any suggestions? thanks, -robert ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Masked array usage
2008/11/27 Robert Ferrell [EMAIL PROTECTED]: I have a question about assigning to masked arrays. a is a len ==3 masked array, with 2 unmasked elements. b is a len == 2 array. I want to put the elements of b into the unmasked elements of a. How do I do that? In [598]: a Out[598]: masked_array(data = [1 -- 3], mask = [False True False], fill_value=99) In [599]: b Out[599]: array([7, 8]) I'd like an operation that gives me: masked_array(data = [7 -- 8], mask = [False True False], fill_value=99) Seems like it shouldn't be that hard, but I can't figure it out. Any suggestions? How about: c = a.copy() c[~a.mask] = b Angus. -- AJC McMorland Post-doctoral research fellow Neurobiology, University of Pittsburgh ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Masked array usage
Sweet. So simple. That works great. thanks, -robert On Nov 27, 2008, at 8:41 AM, Angus McMorland wrote: 2008/11/27 Robert Ferrell [EMAIL PROTECTED]: I have a question about assigning to masked arrays. a is a len ==3 masked array, with 2 unmasked elements. b is a len == 2 array. I want to put the elements of b into the unmasked elements of a. How do I do that? In [598]: a Out[598]: masked_array(data = [1 -- 3], mask = [False True False], fill_value=99) In [599]: b Out[599]: array([7, 8]) I'd like an operation that gives me: masked_array(data = [7 -- 8], mask = [False True False], fill_value=99) Seems like it shouldn't be that hard, but I can't figure it out. Any suggestions? How about: c = a.copy() c[~a.mask] = b Angus. -- AJC McMorland Post-doctoral research fellow Neurobiology, University of Pittsburgh ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] split matrix
Hi, is there an effective way to remove a row with a given index from a matrix ? Greetings, Uwe -- Dr. rer. nat. Uwe Schmitt FE Mathematik mineway GmbH Science Park 2 D-66123 Saarbrücken Telefon: +49 (0)681 8390 5334 Telefax: +49 (0)681 830 4376 [EMAIL PROTECTED] www.mineway.de Geschäftsführung: Dr.-Ing. Mathias Bauer Amtsgericht Saarbrücken HRB 12339 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Ironclad v0.7 released (NumPy on IronPython)
Hi all Hopefully someone here will be interested in this, and it won't be considered too spammy... please let me know if this isn't welcome, and I'll desist in future. I'm delighted to announce the release of Ironclad v0.7, which is now available from http://code.google.com/p/ironclad/downloads/list . This release is a major step forward: * Runs transparently on vanilla IronPython 2.0RC2, without creating extra PythonEngines or breaking .NET namespace imports * Many numpy 1.2 tests (from the core, fft, lib, linalg and random subpackages) now reliably pass (run ipy numpytests.py from the build directory) * Significant performance improvements (by several orders of magnitude in some places :D) So... if you want to use numpy (or other C extension modules) with IronPython on Win32, please download it and try it out; I'm very keen to hear your experiences, and to know which neglected features will be most useful to you. Cheers William ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Ironclad v0.7 released (NumPy on IronPython)
William Reade wrote: Hi all Hopefully someone here will be interested in this, and it won't be considered too spammy... please let me know if this isn't welcome, and I'll desist in future. I welcome these announcements, so my opinion is that you continue. Thanks for the work. It's great to see a path for running C extensions on IronPython. -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What happened to numpy-docs ?
Thu, 27 Nov 2008 01:13:19 -0500, Pierre GM wrote: [clip] Pauli, do you think you could put your numpyext in the doc/ directory as well ? Yes, Numpy SVN would probably be a more natural place for the stuff. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] split matrix
On Thu, 27 Nov 2008 17:03:51 +0100 Uwe Schmitt [EMAIL PROTECTED] wrote: Hi, is there an effective way to remove a row with a given index from a matrix ? A = rand(10,5) A array([[ 0.15976517, 0.29574162, 0.21537014, 0.69341324, 0.68713389], [ 0.28992634, 0.89714962, 0.90299203, 0.22203182, 0.57831945], [ 0.23814492, 0.09436163, 0.67062125, 0.85923647, 0.64548996], [ 0.83215097, 0.85178335, 0.49873409, 0.59021905, 0.94631569], [ 0.5494401 , 0.08831399, 0.54776161, 0.10043204, 0.88260609], [ 0.90951225, 0.4096, 0.78577964, 0.17414472, 0.59568316], [ 0.97491997, 0.76869065, 0.88901626, 0.69693058, 0.73576195], [ 0.25971704, 0.67759869, 0.42972164, 0.15069627, 0.13269489], [ 0.50012917, 0.5866074 , 0.32205757, 0.3347558 , 0.02555147], [ 0.66448744, 0.14755343, 0.09963282, 0.22277848, 0.35620143]]) ind array([0, 0, 0, 0, 1, 0, 0, 0, 0, 0]) A = A[ind==0,:][:,:] A array([[ 0.15976517, 0.29574162, 0.21537014, 0.69341324, 0.68713389], [ 0.28992634, 0.89714962, 0.90299203, 0.22203182, 0.57831945], [ 0.23814492, 0.09436163, 0.67062125, 0.85923647, 0.64548996], [ 0.83215097, 0.85178335, 0.49873409, 0.59021905, 0.94631569], [ 0.90951225, 0.4096, 0.78577964, 0.17414472, 0.59568316], [ 0.97491997, 0.76869065, 0.88901626, 0.69693058, 0.73576195], [ 0.25971704, 0.67759869, 0.42972164, 0.15069627, 0.13269489], [ 0.50012917, 0.5866074 , 0.32205757, 0.3347558 , 0.02555147], [ 0.66448744, 0.14755343, 0.09963282, 0.22277848, 0.35620143]]) Nils ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] What happened to numpy-docs ?
2008/11/27 Pauli Virtanen [EMAIL PROTECTED]: Thu, 27 Nov 2008 08:39:32 +0200, Scott Sinclair wrote: [clip] I have been under the impression that the documentation on the doc wiki http://docs.scipy.org/numpy/Front%20Page/ immediately (or at least very quickly) reflected changes in SVN and that changes to the docs in the wiki need to be manually checked in to SVN. Admittedly I have no good reason to make this assumption. It's manual, somebody with admin privileges must go and click a button to update it. But there's no reason why it couldn't be automatic. It should be trivial to rig up a cron job that runs whenever there are new revisions in SVN, so let's put this in the todo list. I think this is a sensible goal, people editing in the wiki may not be aware of what's happening in SVN. Nice to see that the Scipy docs are now available as well! Cheers, Scott ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Name changes and suggested file name change for Pauli.
Hi All, I'm thinking of changing the names of fmax and fmin to fmaximum and fminimum so that fmax and fmin can play the roles corresponding to max and min. Should I add the names atanh, asinh, and acosh as aliases for arctanh, arcsinh, and arccosh? The vote looked pretty evenly split. If we add them, I suggest we merely add a note to the documentation of the old functions suggesting use of the new names to conform to general practice. A while ago I added deg2rad and rad2deg as aliases for radians and degrees respectively, so this can be seen as more of the same. Pauli, can you change the name of code_generators/docstrings to something more descriptive? I think ufunc_docstrings would be a bit clearer. I expect this requires various fixups here and there, so I'm tossing the problem over to you. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
