Re: [Numpy-discussion] numpy.loadtext() fails with dtype + usecols

2008-07-21 Thread David Huard 
Looks good to me. I committed the patch to the trunk and added a regression
test (r5495).

David




2008/7/18 Charles R Harris [EMAIL PROTECTED]:



 On Fri, Jul 18, 2008 at 4:16 PM, Ryan May [EMAIL PROTECTED] wrote:

 Hi,

 I was trying to use loadtxt() today to read in some text data, and I had a
 problem when I specified a dtype that only contained as many elements as in
 columns in usecols.  The example below shows the problem:

 import numpy as np
 import StringIO
 data = '''STID RELH TAIR
 JOE 70.1 25.3
 BOB 60.5 27.9
 '''
 f = StringIO.StringIO(data)
 names = ['stid', 'temp']
 dtypes = ['S4', 'f8']
 arr = np.loadtxt(f, usecols=(0,2),dtype=zip(names,dtypes), skiprows=1)

 With current 1.1 (and SVN head), this yields:

 IndexErrorTraceback (most recent call
 last)

 /home/rmay/ipython console in module()

 /usr/lib64/python2.5/site-packages/numpy/lib/io.pyc in loadtxt(fname,
 dtype, comments, delimiter, converters, skiprows, usecols, unpack)
309 for j in xrange(len(vals))]
310 if usecols is not None:
 -- 311 row = [converterseq[j](vals[j]) for j in usecols]
312 else:
313 row = [converterseq[j](val) for j,val in
 enumerate(vals)]

 IndexError: list index out of range
 --

 I've added a patch that checks for usecols, and if present, correctly
 creates the converters dictionary to map each specified column with
 converter for the corresponding field in the dtype. With the attached patch,
 this works fine:

 arr
 array([('JOE', 25.301), ('BOB', 27.899)],
  dtype=[('stid', '|S4'), ('temp', 'f8')])

 Comments?  Can I get this in for 1.1.1?


 Can someone familiar with loadtxt comment on this patch?

 Chuck



 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] numpy.loadtext() fails with dtype + usecols

2008-07-18 Thread Ryan May 

Hi,

I was trying to use loadtxt() today to read in some text data, and I had 
a problem when I specified a dtype that only contained as many elements 
as in columns in usecols.  The example below shows the problem:


import numpy as np
import StringIO
data = '''STID RELH TAIR
JOE 70.1 25.3
BOB 60.5 27.9
'''
f = StringIO.StringIO(data)
names = ['stid', 'temp']
dtypes = ['S4', 'f8']
arr = np.loadtxt(f, usecols=(0,2),dtype=zip(names,dtypes), skiprows=1)

With current 1.1 (and SVN head), this yields:

IndexErrorTraceback (most recent call last)

/home/rmay/ipython console in module()

/usr/lib64/python2.5/site-packages/numpy/lib/io.pyc in loadtxt(fname, 
dtype, comments, delimiter, converters, skiprows, usecols, unpack)

309 for j in xrange(len(vals))]
310 if usecols is not None:
-- 311 row = [converterseq[j](vals[j]) for j in usecols]
312 else:
313 row = [converterseq[j](val) for j,val in 
enumerate(vals)]


IndexError: list index out of range
--

I've added a patch that checks for usecols, and if present, correctly 
creates the converters dictionary to map each specified column with 
converter for the corresponding field in the dtype. With the attached 
patch, this works fine:


arr
array([('JOE', 25.301), ('BOB', 27.899)],
  dtype=[('stid', '|S4'), ('temp', 'f8')])

Comments?  Can I get this in for 1.1.1?

Thanks,

Ryan

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
--- io.py.bak	2008-07-18 18:12:17.0 -0400
+++ io.py	2008-07-16 22:49:13.0 -0400
@@ -292,8 +292,13 @@
 if converters is None:
 converters = {}
 if dtype.names is not None:
-converterseq = [_getconv(dtype.fields[name][0]) \
-for name in dtype.names]
+if usecols is None:
+converterseq = [_getconv(dtype.fields[name][0]) \
+for name in dtype.names]
+else:
+converters.update([(col,_getconv(dtype.fields[name][0])) \
+for col,name in zip(usecols, dtype.names)])
+
 
 for i,line in enumerate(fh):
 if iskiprows: continue
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy.loadtext() fails with dtype + usecols

2008-07-18 Thread Charles R Harris 
On Fri, Jul 18, 2008 at 4:16 PM, Ryan May [EMAIL PROTECTED] wrote:

 Hi,

 I was trying to use loadtxt() today to read in some text data, and I had a
 problem when I specified a dtype that only contained as many elements as in
 columns in usecols.  The example below shows the problem:

 import numpy as np
 import StringIO
 data = '''STID RELH TAIR
 JOE 70.1 25.3
 BOB 60.5 27.9
 '''
 f = StringIO.StringIO(data)
 names = ['stid', 'temp']
 dtypes = ['S4', 'f8']
 arr = np.loadtxt(f, usecols=(0,2),dtype=zip(names,dtypes), skiprows=1)

 With current 1.1 (and SVN head), this yields:

 IndexErrorTraceback (most recent call last)

 /home/rmay/ipython console in module()

 /usr/lib64/python2.5/site-packages/numpy/lib/io.pyc in loadtxt(fname,
 dtype, comments, delimiter, converters, skiprows, usecols, unpack)
309 for j in xrange(len(vals))]
310 if usecols is not None:
 -- 311 row = [converterseq[j](vals[j]) for j in usecols]
312 else:
313 row = [converterseq[j](val) for j,val in
 enumerate(vals)]

 IndexError: list index out of range
 --

 I've added a patch that checks for usecols, and if present, correctly
 creates the converters dictionary to map each specified column with
 converter for the corresponding field in the dtype. With the attached patch,
 this works fine:

 arr
 array([('JOE', 25.301), ('BOB', 27.899)],
  dtype=[('stid', '|S4'), ('temp', 'f8')])

 Comments?  Can I get this in for 1.1.1?


Can someone familiar with loadtxt comment on this patch?

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion