[Numpy-discussion] [F2PY] Fortran call fails in IDLE / PyScripter

2008-12-03 Thread Christophe Chappet
What version of gfortran are you using (i.e. exactly which binary did
you download)?
GNU Fortran (GCC) 4.4.0 20080603 (experimental) [trunk revision 136333]

Is this a write to standard output write (*,*) szText ?
yes, it is.

I forgot to say that it also works with pydev in Eclipse but I'm looking for a 
simple interactive python shell that can execute some programs.
IPython does the job but is less friendly than IDLE in term of program editing. 
Anyway, I think I will use it for now.
Thanks for your reply.
Regards,
Christophe


On Tue, Dec 2, 2008 at 08:26, Christophe Chappet
[EMAIL PROTECTED] 
http://projects.scipy.org/mailman/listinfo/numpy-discussion wrote:
/ Hi all,
// I compile the followinq code using f2py -c --fcompiler=gnu95
// --compiler=mingw32 -m hello
//   subroutine AfficheMessage(szText)
//   character szText*100
//   write (*,*) szText
//   return
//   end
//
// Using python console :
//import hello
//hello.affichemessage(
// Hello)
// works fine !
//
// I do the same in the program window of IDLE and :
// - no message is displayed.
// - the shell restart (or IDLE crah if launched with -n)
//
// Same problem with PyScripter IDE. (crash)./


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


[Numpy-discussion] trouble subclassing ndarray

2008-12-03 Thread Sébastien Barthélemy
Hello,

I'm trying to write a small library of differential geometry, and I
have some trouble subclassing ndarray.

I'd like an HomogeneousMatrix class that subclasse ndarray and
overloads some methods, such as inv().

Here is my first try, the inv() function and the inv_v1() method work
as expected, but the inv_v2() and inv_v3() methods do not change the
object at all. Can somebody explain me what is happening here ?

import numpy as np
def inv(H):

inverse of an homogeneous matrix

R = H[0:3,0:3]
p = H[0:3,3:4]
return np.vstack( (np.hstack((R.T,-np.dot(R.T,p))), [0,0,0,1]))

class HomogeneousMatrix(np.ndarray):
def __new__(subtype, data=np.eye(4)):
subarr = np.array(data)
if htr.ishomogeneousmatrix(subarr):
return subarr.view(subtype)
else:
raise ValueError
def inv_v1(self):
self[0:4,0:4] = htr.inv(self)
def inv_v2(self):
data = htr.inv(self)
self = HomogeneousMatrix(data)
def inv_v3(self):
self = htr.inv(self)


Thank you !

-- 
Sébastien
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] trouble subclassing ndarray

2008-12-03 Thread Fabrice Silva
Le mercredi 03 décembre 2008, Sébastien Barthélemy a écrit :
 Hello,
Hi Sebastien!

 I'm trying to write a small library of differential geometry, and I
 have some trouble subclassing ndarray.
 I'd like an HomogeneousMatrix class that subclasse ndarray and
 overloads some methods, such as inv().
 Here is my first try, the inv() function and the inv_v1() method work
 as expected, but the inv_v2() and inv_v3() methods do not change the
 object at all. Can somebody explain me what is happening here ?
 
 import numpy as np
 def inv(H):
 
 inverse of an homogeneous matrix
 
 R = H[0:3,0:3]
 p = H[0:3,3:4]
 return np.vstack( (np.hstack((R.T,-np.dot(R.T,p))), [0,0,0,1]))
 
 class HomogeneousMatrix(np.ndarray):
 def __new__(subtype, data=np.eye(4)):
 subarr = np.array(data)
 if htr.ishomogeneousmatrix(subarr):
 return subarr.view(subtype)
 else:
 raise ValueError
 def inv_v1(self):
 self[0:4,0:4] = htr.inv(self)
 def inv_v2(self):
 data = htr.inv(self)
 self = HomogeneousMatrix(data)
 def inv_v3(self):
 self = htr.inv(self)

There is something I missed: what is htr? I guess htr.inv is the inv
function defined before the class.
Another point: it seems weird to me that, in the class' methods inv_v2
and inv_v3, you 'unref' the previous instance of HomogeneousMatrix and
link the 'self' label to a new instance... In inv_v1, you just modify
the coefficient of the Homogeneous Matrix with the coefficient of
htr.inv(self)

-- 
Fabrice Silva [EMAIL PROTECTED]
LMA UPR CNRS 7051 - équipe S2M

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] trouble subclassing ndarray

2008-12-03 Thread Sébastien Barthélemy
2008/12/3 Kevin Jacobs [EMAIL PROTECTED] [EMAIL PROTECTED]:
 On Wed, Dec 3, 2008 at 9:19 AM, Sébastien Barthélemy [EMAIL PROTECTED]
 wrote:

def inv_v1(self):
self[0:4,0:4] = htr.inv(self)
def inv_v2(self):
data = htr.inv(self)
self = HomogeneousMatrix(data)
def inv_v3(self):
self = htr.inv(self)

 self is a reference, so you're just overwriting it with references to new
 values in v2 and v3.  The original object is unchanged.  Only v1 changes
 self.  You may want to use self[:] = .

okay, it seems obvious now. I definitely spent to much time with matlab.

Thanks
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] trouble subclassing ndarray

2008-12-03 Thread Sébastien Barthélemy
2008/12/3 Fabrice Silva [EMAIL PROTECTED]:
 Le mercredi 03 décembre 2008, Sébastien Barthélemy a écrit :
 Hello,
 Hi Sebastien!

Hello Fabrice

 There is something I missed: what is htr? I guess htr.inv is the inv
 function defined before the class.

yes, I cut-n-pasted the function definition from the htr module and
forgot to tell it, sorry

Thank you
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Ryan May

Pierre GM wrote:

I think that treating an explicitly-passed-in ' ' delimiter as
identical to 'no delimiter' is a bad idea. If I say that ' ' is the
delimiter, or '\t' is the delimiter, this should be treated *just*
like ',' being the delimiter, where the expected output is:
['1', '2', '3', '4', '', '5']



Valid point.
Well, all, stay tuned for yet another yet another implementation...



Found a problem.  If you read the names from the file and specify 
usecols, you end up with the first N names read from the file as the 
fields in your output (where N is the number of entries in usecols), 
instead of having the names of the columns you asked for.


For instance:

from StringIO import StringIO
from genload_proposal import loadtxt
f = StringIO('stid stnm relh tair\nnrmn 121 45 9.1')
loadtxt(f, usecols=('stid', 'relh', 'tair'), names=True, dtype=None)
array(('nrmn', 45, 9.0996),
  dtype=[('stid', '|S4'), ('stnm', 'i8'), ('relh', 'f8')])

What I want to come out is:

array(('nrmn', 45, 9.0996),
  dtype=[('stid', '|S4'), ('relh', 'i8'), ('tair', 'f8')])

I've attached a version that fixes this by setting a flag internally if 
the names are read from the file.  If this flag is true, at the end the 
names are filtered down to only the ones that are given in usecols.


I also have one other thought.  Is there any way we can make this handle 
object arrays, or rather, a field containing objects, specifically 
datetime objects?  Right now, this does not work because calling view 
does not work for object arrays.  I'm just looking for a simple way to 
store date/time in my record array (currently a string field).


Ryan

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Proposal : 
Here's an extension to np.loadtxt, designed to take missing values into account.





import itertools
import numpy as np
import numpy.ma as ma


def _is_string_like(obj):

Check whether obj behaves like a string.

try:
obj + ''
except (TypeError, ValueError):
return False
return True

def _to_filehandle(fname, flag='r', return_opened=False):

Returns the filehandle corresponding to a string or a file.
If the string ends in '.gz', the file is automatically unzipped.

Parameters
--
fname : string, filehandle
Name of the file whose filehandle must be returned.
flag : string, optional
Flag indicating the status of the file ('r' for read, 'w' for write).
return_opened : boolean, optional
Whether to return the opening status of the file.

if _is_string_like(fname):
if fname.endswith('.gz'):
import gzip
fhd = gzip.open(fname, flag)
elif fname.endswith('.bz2'):
import bz2
fhd = bz2.BZ2File(fname)
else:
fhd = file(fname, flag)
opened = True
elif hasattr(fname, 'seek'):
fhd = fname
opened = False
else:
raise ValueError('fname must be a string or file handle')
if return_opened:
return fhd, opened
return fhd


def flatten_dtype(ndtype):

Unpack a structured data-type.


names = ndtype.names
if names is None:
return [ndtype]
else:
types = []
for field in names:
(typ, _) = ndtype.fields[field]
flat_dt = flatten_dtype(typ)
types.extend(flat_dt)
return types


def nested_masktype(datatype):

Construct the dtype of a mask for nested elements.


names = datatype.names
if names:
descr = []
for name in names:
(ndtype, _) = datatype.fields[name]
descr.append((name, nested_masktype(ndtype)))
return descr
# Is this some kind of composite a la (np.float,2)
elif datatype.subdtype:
mdescr = list(datatype.subdtype)
mdescr[0] = np.dtype(bool)
return tuple(mdescr)
else:
return np.bool


class LineSplitter:

Defines a function to split a string at a given delimiter or at given places.

Parameters
--
comment : {'#', string}
Character used to mark the beginning of a comment.
delimiter : var


def __init__(self, delimiter=None, comments='#'):
self.comments = comments
# Delimiter is a character
if delimiter is None:
self._isfixed = False
self.delimiter = None
elif _is_string_like(delimiter):
self._isfixed = False
self.delimiter = delimiter.strip() or None
# Delimiter is a list of field widths
elif hasattr(delimiter, '__iter__'):
self._isfixed = True
idx = np.cumsum([0]+list(delimiter))
self.slices = [slice(i,j) for (i,j) in zip(idx[:-1], idx[1:])]
# Delimiter is a single integer
elif int(delimiter):
self._isfixed = True

Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Alan G Isaac
If I know my data is already clean
and is handled nicely by the
old loadtxt, will I be able to turn
off and the special handling in
order to retain the old load speed?

Alan Isaac

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Christopher Barker
Pierre GM wrote:
 I can try, but in that case, please write me a unittest, so that I  
 have a clear and unambiguous idea of what you expect.

fair enough, though I'm not sure when I'll have time to do it.

I do wonder if anyone else thinks it would be useful to have multiple 
delimiters as an option. I got the idea because with fromfile(), if you 
specify, say ',' as the delimiter, it won't use '\n', only  a comma, so 
there is no way to quickly read a whole bunch of comma delimited data like:

1,2,3,4
5,6,7,8


so I'd like to be able to say to use either ',' or '\n' as the delimiter.

However, if I understand loadtxt() correctly, it's handling the new 
lines separately anyway (to get a 2-d array), so this use case isn't an 
issue. So how likely is it that someone would have:

1  2  3, 4, 5
6  7  8, 8, 9

and want to read that into a single 2-d array?

I'm not sure I've seen it.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/ORR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Pierre GM

On Dec 3, 2008, at 12:48 PM, Christopher Barker wrote:

 Pierre GM wrote:
 I can try, but in that case, please write me a unittest, so that I
 have a clear and unambiguous idea of what you expect.

 fair enough, though I'm not sure when I'll have time to do it.

Oh, don;t worry, nothing too fancy: give me a couple lines of input  
data and a line with what you expect. Using Ryan's recent example:

 f = StringIO('stid stnm relh tair\nnrmn 121 45 9.1')
  test = loadtxt(f, usecols=('stid', 'relh', 'tair'), names=True,  
dtype=None)
  control=array(('nrmn', 45, 9.0996),
 dtype=[('stid', '|S4'), ('relh', 'i8'), 
('tair', 'f8')])

That's quite enough for a test.

 I do wonder if anyone else thinks it would be useful to have multiple
 delimiters as an option. I got the idea because with fromfile(), if  
 you
 specify, say ',' as the delimiter, it won't use '\n', only  a comma,  
 so
 there is no way to quickly read a whole bunch of comma delimited  
 data like:

 1,2,3,4
 5,6,7,8
 

 so I'd like to be able to say to use either ',' or '\n' as the  
 delimiter.

I'm not quite sure I follow you.
Do you want to delimiters, one for the field of a record (','), one  
for the records (\n) ?




 However, if I understand loadtxt() correctly, it's handling the new
 lines separately anyway (to get a 2-d array), so this use case isn't  
 an
 issue. So how likely is it that someone would have:

 1  2  3, 4, 5
 6  7  8, 8, 9

 and want to read that into a single 2-d array?

With the current behaviour, you gonna have
[(1 2 3, 4, 5), (6 7 8, 8, 9)] if you use , as a delimiter,
[(1,2,3,,4,,5),(6,7,8,,8,,9)] if you use   as a delimiter.

Mixing delimiter is doable, but I don't think it's that a good idea.  
I'm in favor of sticking to one and only field delimiter, and the  
default line spearator for record delimiter. In other terms, not  
changing anythng.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Pierre GM

On Dec 3, 2008, at 12:32 PM, Alan G Isaac wrote:

 If I know my data is already clean
 and is handled nicely by the
 old loadtxt, will I be able to turn
 off and the special handling in
 order to retain the old load speed?

Hopefully. I'm looking for the best way to do it. Do you have an  
example you could send me off-list so that I can play with timers ?  
Thx in advance.
P.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Christopher Barker
by the way, should this work:

io.loadtxt('junk.dat', delimiter=' ')

for more than one space between numbers, like:

1  2  3  4   5
6  7  8  9  10

I get:

io.loadtxt('junk.dat', delimiter=' ')
Traceback (most recent call last):
   File stdin, line 1, in module
   File 
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/io.py,
 
line 403, in loadtxt
 X.append(tuple([conv(val) for (conv, val) in zip(converters, vals)]))
ValueError: empty string for float()

with the current version.

  io.loadtxt('junk.dat', delimiter=None)
array([[  1.,   2.,   3.,   4.,   5.],
[  6.,   7.,   8.,   9.,  10.]])

does work.



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/ORR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Christopher Barker
Pierre GM wrote:
 Oh, don;t worry, nothing too fancy: give me a couple lines of input  
 data and a line with what you expect.

I just went and looked at the existing tests, and you're right, it's 
very easy -- my first foray into the new nose tests -- very nice!

 specify, say ',' as the delimiter, it won't use '\n', only  a comma,  
 so
 there is no way to quickly read a whole bunch of comma delimited  
 data like:

 1,2,3,4
 5,6,7,8
 

 so I'd like to be able to say to use either ',' or '\n' as the  
 delimiter.
 
 I'm not quite sure I follow you.
 Do you want to delimiters, one for the field of a record (','), one  
 for the records (\n) ?

well, in the case of fromfile(), it doesn't do records -- it will only 
give you a 1-d array, so I want it all as a flat array, and you can 
re-size it yourself later. Clearly this is more work (and requires more 
knowledge of your data) than using loadtxt, but sometimes I really want 
FAST data reading of simple formats.

However, this isn't fromfile() we are talking about now, it's loadtxt()...

 So how likely is it that someone would have:

 1  2  3, 4, 5
 6  7  8, 8, 9

 and want to read that into a single 2-d array?
 
 With the current behaviour, you gonna have
 [(1 2 3, 4, 5), (6 7 8, 8, 9)] if you use , as a delimiter,
 [(1,2,3,,4,,5),(6,7,8,,8,,9)] if you use   as a delimiter.

right.

 Mixing delimiter is doable, but I don't think it's that a good idea.  

I can't come up with a use case at this point, so..

 I'm in favor of sticking to one and only field delimiter, and the  
 default line spearator for record delimiter. In other terms, not  
 changing anything.

I agree -- sorry for the noise!

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/ORR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Christopher Barker
Alan G Isaac wrote:
 If I know my data is already clean
 and is handled nicely by the
 old loadtxt, will I be able to turn
 off and the special handling in
 order to retain the old load speed?

what I'd like to see is a version of loadtxt built on a slightly 
enhanced fromfile() -- that would be blazingly fast for the easy cases 
(simple tabular data of one dtype).

I don't know if the special-casing should be automatic, or just have it 
be a separate function.

Also, fromfile() needs some work, and it needs to be done in C, which is 
less fun, so who knows when it will get done.

As I think about it, maybe what I really want is a simple version of 
loadtxt written in C:

  It would only handle one data type at a time.
  It would support simple comment lines.
  It would only support one delimiter (plus newline).
  It would create a 2-d array from normal, tabular data.
  You could specify:
 how many numbers you wanted,
 or how many rows,
 or read 'till EOF

Actually, this is a lot like matlab's fscanf()

someday

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/ORR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Pierre GM

On Dec 3, 2008, at 1:00 PM, Christopher Barker wrote:

 by the way, should this work:

 io.loadtxt('junk.dat', delimiter=' ')

 for more than one space between numbers, like:

 1  2  3  4   5
 6  7  8  9  10


On the version I'm working on, both delimiter='' and delimiter=None  
(default) would give you the expected output. delimiter=' ' would  
fail, delimiter='  ' would work.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Manuel Metz
Alan G Isaac wrote:
 If I know my data is already clean
 and is handled nicely by the
 old loadtxt, will I be able to turn
 off and the special handling in
 order to retain the old load speed?
 
 Alan Isaac
 

Hi all,
  that's going in the same direction I was thinking about.
When I thought about an improved version of loadtxt, I wished it was
fault tolerant without loosing too much performance.
  So my solution was much simpler than the very nice genloadtxt function
-- and it works for me.

My ansatz is to leave the existing loadtxt function unchanged. I only
replaced the default converter calls by a fault tolerant converter
class. I attached a patch against io.py in numpy 1.2.1

The nice thing is that it not only handles missing values, but for
example also columns/fields with non-number characters. It just returns
nan in these cases. This is of practical importance for many datafiles
of astronomical catalogues, for example the Hipparcos catalogue data.

Regarding the performance, it is a little bit slower than the original
loadtxt, but not much: on my machine, 10x reading in a clean testfile
with 3 columns and 2 rows I get the following results:

original loadtxt:  ~1.3s
modified loadtxt:  ~1.7s
new genloadtxt  :  ~2.7s

So you see, there is some loss of performance, but not as much as with
the new converter class.

I hope this solution is of interest ...

Manuel
237a238,247
 class _faultsaveconv(object):
 def __init__(self,conv):
 self._conv = conv
 
 def __call__(self, x):
 try:
 return self._conv(x)
 except:
 return np.nan
 
241c251
 return lambda x: bool(int(x))
---
 return _faultsaveconv(lambda x: bool(int(x)))
243c253
 return lambda x: int(float(x))
---
 return _faultsaveconv(lambda x: int(float(x)))
245c255
 return float
---
 return _faultsaveconv(float)
247c257
 return complex
---
 return _faultsaveconv(complex)
249c259
 return str
---
 return _faultsaveconv(str)
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Christopher Barker
Pierre GM wrote:
 On Dec 3, 2008, at 1:00 PM, Christopher Barker wrote:

 for more than one space between numbers, like:

 1  2  3  4   5
 6  7  8  9  10
 
 
 On the version I'm working on, both delimiter='' and delimiter=None  
 (default) would give you the expected output.

so empty string and None both mean any white space? also tabs, etc?

 delimiter=' ' would  fail,

s only exactly that delimiter.

Is that so things like '\t' will work right?

but what about:

4,  5, 34,123, 

In that case, ',' is the delimiter, but whitespace is ignored.
or

4\t  5\t 34\t  123.

we're ignoring extra whitespace there, too, so I'm not sure why we 
shouldn't ignore it in the ' ' case also.

  delimiter='  ' would work.

but in my example, there were sometimes two spaces, sometimes three -- 
so I think it would fail, no?
  1  2  3  4   5.split('  ')
['1', '2', '3', '4', ' 5']

actually, that would work, but four spaces wouldn't.

  1  2  3  45.split('  ')
['1', '2', '3', '4', '', '5']

I guess the solution is to use delimiter=None in that case, and is does 
make sense that you can't have ' ' mean one or more spaces, but \t 
mean only one tab.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/ORR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Manuel Metz
Manuel Metz wrote:
 Alan G Isaac wrote:
 If I know my data is already clean
 and is handled nicely by the
 old loadtxt, will I be able to turn
 off and the special handling in
 order to retain the old load speed?

 Alan Isaac

 
 Hi all,
   that's going in the same direction I was thinking about.
 When I thought about an improved version of loadtxt, I wished it was
 fault tolerant without loosing too much performance.
   So my solution was much simpler than the very nice genloadtxt function
 -- and it works for me.
 
 My ansatz is to leave the existing loadtxt function unchanged. I only
 replaced the default converter calls by a fault tolerant converter
 class. I attached a patch against io.py in numpy 1.2.1
 
 The nice thing is that it not only handles missing values, but for
 example also columns/fields with non-number characters. It just returns
 nan in these cases. This is of practical importance for many datafiles
 of astronomical catalogues, for example the Hipparcos catalogue data.
 
 Regarding the performance, it is a little bit slower than the original
 loadtxt, but not much: on my machine, 10x reading in a clean testfile
 with 3 columns and 2 rows I get the following results:
 
 original loadtxt:  ~1.3s
 modified loadtxt:  ~1.7s
 new genloadtxt  :  ~2.7s
 
 So you see, there is some loss of performance, but not as much as with
 the new converter class.
 
 I hope this solution is of interest ...
 
 Manuel


Oops, wrong version of the diff file. Wanted to name the class
_faulttolerantconv ...



237a238,247
 class _faulttolerantconv(object):
 def __init__(self,conv):
 self._conv = conv
 
 def __call__(self, x):
 try:
 return self._conv(x)
 except:
 return np.nan
 
241c251
 return lambda x: bool(int(x))
---
 return _faulttolerantconv(lambda x: bool(int(x)))
243c253
 return lambda x: int(float(x))
---
 return _faulttolerantconv(lambda x: int(float(x)))
245c255
 return float
---
 return _faulttolerantconv(float)
247c257
 return complex
---
 return _faulttolerantconv(complex)
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] np.loadtxt : yet a new implementation...

2008-12-03 Thread Pierre GM
Manuel,
Looks nice, I gonna try to see how I can incorporate yours. Note that  
returning np.nan by default will not work w/ Python 2.6 if you want an  
int...


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


[Numpy-discussion] Apply a function to an array elementwise

2008-12-03 Thread Elfnor

Hi

I want to apply a function (myfunc which takes and returns a scalar) to each
element in a multi-dimensioned array (data):

I can do this:

newdata = numpy.array([myfunc(d) for d in data.flat]).reshape(data.shape)

But I'm wondering if there's a faster more numpy way. I've looked at the
vectorize function but can't work it out.

thanks Eleanor
-- 
View this message in context: 
http://www.nabble.com/Apply-a-function-to-an-array-elementwise-tp20823768p20823768.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Apply a function to an array elementwise

2008-12-03 Thread Travis Oliphant
Elfnor wrote:
 Hi

 I want to apply a function (myfunc which takes and returns a scalar) to each
 element in a multi-dimensioned array (data):

 I can do this:

 newdata = numpy.array([myfunc(d) for d in data.flat]).reshape(data.shape)

 But I'm wondering if there's a faster more numpy way. I've looked at the
 vectorize function but can't work it out.

   

from numpy import vectorize

new_func = vectorize(myfunc)
newdata = new_func(data)

Should work.

-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Compiler options for mingw?

2008-12-03 Thread David Cournapeau
On Sun, Nov 30, 2008 at 3:02 PM, David Cournapeau
[EMAIL PROTECTED] wrote:


 No at the moment, but you can easily decompress the .exe content to get
 the internal .exe (which are straight installers built by python
 setup.py setup.py bdist_wininst). It should be possible to force an
 architecture at install time using a command line option, but I don't
 have the time ATM to support this.

I needed it to help me fixing a couple of bugs for old CPU, so it
ended up being implemented in the nsis script for scipy now (I will
add it to numpy installers too). So from now, any newly releases of
both numpy and scipy installers could be overriden:

installer-name.exe /arch native - default behavior
installer-name.exe /arch nosse - Force installation wo sse, even if
SSE-cpu is detected.

It does not check that the option is valid, so you can end up
requesting SSE3 installer on a SSE2 CPU. But well...

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion