[Numpy-discussion] Simple Loadtxt question
I have a file with thousands of lines like this:
Signal was returned in 204 microseconds
Signal was returned in 184 microseconds
Signal was returned in 199 microseconds
Signal was returned in 4274 microseconds
Signal was returned in 202 microseconds
Signal was returned in 189 microseconds
I try to read it like this:
data = np.loadtxt('dummy.data', dtype={'names':('label','times','musec'),
'fmts':('|S23','i8','|S13')})
It fails, I think, because it wants a string format and field for each of the
words 'Signal' 'was' 'returned' etc.
Can I make it treat that whole string before the number as one string, one
field? All I really care about is the numbers anyway.
Any advice appreciated.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] dtype and shape for 1.6.1 seems broken?
This works under 1.5.1 and 1.6.0 but gives me errors in 1.6.1
import numpy as np
def main():
printnumpy version: + np.__version__
zdt = np.dtype([('et','i4'),('r','f8',3)])
zdata = np.loadtxt('zdum.txt', zdt)
In 1.6.1 I get this error:
ValueError: setting an array element with a sequence. Is this a known problem?___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Quaternion dtype for NumPy - initial implementation available
On Jul 28, 2011, at 7:42 AM, Martin Ling wrote: On Wed, Jul 27, 2011 at 10:29:08PM -0500, Robert Love wrote: To use quaternions I find I often need conversion to/from matrices and to/from Euler angles. Will you add that functionality? Yes, I intend to. Note that these conversions are already available in the standalone (non-dtype) implementation in imusim.maths.quaternions: http://www.imusim.org/docs/api/imusim.maths.quaternions.Quaternion-class.html#setFromEuler http://www.imusim.org/docs/api/imusim.maths.quaternions.Quaternion-class.html#toEuler http://www.imusim.org/docs/api/imusim.maths.quaternions.Quaternion-class.html#setFromMatrix http://www.imusim.org/docs/api/imusim.maths.quaternions.Quaternion-class.html#toMatrix I should do a new release though - the Euler methods there only support ZYX and ZXY order conversions, my development version supports any order. Will you handle the left versor and right versor versions? I don't know what this means. Please enlighten me and I'll be happy to try! I thought a 'right versor' was a unit quaternion representing an angle of 90 degrees (as in 'right angle') - I don't see what a 'left' one would be. Quaternions have a handedness or a sign convention. The recently departed Space Shuttle used a Left versor convention while most things, including Space Station, use the right versor convention, in their flight software. Made for frequent confusion. Let me see if I can illustrate by showing the functions I use for converting a matrix to a quaternion. def Quaternion_Of(m): Returns a quaternion in the right versor sense. q = N.zeros(4,float) q[0] = 0.5*sqrt(1.0 + m[0,0] + m[1,1] + m[2,2]) q04_inv = 1.0/(4.0*q[0]) q[1] = (m[1,2] - m[2,1])*q04_inv q[2] = (m[2,0] - m[0,2])*q04_inv q[3] = (m[0,1] - m[1,0])*q04_inv return q def Quaternion_Of(m): Returns a quaternion in the left versor sense. q = N.zeros(4,float) q[0] = 0.5*sqrt(1.0 + m[0,0] + m[1,1] + m[2,2]) q04_inv = 1.0/(4.0*q[0]) q[1] = (m[2,1] - m[1,2])*q04_inv q[2] = (m[0,2] - m[2,0])*q04_inv q[3] = (m[1,0] - m[0,1])*q04_inv return q Or transforming a vector using the different conventions. def Transform(q,v): Returns the vector part of q*vq which transforms v from one coordinate system to another. Right Versor u = Q.Vector_Part(q) return 2.0*(q[0]*N.cross(v,u) + N.dot(v,u)*u + (q[0]*q[0] - 0.5)*v) def Transform(q,v): Returns the vector part of q*vq which transforms v from one coordinate system to another. Left Versor u = Q.Vector_Part(q) return 2.0*(q[0]*N.cross(u,v) + N.dot(u,v)*u + (q[0]*q[0] - 0.5)*v) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Quaternion dtype for NumPy - initial implementation available
To use quaternions I find I often need conversion to/from matrices and to/from Euler angles. Will you add that functionality? Will you handle the left versor and right versor versions? I have a set of pure python code I've sketched out for my needs (aerospace) but would be happy to have an intrinsic Numpy solution. On Jul 16, 2011, at 9:50 AM, Martin Ling wrote: Hi all, I have just pushed a package to GitHub which adds a quaternion dtype to NumPy: https://github.com/martinling/numpy_quaternion Some backstory: on Wednesday I gave a talk at SciPy 2011 about an inertial sensing simulation package I have been working on (http://www.imusim.org/). One component I suggested might be reusable from that code was the quaternion math implementation, written in Cython. One of its features is a wrapper class for Nx4 NumPy arrays that supports efficient operations using arrays of quaternion values. Travis Oliphant suggested that a quaternion dtype would be a better solution, and got me talking to Mark Weibe about this. With Mark's help I completed this initial version at yesterday's sprint session. Incidentally, how to do something like this isn't well documented and I would have had little hope without both Mark's in-person help and his previous code (for adding a half-precision float dtype) to refer to. I don't know what the consensus is about whether people writing custom dtypes is a desirable thing, but if it is then the process needs to be made a lot easier. That said, the fact this is doable without patching the numpy core at all is really, really nice. Example usage: import numpy as np import quaternion np.quaternion(1,0,0,0) quaternion(1, 0, 0, 0) q1 = np.quaternion(1,2,3,4) q2 = np.quaternion(5,6,7,8) q1 * q2 quaternion(-60, 12, 30, 24) a = np.array([q1, q2]) a array([quaternion(1, 2, 3, 4), quaternion(5, 6, 7, 8)], dtype=quaternion) exp(a) array([quaternion(1.69392, -0.78956, -1.18434, -1.57912), quaternion(138.909, -25.6861, -29.9671, -34.2481)], dtype=quaternion) The following ufuncs are implemented: add, subtract, multiply, divide, log, exp, power, negative, conjugate, copysign, equal, not_equal, less, less_equal, isnan, isinf, isfinite, absolute Quaternion components are stored as doubles. The package could be extended to support e.g. qfloat, qdouble, qlongdouble Comparison operations follow the same lexicographic ordering as tuples. The unary tests isnan, isinf and isfinite return true if they would return true for any individual component. Real types may be cast to quaternions, giving quaternions with zero for all three imaginary components. Complex types may also be cast to quaternions, with their single imaginary component becoming the first imaginary component of the quaternion. Quaternions may not be cast to real or complex types. Comments very welcome. This is my first attempt at NumPy hacking :-) Martin ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: EPD 7.1 released
How does this match up with the recently announced release of ETS-4.0? Are the versions of the python modules the same? On Jul 8, 2011, at 12:37 AM, Ilan Schnell wrote: Hello, I am pleased to announce that EPD (Enthought Python Distribution) version 7.1 has been released. The most significant change is the addition of an EPD Free version, which has its own very liberal license, and can be downloaded and used free of any charge by anyone (not only academics). EPD Free includes a subset of the packages included in the full EPD. The highlights of this subset are: numpy, scipy, matplotlib, traits and chaco. To see which libraries are included in the free vs. full version, please see: http://www.enthought.com/products/epdlibraries.php In addition we have opened our PyPI build mirror for everyone. This means that one can type enpkg xyz for 10,000+ packages. However, there are still benefits to becoming an EPD subscriber. http://www.enthought.com/products/getepd.php Apart from the addition of EPD Free, this release includes updates to over 30 packages, including numpy, scipy, ipython and ETS. We have also added PySide, Qt and MDP to this release. Please find the complete list of additions, updates and bug fixes in the change log: http://www.enthought.com/products/changelog.php About EPD - The Enthought Python Distribution (EPD) is a kitchen-sink-included distribution of the Python programming language, including over 90 additional tools and libraries. The EPD bundle includes NumPy, SciPy, IPython, 2D and 3D visualization, and many other tools. EPD is currently available as a single-click installer for Windows XP, Vista and 7, MacOSX (10.5 and 10.6), RedHat 3, 4 and 5, as well as Solaris 10 (x86 and x86_64/amd64 on all platforms). All versions of EPD (32 and 64-bit) are free for academic use. An annual subscription including installation support is available for individual and commercial use. Additional support options, including customization, bug fixes and training classes are also available: http://www.enthought.com/products/epd_sublevels.php - Ilan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Importing a list into a structured numpy array, what is my error?
my_list = [[1960, 'W0'],[2001, 'D5']]
my_arr = np.array(my_list, dtype=('uint', '|S2'))
gives me an error I don't understand:
ValueError: mismatch in size of old and new data-descriptor
What are the old and new? I get the same problem if try np.asarray()
If I make the dtype ('|S4', '|S4') I can read in as strings without errors
How do I convert a list of mixed types to a structured array?
Thanks for any pointers.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Arrays as nested dtype
Using np.loadtxt I can easily read my file that has columns of time, mode, 3
float64 for position and 3 for velocity like this.
dt = dtype([('time', '|S12'),
('mode','|S3'),('rx','f8'),('ry','f8'),('rz','f8'),('vx','f8'),('vy','f8'),('vz','f8')])
data = np.loadtxt('file', dtype=dt)
I can then put the two pairs of 3 components into np.arrays and start
performing the vector operations I need.
How can I read them directly into np.arrays?
dt = dtype([('time', '|S12'), ('mode','|S3'),np.array('r'), np.array('v')])
I've seen examples for nested data that create a tuple but not an array. Any
tips appreciated.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Reading and Comparing Two Files
On Feb 28, 2010, at 6:24 AM, Friedrich Romstedt wrote: 2010/2/28 Robert Love rblove_li...@comcast.net: What is the efficient numpy way to compare data from two different files? For the nth line in each file I want to operate on the numbers. I've been using loadtxt() data_5x5 = N.loadtxt(file5) data_8x8 = N.loadtxt(file8) for line in data_5x5: pos5 = N.array([line[0], line[1], line[2]]) I believe there are several ways of doing that, and mine might not be the most efficient at all: for line5, line8 in zip(data_5x5, data_8x8): # line5 and line8 are row vectors of paired lines pass complete = numpy.hstack(data_5x5, data_8x8) # If data_5x5.shape[0] == data_8x8.shape[0], i.e., same number of rows. for line in complete: # complete is comprised of concatenated row vectors. pass for idx in xrange(0, min(data_5x5.shape[0], data_8x8.shape[0])): line5 = data_5x5[idx] line8 = data_8x8[idx] # Do sth with the vectors. Or: a1 = data_5x5[idx, (0, 1, 2)] # Extract items 0, 1, 2 of line idx of first file. a2 = data_8x8[idx, (0, 42)] # Extract items 0, 42 of line idx of second file. Thank you, I will try this last method listed. I need to actually compute with the values from the two files to perform my comparison and the time tag is in different formats. Your method will get me access to the contents of two files. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Reading and Comparing Two Files
What is the efficient numpy way to compare data from two different files? For the nth line in each file I want to operate on the numbers. I've been using loadtxt() data_5x5 = N.loadtxt(file5) data_8x8 = N.loadtxt(file8) for line in data_5x5: pos5 = N.array([line[0], line[1], line[2]]) This works fine for one file but how to I get the same line's worth of data from the other file? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Enthought Python Distribution
On Jul 2, 2008, at 9:34 AM, Travis Vaught wrote: Greetings, We're pleased to announce the beta release of the Enthought Python Distribution for *Mac OS X*. Oh Happiness, Oh Joy. I'm downloading it now. Have a virtual beer on me until I can get to Austin and buy you all a real one. /s/ Bob ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: Enthought Python Distribution - Beta
I'd drive to Austin and wash and wax their cars if there was an OSX distribution. OK, mild exaggeration but I'd buy them a beer or two. On Feb 28, 2008, at 9:18 AM, Steve Lianoglou wrote: On Wed, 27 Feb 2008, Travis Vaught apparently wrote: http://www.enthought.com/epd Looks good. An increasing number of my students are buying Macs, so the OSX support will be very welcome. Yeah ... agreed, this is great! -steve ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
