Mark Janikas wrote:
I am finding that directly packing numpy arrays into binary using the
tostring and fromstring methods
For starters, use fromfile and tofile, to save the overhead of creating
an entire extra string.
fromfile is a function (as it is an alternate constructor for arrays):
numpy.fromfile()
ndarray.tofile() is an array method.
Enclosed is your test, including a test for tofile(), I needed to make
the arrays much larger, and use time.time() rather than time.clock() to
get enough time resolution to see anything, though if you really want to
be accurate, you need to use the timeit module.
My results:
Using lists 0.457561016083
Using tostring 0.00922703742981
Using tofile 0.00431108474731
Another note: where is the data coming from -- there may be ways to
optimize this whole process if we saw that.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
[EMAIL PROTECTED]
#!/usr/bin/env python
from numpy import *
import numpy.random as RAND
import time as TIME
x = RAND.random(100000)
xl = x.tolist()
t1 = TIME.time()
xStringOut = [ str(i) for i in xl ]
xStringOut = " ".join(xStringOut)
f = file('blah.dat','w'); f.write(xStringOut)
t2 = TIME.time()
total = t2 - t1
print "Using lists", total
t1 = TIME.time()
f = file('blah.bwt','wb')
xBinaryOut = x.tostring()
f.write(xBinaryOut)
t2 = TIME.time()
total1 = t2 - t1
print "Using tostring", total1
t1 = TIME.time()
f = file('blah.bwt','wb')
x.tofile(f)
t2 = TIME.time()
total1 = t2 - t1
print "Using tofile", total1
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
