At 09:00 AM 11/1/2007, you wrote:
I saw that Numeric did also (I still use Numeric for smaller array
speed) but much more slowly.
I will try to repeat with a small demo and post.
It turns out to be some aspect of mixing numpy and Numeric;
the attached *Stable.py files allocate memory that stays exactly the
same
mixedGrows.py grows by about 50kB/s while changing slightly,
randomly; note that the array is assigned as numpy and operated on
with Numeric functions.
My main app also uses mmap/numpy to assign the arrays to shared
memory for multiple processes, and does other operations as well,
possibly accounting for the faster memory growth.
I'll re-factor to an all-numpy solution and test.
I used Numeric functions for the ~40% speed increase, but I don't
have a compiled version of arrayfrombuffer() for Numeric
http://www.canonical.org/~kragen/sw/arrayfrombuffer/ (compiler
version woes)
so, I must use numpy.frombuffer() with mmap to create the shared
arrays...
Ray
import numpy
import numpy.fft as fft
CIRC_BUFFER_SIZE = 2**16
LARGE_FFT_SIZE = 2048*8
numpy.random.seed()
dataArray = numpy.zeros((CIRC_BUFFER_SIZE*2,), numpy.int32)
while 1:
fftArray = abs(fft.rfft(dataArray[:LARGE_FFT_SIZE]))
fftArray[:-1] = fftArray[1:] ## shift the bins lose the DC
maxBin = numpy.argmax(fftArray[1000:5000])+1000
print "\r%d" % (maxBin),
dataArray[0:CIRC_BUFFER_SIZE] = numpy.random.binomial(5000000, .5,
(CIRC_BUFFER_SIZE,))
import RandomArray
import Numeric
import FFT
CIRC_BUFFER_SIZE = 2**16
LARGE_FFT_SIZE = 2048*8
RandomArray.seed()
def freqAnalysis():
global infoArray,dataArray
fftArray = abs(FFT.real_fft(dataArray[:LARGE_FFT_SIZE]))
maxBin = Numeric.argmax(fftArray[1000:5000])+1000
return maxBin
def getMoreData():
global infoArray,dataArray
dataArray[0:CIRC_BUFFER_SIZE] = RandomArray.binomial(5000000, .5,
(CIRC_BUFFER_SIZE,))
## small array to hold shared variables
## 0 -> bufferInsertPos ## 1 -> dataTotSamples
## 2 -> sampPerRot ## 3 -> run signal
infoArray = Numeric.zeros((4,), Numeric.Float32)
dataArray = Numeric.zeros((CIRC_BUFFER_SIZE*2,), Numeric.Int32)
while 1:
infoArray[2] = freqAnalysis() ## sampPerRot
print "\r%d" % (infoArray[2]),
getMoreData()
import RandomArray
import Numeric
import FFT
import mmap
import numpy
CIRC_BUFFER_SIZE = 2**16
LARGE_FFT_SIZE = 2048*8
RandomArray.seed()
dataArray = numpy.zeros((CIRC_BUFFER_SIZE*2,), numpy.int32)
while 1:
fftArray = abs(FFT.real_fft(dataArray[:LARGE_FFT_SIZE]))
fftArray[:-1] = fftArray[1:] ## shift the bins lose the DC
maxBin = Numeric.argmax(fftArray[1000:5000])+1000
print "\r%d" % (maxBin),
dataArray[0:CIRC_BUFFER_SIZE] = RandomArray.normal(5000000, 2000000,
(CIRC_BUFFER_SIZE,))
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
