I have never tried it but you can save a file in process A and open it from another process, that's what I do. The H5py or pytables forum/list should be able to give you a better answer.
-mab "The world goin' one way, people another!" - Poot > Date: Mon, 9 May 2011 10:26:23 -0400 > Subject: Re: [PyQt] pyqt4 amd multiprocessing > From: [email protected] > To: [email protected] > CC: [email protected]; [email protected] > > a dummy question, is possible, simple and fast to send a big array of > integer, floats, etc, lets say, a 512x512x1500 array of unsigned int > using hdf5 from one process to another? > > > > On Mon, May 9, 2011 at 10:16 AM, Belzile Marc-André <[email protected]> > wrote: > > Indeed, QProcess is meant to run external python scripts, which allows > > python to run one script per available core. > > e.g. > > p = QProcess('myscript.py "data"' ) > > p.start() > > Look at the following files for a more an in-depth example: > > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/process_pool.py > > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/loader_process.py > > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/iceloader.py > > > > I'm using the H5py module for dealing with HDF5 files, it has nice support > > for numpy and easy to use. You can also use pytable which is also a popular > > choice. > > -mab > > "The world goin' one way, people another!" - Poot > > > > > > > >> Date: Mon, 9 May 2011 09:46:43 -0400 > >> Subject: Re: [PyQt] pyqt4 amd multiprocessing > >> From: [email protected] > >> To: [email protected] > >> CC: [email protected]; [email protected] > >> > >> sound interesting but, how do you use QProcess, what I have read about > >> it is that this class is for executing external applications (yes, > >> processes), how to use QProcess in a pool. Could you tell me more > >> about HDF5, for example, how do you use it with python. My dataset > >> could be very big (or what I think is big), for example, a serie of > >> 1500 dicom images so I think that HDF5 could be useful to me. > >> > >> On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André <[email protected]> > >> wrote: > >> > As an alternative to python multiprocessing module, I'm using QProcess > >> > for > >> > loading and exporting files with my own pool manager. It's pretty darn > >> > fast > >> > and works well, my data sets are managed with HDF5 which takes care of > >> > handling huge data sets pretty easily. If you want to pass data back to > >> > the > >> > UI you can use QSharedMemory to give the GUI access to your data. > >> > Check out my project for a QProcess example. > >> > http://code.google.com/p/ice-cache-explorer/ > >> > > >> > -mab > >> > "The world goin' one way, people another!" - Poot > >> > > >> > > >> > > >> >> Date: Mon, 9 May 2011 09:10:20 -0400 > >> >> From: [email protected] > >> >> To: [email protected] > >> >> CC: [email protected] > >> >> Subject: Re: [PyQt] pyqt4 amd multiprocessing > >> >> > >> >> I am very interested in the answer because I have plan to use > >> >> multiprocessing module to do my hard work (because of the python GIL), > >> >> my future design is something like Samuele say, a pool of process > >> >> ready to do some hard job (using multiprocessing) and the GUI with > >> >> pyqt4 and vtk, data pass back and forth between the gui and the worker > >> >> processes (data will be vtk datasets) > >> >> > >> >> On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno > >> >> <[email protected]> > >> >> wrote: > >> >> > Hi, > >> >> > > >> >> > I would like to use the multiprocessing module to create a set of > >> >> > sounds > >> >> > (as numpy arrays) in parallel. > >> >> > The parameters of the sounds (duration, volume, etc..) are controlled > >> >> > by > >> >> > a pyqt4 GUI, and the sounds > >> >> > need to be passed back to the GUI for further processing and > >> >> > presentation. The sounds are generated > >> >> > in a separate module from the GUI. I'm attaching a minimal example > >> >> > with > >> >> > a file that creates the gui and > >> >> > a second file that contains the functions for generating the sounds. > >> >> > > >> >> > I've had mixed success so far, it seems to work on Kubuntu Natty - > >> >> > Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3 > >> >> > but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 - > >> >> > PyQt 4.7.2 with the following error: > >> >> > > >> >> > Traceback (most recent call last): > >> >> > File "test_gui.py", line 20, in onClickButton1 > >> >> > self.doTrial() > >> >> > File "test_gui.py", line 32, in doTrial > >> >> > x = complexTone(F0, lowHarm, highHarm, level, duration, ramp, > >> >> > channel, fs, maxLevel) > >> >> > File "/media/ntfsShared/mptest/simple_test/stim.py", line 45, in > >> >> > complexTone > >> >> > pool.join() > >> >> > File "/usr/lib/python2.6/multiprocessing/pool.py", line 342, in join > >> >> > p.join() > >> >> > File "/usr/lib/python2.6/multiprocessing/process.py", line 119, in > >> >> > join > >> >> > res = self._popen.wait(timeout) > >> >> > File "/usr/lib/python2.6/multiprocessing/forking.py", line 117, in > >> >> > wait > >> >> > return self.poll(0) > >> >> > File "/usr/lib/python2.6/multiprocessing/forking.py", line 106, in > >> >> > poll > >> >> > pid, sts = os.waitpid(self.pid, flag) > >> >> > OSError: [Errno 4] Interrupted system call > >> >> > > >> >> > Am I doing something fundamentally unsafe by using multiprocessing in > >> >> > this way with a pyqt4 GUI? > >> >> > What is the best approach to use multiprocessing together with pyqt4? > >> >> > > >> >> > Thanks for any suggestions! > >> >> > > >> >> > Sam > >> >> > > >> >> > ###---------GUI----------File 1 > >> >> > from PyQt4 import QtGui > >> >> > from PyQt4 import QtCore > >> >> > import sys > >> >> > from stim import* #import module for stimulus generation > >> >> > > >> >> > > >> >> > class Example(QtGui.QWidget): > >> >> > > >> >> > def __init__(self): > >> >> > super(Example, self).__init__() > >> >> > self.setGeometry(300, 300, 250, 150) > >> >> > self.setWindowTitle('Multiprocessing Test') > >> >> > > >> >> > self.button1 = QtGui.QPushButton('Button 1', self) > >> >> > QtCore.QObject.connect(self.button1, > >> >> > QtCore.SIGNAL('clicked()'), > >> >> > self.onClickButton1) > >> >> > > >> >> > def onClickButton1(self): > >> >> > self.doTrial() > >> >> > > >> >> > def doTrial(self): > >> >> > F0 = 200 > >> >> > lowHarm=1 > >> >> > highHarm = 20 > >> >> > level = 50 > >> >> > duration = 180 > >> >> > ramp = 10 > >> >> > channel = "Both" > >> >> > fs = 44100 > >> >> > maxLevel = 100.0 > >> >> > x = complexTone(F0, lowHarm, highHarm, level, duration, ramp, > >> >> > channel, fs, maxLevel) > >> >> > > >> >> > > >> >> > if __name__ == '__main__': > >> >> > app = QtGui.QApplication(sys.argv) > >> >> > ex = Example() > >> >> > ex.show() > >> >> > app.exec_() > >> >> > #----------END OF FILE 1 > >> >> > > >> >> > > >> >> > ###-SOUND GENERATION-- File stim.py > >> >> > from numpy import* > >> >> > import multiprocessing > >> >> > > >> >> > def pureTone(frequency, phase, level, duration, ramp, channel, fs, > >> >> > maxLevel): > >> >> > > >> >> > amp = 10**((level - maxLevel) / 20.) > >> >> > duration = duration / 1000. #convert from ms to sec > >> >> > ramp = ramp / 1000. > >> >> > > >> >> > nSamples = int(round(duration * fs)) > >> >> > nRamp = int(round(ramp * fs)) > >> >> > nTot = nSamples + (nRamp * 2) > >> >> > > >> >> > timeAll = arange(0., nTot) / fs > >> >> > timeRamp = arange(0., nRamp) > >> >> > > >> >> > snd = zeros((nTot, 2)) > >> >> > > >> >> > if channel == "Right": > >> >> > snd[0:nRamp, 1] = amp * ((1-cos(pi * timeRamp/nRamp))/2) * > >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase) > >> >> > snd[nRamp:nRamp+nSamples, 1] = amp* sin(2*pi*frequency * > >> >> > timeAll[nRamp:nRamp+nSamples] + phase) > >> >> > snd[nRamp+nSamples:len(timeAll), 1] = amp * ((1+cos(pi * > >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency * > >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase) > >> >> > elif channel == "Left": > >> >> > snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) * > >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase) > >> >> > snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency * > >> >> > timeAll[nRamp:nRamp+nSamples] + phase) > >> >> > snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi * > >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency * > >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase) > >> >> > elif channel == "Both": > >> >> > snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) * > >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase) > >> >> > snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency * > >> >> > timeAll[nRamp:nRamp+nSamples] + phase) > >> >> > snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi * > >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency * > >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase) > >> >> > snd[:, 1] = snd[:, 0] > >> >> > > >> >> > return snd > >> >> > > >> >> > > >> >> > def complexTone(F0, lowHarm, highHarm, level, duration, ramp, > >> >> > channel, > >> >> > fs, maxLevel): > >> >> > pool = multiprocessing.Pool() > >> >> > tn = [] > >> >> > > >> >> > for i in range(lowHarm, highHarm+1): > >> >> > res = pool.apply_async(pureTone, (F0*i, 0, level, duration, > >> >> > ramp, > >> >> > channel, fs, maxLevel,), callback=tn.append) > >> >> > > >> >> > pool.close() > >> >> > pool.join() > >> >> > > >> >> > for i in range(len(tn)): > >> >> > if i == 0: > >> >> > snd = tn[i] > >> >> > else: > >> >> > snd = snd + tn[i] > >> >> > > >> >> > return snd > >> >> > > >> >> > #----------END OF FILE 2 > >> >> > _______________________________________________ > >> >> > PyQt mailing list [email protected] > >> >> > http://www.riverbankcomputing.com/mailman/listinfo/pyqt > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> Lic. José M. Rodriguez Bacallao > >> >> Centro de Biofisica Medica > >> >> ----------------------------------------------------------------- > >> >> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo > >> >> mismo. > >> >> > >> >> Recuerda: El arca de Noe fue construida por aficionados, el titanic > >> >> por profesionales > >> >> ----------------------------------------------------------------- > >> >> _______________________________________________ > >> >> PyQt mailing list [email protected] > >> >> http://www.riverbankcomputing.com/mailman/listinfo/pyqt > >> > > >> > >> > >> > >> -- > >> Lic. José M. Rodriguez Bacallao > >> Centro de Biofisica Medica > >> ----------------------------------------------------------------- > >> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo > >> mismo. > >> > >> Recuerda: El arca de Noe fue construida por aficionados, el titanic > >> por profesionales > >> ----------------------------------------------------------------- > > > > > > -- > Lic. José M. Rodriguez Bacallao > Centro de Biofisica Medica > ----------------------------------------------------------------- > Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo mismo. > > Recuerda: El arca de Noe fue construida por aficionados, el titanic > por profesionales > -----------------------------------------------------------------
_______________________________________________ PyQt mailing list [email protected] http://www.riverbankcomputing.com/mailman/listinfo/pyqt
