my problem is that I am using vtk and gdcm for loading files but the python GIL is in the F$@#$... middle and my GUI is freezing because vtk and gdcm doesn't release the GIL (a patch exist but is for vtk 5.4.2 and I am using the git version) so I decided to load the dicom files in another process and send the data just read to the main process (the GUI) for visualization.
PS: by the way, your project looks interesting, how is the memory consumption and the performance of visualization. Why you didn't use vtk for visualization? On Mon, May 9, 2011 at 10:43 AM, Belzile Marc-André <mab...@hotmail.com> wrote: > I have never tried it but you can save a file in process A and open it from > another process, that's what I do. The H5py or pytables forum/list should be > able to give you a better answer. > > -mab > "The world goin' one way, people another!" - Poot > > > >> Date: Mon, 9 May 2011 10:26:23 -0400 >> Subject: Re: [PyQt] pyqt4 amd multiprocessing >> From: jmr...@gmail.com >> To: mab...@hotmail.com >> CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com >> >> a dummy question, is possible, simple and fast to send a big array of >> integer, floats, etc, lets say, a 512x512x1500 array of unsigned int >> using hdf5 from one process to another? >> >> >> >> On Mon, May 9, 2011 at 10:16 AM, Belzile Marc-André <mab...@hotmail.com> >> wrote: >> > Indeed, QProcess is meant to run external python scripts, which allows >> > python to run one script per available core. >> > e.g. >> > p = QProcess('myscript.py "data"' ) >> > p.start() >> > Look at the following files for a more an in-depth example: >> > >> > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/process_pool.py >> > >> > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/loader_process.py >> > >> > http://code.google.com/p/ice-cache-explorer/source/browse/trunk/iceloader.py >> > >> > I'm using the H5py module for dealing with HDF5 files, it has nice >> > support >> > for numpy and easy to use. You can also use pytable which is also a >> > popular >> > choice. >> > -mab >> > "The world goin' one way, people another!" - Poot >> > >> > >> > >> >> Date: Mon, 9 May 2011 09:46:43 -0400 >> >> Subject: Re: [PyQt] pyqt4 amd multiprocessing >> >> From: jmr...@gmail.com >> >> To: mab...@hotmail.com >> >> CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com >> >> >> >> sound interesting but, how do you use QProcess, what I have read about >> >> it is that this class is for executing external applications (yes, >> >> processes), how to use QProcess in a pool. Could you tell me more >> >> about HDF5, for example, how do you use it with python. My dataset >> >> could be very big (or what I think is big), for example, a serie of >> >> 1500 dicom images so I think that HDF5 could be useful to me. >> >> >> >> On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André <mab...@hotmail.com> >> >> wrote: >> >> > As an alternative to python multiprocessing module, I'm using >> >> > QProcess >> >> > for >> >> > loading and exporting files with my own pool manager. It's pretty >> >> > darn >> >> > fast >> >> > and works well, my data sets are managed with HDF5 which takes care >> >> > of >> >> > handling huge data sets pretty easily. If you want to pass data back >> >> > to >> >> > the >> >> > UI you can use QSharedMemory to give the GUI access to your data. >> >> > Check out my project for a QProcess example. >> >> > http://code.google.com/p/ice-cache-explorer/ >> >> > >> >> > -mab >> >> > "The world goin' one way, people another!" - Poot >> >> > >> >> > >> >> > >> >> >> Date: Mon, 9 May 2011 09:10:20 -0400 >> >> >> From: jmr...@gmail.com >> >> >> To: sam.carca...@gmail.com >> >> >> CC: pyqt@riverbankcomputing.com >> >> >> Subject: Re: [PyQt] pyqt4 amd multiprocessing >> >> >> >> >> >> I am very interested in the answer because I have plan to use >> >> >> multiprocessing module to do my hard work (because of the python >> >> >> GIL), >> >> >> my future design is something like Samuele say, a pool of process >> >> >> ready to do some hard job (using multiprocessing) and the GUI with >> >> >> pyqt4 and vtk, data pass back and forth between the gui and the >> >> >> worker >> >> >> processes (data will be vtk datasets) >> >> >> >> >> >> On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno >> >> >> <sam.carca...@gmail.com> >> >> >> wrote: >> >> >> > Hi, >> >> >> > >> >> >> > I would like to use the multiprocessing module to create a set of >> >> >> > sounds >> >> >> > (as numpy arrays) in parallel. >> >> >> > The parameters of the sounds (duration, volume, etc..) are >> >> >> > controlled >> >> >> > by >> >> >> > a pyqt4 GUI, and the sounds >> >> >> > need to be passed back to the GUI for further processing and >> >> >> > presentation. The sounds are generated >> >> >> > in a separate module from the GUI. I'm attaching a minimal example >> >> >> > with >> >> >> > a file that creates the gui and >> >> >> > a second file that contains the functions for generating the >> >> >> > sounds. >> >> >> > >> >> >> > I've had mixed success so far, it seems to work on Kubuntu Natty - >> >> >> > Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3 >> >> >> > but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 >> >> >> > - >> >> >> > PyQt 4.7.2 with the following error: >> >> >> > >> >> >> > Traceback (most recent call last): >> >> >> > File "test_gui.py", line 20, in onClickButton1 >> >> >> > self.doTrial() >> >> >> > File "test_gui.py", line 32, in doTrial >> >> >> > x = complexTone(F0, lowHarm, highHarm, level, duration, ramp, >> >> >> > channel, fs, maxLevel) >> >> >> > File "/media/ntfsShared/mptest/simple_test/stim.py", line 45, in >> >> >> > complexTone >> >> >> > pool.join() >> >> >> > File "/usr/lib/python2.6/multiprocessing/pool.py", line 342, in >> >> >> > join >> >> >> > p.join() >> >> >> > File "/usr/lib/python2.6/multiprocessing/process.py", line 119, >> >> >> > in >> >> >> > join >> >> >> > res = self._popen.wait(timeout) >> >> >> > File "/usr/lib/python2.6/multiprocessing/forking.py", line 117, >> >> >> > in >> >> >> > wait >> >> >> > return self.poll(0) >> >> >> > File "/usr/lib/python2.6/multiprocessing/forking.py", line 106, >> >> >> > in >> >> >> > poll >> >> >> > pid, sts = os.waitpid(self.pid, flag) >> >> >> > OSError: [Errno 4] Interrupted system call >> >> >> > >> >> >> > Am I doing something fundamentally unsafe by using multiprocessing >> >> >> > in >> >> >> > this way with a pyqt4 GUI? >> >> >> > What is the best approach to use multiprocessing together with >> >> >> > pyqt4? >> >> >> > >> >> >> > Thanks for any suggestions! >> >> >> > >> >> >> > Sam >> >> >> > >> >> >> > ###---------GUI----------File 1 >> >> >> > from PyQt4 import QtGui >> >> >> > from PyQt4 import QtCore >> >> >> > import sys >> >> >> > from stim import* #import module for stimulus generation >> >> >> > >> >> >> > >> >> >> > class Example(QtGui.QWidget): >> >> >> > >> >> >> > def __init__(self): >> >> >> > super(Example, self).__init__() >> >> >> > self.setGeometry(300, 300, 250, 150) >> >> >> > self.setWindowTitle('Multiprocessing Test') >> >> >> > >> >> >> > self.button1 = QtGui.QPushButton('Button 1', self) >> >> >> > QtCore.QObject.connect(self.button1, >> >> >> > QtCore.SIGNAL('clicked()'), >> >> >> > self.onClickButton1) >> >> >> > >> >> >> > def onClickButton1(self): >> >> >> > self.doTrial() >> >> >> > >> >> >> > def doTrial(self): >> >> >> > F0 = 200 >> >> >> > lowHarm=1 >> >> >> > highHarm = 20 >> >> >> > level = 50 >> >> >> > duration = 180 >> >> >> > ramp = 10 >> >> >> > channel = "Both" >> >> >> > fs = 44100 >> >> >> > maxLevel = 100.0 >> >> >> > x = complexTone(F0, lowHarm, highHarm, level, duration, >> >> >> > ramp, >> >> >> > channel, fs, maxLevel) >> >> >> > >> >> >> > >> >> >> > if __name__ == '__main__': >> >> >> > app = QtGui.QApplication(sys.argv) >> >> >> > ex = Example() >> >> >> > ex.show() >> >> >> > app.exec_() >> >> >> > #----------END OF FILE 1 >> >> >> > >> >> >> > >> >> >> > ###-SOUND GENERATION-- File stim.py >> >> >> > from numpy import* >> >> >> > import multiprocessing >> >> >> > >> >> >> > def pureTone(frequency, phase, level, duration, ramp, channel, fs, >> >> >> > maxLevel): >> >> >> > >> >> >> > amp = 10**((level - maxLevel) / 20.) >> >> >> > duration = duration / 1000. #convert from ms to sec >> >> >> > ramp = ramp / 1000. >> >> >> > >> >> >> > nSamples = int(round(duration * fs)) >> >> >> > nRamp = int(round(ramp * fs)) >> >> >> > nTot = nSamples + (nRamp * 2) >> >> >> > >> >> >> > timeAll = arange(0., nTot) / fs >> >> >> > timeRamp = arange(0., nRamp) >> >> >> > >> >> >> > snd = zeros((nTot, 2)) >> >> >> > >> >> >> > if channel == "Right": >> >> >> > snd[0:nRamp, 1] = amp * ((1-cos(pi * timeRamp/nRamp))/2) * >> >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase) >> >> >> > snd[nRamp:nRamp+nSamples, 1] = amp* sin(2*pi*frequency * >> >> >> > timeAll[nRamp:nRamp+nSamples] + phase) >> >> >> > snd[nRamp+nSamples:len(timeAll), 1] = amp * ((1+cos(pi * >> >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency * >> >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase) >> >> >> > elif channel == "Left": >> >> >> > snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) * >> >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase) >> >> >> > snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency * >> >> >> > timeAll[nRamp:nRamp+nSamples] + phase) >> >> >> > snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi * >> >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency * >> >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase) >> >> >> > elif channel == "Both": >> >> >> > snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) * >> >> >> > sin(2*pi*frequency * timeAll[0:nRamp] + phase) >> >> >> > snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency * >> >> >> > timeAll[nRamp:nRamp+nSamples] + phase) >> >> >> > snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi * >> >> >> > timeRamp/nRamp))/2) * sin(2*pi*frequency * >> >> >> > timeAll[nRamp+nSamples:len(timeAll)] + phase) >> >> >> > snd[:, 1] = snd[:, 0] >> >> >> > >> >> >> > return snd >> >> >> > >> >> >> > >> >> >> > def complexTone(F0, lowHarm, highHarm, level, duration, ramp, >> >> >> > channel, >> >> >> > fs, maxLevel): >> >> >> > pool = multiprocessing.Pool() >> >> >> > tn = [] >> >> >> > >> >> >> > for i in range(lowHarm, highHarm+1): >> >> >> > res = pool.apply_async(pureTone, (F0*i, 0, level, duration, >> >> >> > ramp, >> >> >> > channel, fs, maxLevel,), callback=tn.append) >> >> >> > >> >> >> > pool.close() >> >> >> > pool.join() >> >> >> > >> >> >> > for i in range(len(tn)): >> >> >> > if i == 0: >> >> >> > snd = tn[i] >> >> >> > else: >> >> >> > snd = snd + tn[i] >> >> >> > >> >> >> > return snd >> >> >> > >> >> >> > #----------END OF FILE 2 >> >> >> > _______________________________________________ >> >> >> > PyQt mailing list PyQt@riverbankcomputing.com >> >> >> > http://www.riverbankcomputing.com/mailman/listinfo/pyqt >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Lic. José M. Rodriguez Bacallao >> >> >> Centro de Biofisica Medica >> >> >> ----------------------------------------------------------------- >> >> >> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos >> >> >> lo >> >> >> mismo. >> >> >> >> >> >> Recuerda: El arca de Noe fue construida por aficionados, el titanic >> >> >> por profesionales >> >> >> ----------------------------------------------------------------- >> >> >> _______________________________________________ >> >> >> PyQt mailing list PyQt@riverbankcomputing.com >> >> >> http://www.riverbankcomputing.com/mailman/listinfo/pyqt >> >> > >> >> >> >> >> >> >> >> -- >> >> Lic. José M. Rodriguez Bacallao >> >> Centro de Biofisica Medica >> >> ----------------------------------------------------------------- >> >> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo >> >> mismo. >> >> >> >> Recuerda: El arca de Noe fue construida por aficionados, el titanic >> >> por profesionales >> >> ----------------------------------------------------------------- >> > >> >> >> >> -- >> Lic. José M. Rodriguez Bacallao >> Centro de Biofisica Medica >> ----------------------------------------------------------------- >> Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo >> mismo. >> >> Recuerda: El arca de Noe fue construida por aficionados, el titanic >> por profesionales >> ----------------------------------------------------------------- > -- Lic. José M. Rodriguez Bacallao Centro de Biofisica Medica ----------------------------------------------------------------- Todos somos muy ignorantes, lo que ocurre es que no todos ignoramos lo mismo. Recuerda: El arca de Noe fue construida por aficionados, el titanic por profesionales ----------------------------------------------------------------- _______________________________________________ PyQt mailing list PyQt@riverbankcomputing.com http://www.riverbankcomputing.com/mailman/listinfo/pyqt