Re: [PyQt] pyqt4 amd multiprocessing
I am very interested in the answer because I have plan to use
multiprocessing module to do my hard work (because of the python GIL),
my future design is something like Samuele say, a pool of process
ready to do some hard job (using multiprocessing) and the GUI with
pyqt4 and vtk, data pass back and forth between the gui and the worker
processes (data will be vtk datasets)
On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno sam.carca...@gmail.com wrote:
Hi,
I would like to use the multiprocessing module to create a set of sounds (as
numpy arrays) in parallel.
The parameters of the sounds (duration, volume, etc..) are controlled by a
pyqt4 GUI, and the sounds
need to be passed back to the GUI for further processing and presentation.
The sounds are generated
in a separate module from the GUI. I'm attaching a minimal example with a
file that creates the gui and
a second file that contains the functions for generating the sounds.
I've had mixed success so far, it seems to work on Kubuntu Natty - Python
2.7.1+ - Qt 4.7.2 - PyQt 4.8.3
but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 - PyQt
4.7.2 with the following error:
Traceback (most recent call last):
File test_gui.py, line 20, in onClickButton1
self.doTrial()
File test_gui.py, line 32, in doTrial
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp, channel, fs,
maxLevel)
File /media/ntfsShared/mptest/simple_test/stim.py, line 45, in complexTone
pool.join()
File /usr/lib/python2.6/multiprocessing/pool.py, line 342, in join
p.join()
File /usr/lib/python2.6/multiprocessing/process.py, line 119, in join
res = self._popen.wait(timeout)
File /usr/lib/python2.6/multiprocessing/forking.py, line 117, in wait
return self.poll(0)
File /usr/lib/python2.6/multiprocessing/forking.py, line 106, in poll
pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call
Am I doing something fundamentally unsafe by using multiprocessing in this
way with a pyqt4 GUI?
What is the best approach to use multiprocessing together with pyqt4?
Thanks for any suggestions!
Sam
###-GUI--File 1
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
from stim import* #import module for stimulus generation
class Example(QtGui.QWidget):
def __init__(self):
super(Example, self).__init__()
self.setGeometry(300, 300, 250, 150)
self.setWindowTitle('Multiprocessing Test')
self.button1 = QtGui.QPushButton('Button 1', self)
QtCore.QObject.connect(self.button1,
QtCore.SIGNAL('clicked()'), self.onClickButton1)
def onClickButton1(self):
self.doTrial()
def doTrial(self):
F0 = 200
lowHarm=1
highHarm = 20
level = 50
duration = 180
ramp = 10
channel = Both
fs = 44100
maxLevel = 100.0
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp, channel,
fs, maxLevel)
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
ex = Example()
ex.show()
app.exec_()
#--END OF FILE 1
###-SOUND GENERATION-- File stim.py
from numpy import*
import multiprocessing
def pureTone(frequency, phase, level, duration, ramp, channel, fs, maxLevel):
amp = 10**((level - maxLevel) / 20.)
duration = duration / 1000. #convert from ms to sec
ramp = ramp / 1000.
nSamples = int(round(duration * fs))
nRamp = int(round(ramp * fs))
nTot = nSamples + (nRamp * 2)
timeAll = arange(0., nTot) / fs
timeRamp = arange(0., nRamp)
snd = zeros((nTot, 2))
if channel == Right:
snd[0:nRamp, 1] = amp * ((1-cos(pi * timeRamp/nRamp))/2) *
sin(2*pi*frequency * timeAll[0:nRamp] + phase)
snd[nRamp:nRamp+nSamples, 1] = amp* sin(2*pi*frequency *
timeAll[nRamp:nRamp+nSamples] + phase)
snd[nRamp+nSamples:len(timeAll), 1] = amp * ((1+cos(pi *
timeRamp/nRamp))/2) * sin(2*pi*frequency *
timeAll[nRamp+nSamples:len(timeAll)] + phase)
elif channel == Left:
snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) *
sin(2*pi*frequency * timeAll[0:nRamp] + phase)
snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency *
timeAll[nRamp:nRamp+nSamples] + phase)
snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi *
timeRamp/nRamp))/2) * sin(2*pi*frequency *
timeAll[nRamp+nSamples:len(timeAll)] + phase)
elif channel == Both:
snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2) *
sin(2*pi*frequency * timeAll[0:nRamp] + phase)
snd[nRamp:nRamp+nSamples, 0] = amp* sin(2*pi*frequency *
timeAll[nRamp:nRamp+nSamples] + phase)
snd[nRamp+nSamples:len(timeAll), 0] = amp * ((1+cos(pi *
timeRamp/nRamp))/2) * sin(2*pi*frequency *
timeAll[nRamp+nSamples:len(timeAll)] + phase)
snd[:, 1] = snd[:, 0]
return snd
def complexTone(F0, lowHarm, highHarm, level,
Re: [PyQt] pyqt4 amd multiprocessing
As an alternative to python multiprocessing module, I'm using QProcess for
loading and exporting files with my own pool manager. It's pretty darn fast and
works well, my data sets are managed with HDF5 which takes care of handling
huge data sets pretty easily. If you want to pass data back to the UI you can
use QSharedMemory to give the GUI access to your data.
Check out my project for a QProcess example.
http://code.google.com/p/ice-cache-explorer/
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:10:20 -0400
From: jmr...@gmail.com
To: sam.carca...@gmail.com
CC: pyqt@riverbankcomputing.com
Subject: Re: [PyQt] pyqt4 amd multiprocessing
I am very interested in the answer because I have plan to use
multiprocessing module to do my hard work (because of the python GIL),
my future design is something like Samuele say, a pool of process
ready to do some hard job (using multiprocessing) and the GUI with
pyqt4 and vtk, data pass back and forth between the gui and the worker
processes (data will be vtk datasets)
On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno sam.carca...@gmail.com
wrote:
Hi,
I would like to use the multiprocessing module to create a set of sounds
(as numpy arrays) in parallel.
The parameters of the sounds (duration, volume, etc..) are controlled by a
pyqt4 GUI, and the sounds
need to be passed back to the GUI for further processing and presentation.
The sounds are generated
in a separate module from the GUI. I'm attaching a minimal example with a
file that creates the gui and
a second file that contains the functions for generating the sounds.
I've had mixed success so far, it seems to work on Kubuntu Natty - Python
2.7.1+ - Qt 4.7.2 - PyQt 4.8.3
but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 - PyQt
4.7.2 with the following error:
Traceback (most recent call last):
File test_gui.py, line 20, in onClickButton1
self.doTrial()
File test_gui.py, line 32, in doTrial
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp, channel,
fs, maxLevel)
File /media/ntfsShared/mptest/simple_test/stim.py, line 45, in
complexTone
pool.join()
File /usr/lib/python2.6/multiprocessing/pool.py, line 342, in join
p.join()
File /usr/lib/python2.6/multiprocessing/process.py, line 119, in join
res = self._popen.wait(timeout)
File /usr/lib/python2.6/multiprocessing/forking.py, line 117, in wait
return self.poll(0)
File /usr/lib/python2.6/multiprocessing/forking.py, line 106, in poll
pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call
Am I doing something fundamentally unsafe by using multiprocessing in this
way with a pyqt4 GUI?
What is the best approach to use multiprocessing together with pyqt4?
Thanks for any suggestions!
Sam
###-GUI--File 1
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
from stim import* #import module for stimulus generation
class Example(QtGui.QWidget):
def __init__(self):
super(Example, self).__init__()
self.setGeometry(300, 300, 250, 150)
self.setWindowTitle('Multiprocessing Test')
self.button1 = QtGui.QPushButton('Button 1', self)
QtCore.QObject.connect(self.button1,
QtCore.SIGNAL('clicked()'),
self.onClickButton1)
def onClickButton1(self):
self.doTrial()
def doTrial(self):
F0 = 200
lowHarm=1
highHarm = 20
level = 50
duration = 180
ramp = 10
channel = Both
fs = 44100
maxLevel = 100.0
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,
channel, fs, maxLevel)
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
ex = Example()
ex.show()
app.exec_()
#--END OF FILE 1
###-SOUND GENERATION-- File stim.py
from numpy import*
import multiprocessing
def pureTone(frequency, phase, level, duration, ramp, channel, fs,
maxLevel):
amp = 10**((level - maxLevel) / 20.)
duration = duration / 1000. #convert from ms to sec
ramp = ramp / 1000.
nSamples = int(round(duration * fs))
nRamp = int(round(ramp * fs))
nTot = nSamples + (nRamp * 2)
timeAll = arange(0., nTot) / fs
timeRamp = arange(0., nRamp)
snd = zeros((nTot, 2))
if channel == Right:
snd[0:nRamp, 1] = amp * ((1-cos(pi * timeRamp/nRamp))/2) *
sin(2*pi*frequency * timeAll[0:nRamp] + phase)
snd[nRamp:nRamp+nSamples, 1] = amp* sin(2*pi*frequency *
timeAll[nRamp:nRamp+nSamples] + phase)
snd[nRamp+nSamples:len(timeAll), 1] = amp * ((1+cos(pi *
timeRamp/nRamp))/2) * sin(2*pi*frequency *
timeAll[nRamp+nSamples:len(timeAll)] + phase)
elif channel == Left:
snd[0:nRamp, 0] = amp * ((1-cos(pi * timeRamp/nRamp))/2
Re: [PyQt] pyqt4 amd multiprocessing
sound interesting but, how do you use QProcess, what I have read about
it is that this class is for executing external applications (yes,
processes), how to use QProcess in a pool. Could you tell me more
about HDF5, for example, how do you use it with python. My dataset
could be very big (or what I think is big), for example, a serie of
1500 dicom images so I think that HDF5 could be useful to me.
On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André mab...@hotmail.com wrote:
As an alternative to python multiprocessing module, I'm using QProcess for
loading and exporting files with my own pool manager. It's pretty darn fast
and works well, my data sets are managed with HDF5 which takes care of
handling huge data sets pretty easily. If you want to pass data back to the
UI you can use QSharedMemory to give the GUI access to your data.
Check out my project for a QProcess example.
http://code.google.com/p/ice-cache-explorer/
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:10:20 -0400
From: jmr...@gmail.com
To: sam.carca...@gmail.com
CC: pyqt@riverbankcomputing.com
Subject: Re: [PyQt] pyqt4 amd multiprocessing
I am very interested in the answer because I have plan to use
multiprocessing module to do my hard work (because of the python GIL),
my future design is something like Samuele say, a pool of process
ready to do some hard job (using multiprocessing) and the GUI with
pyqt4 and vtk, data pass back and forth between the gui and the worker
processes (data will be vtk datasets)
On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno sam.carca...@gmail.com
wrote:
Hi,
I would like to use the multiprocessing module to create a set of sounds
(as numpy arrays) in parallel.
The parameters of the sounds (duration, volume, etc..) are controlled by
a pyqt4 GUI, and the sounds
need to be passed back to the GUI for further processing and
presentation. The sounds are generated
in a separate module from the GUI. I'm attaching a minimal example with
a file that creates the gui and
a second file that contains the functions for generating the sounds.
I've had mixed success so far, it seems to work on Kubuntu Natty -
Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3
but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 -
PyQt 4.7.2 with the following error:
Traceback (most recent call last):
File test_gui.py, line 20, in onClickButton1
self.doTrial()
File test_gui.py, line 32, in doTrial
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,
channel, fs, maxLevel)
File /media/ntfsShared/mptest/simple_test/stim.py, line 45, in
complexTone
pool.join()
File /usr/lib/python2.6/multiprocessing/pool.py, line 342, in join
p.join()
File /usr/lib/python2.6/multiprocessing/process.py, line 119, in join
res = self._popen.wait(timeout)
File /usr/lib/python2.6/multiprocessing/forking.py, line 117, in wait
return self.poll(0)
File /usr/lib/python2.6/multiprocessing/forking.py, line 106, in poll
pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call
Am I doing something fundamentally unsafe by using multiprocessing in
this way with a pyqt4 GUI?
What is the best approach to use multiprocessing together with pyqt4?
Thanks for any suggestions!
Sam
###-GUI--File 1
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
from stim import* #import module for stimulus generation
class Example(QtGui.QWidget):
def __init__(self):
super(Example, self).__init__()
self.setGeometry(300, 300, 250, 150)
self.setWindowTitle('Multiprocessing Test')
self.button1 = QtGui.QPushButton('Button 1', self)
QtCore.QObject.connect(self.button1,
QtCore.SIGNAL('clicked()'),
self.onClickButton1)
def onClickButton1(self):
self.doTrial()
def doTrial(self):
F0 = 200
lowHarm=1
highHarm = 20
level = 50
duration = 180
ramp = 10
channel = Both
fs = 44100
maxLevel = 100.0
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,
channel, fs, maxLevel)
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
ex = Example()
ex.show()
app.exec_()
#--END OF FILE 1
###-SOUND GENERATION-- File stim.py
from numpy import*
import multiprocessing
def pureTone(frequency, phase, level, duration, ramp, channel, fs,
maxLevel):
amp = 10**((level - maxLevel) / 20.)
duration = duration / 1000. #convert from ms to sec
ramp = ramp / 1000.
nSamples = int(round(duration * fs))
nRamp = int(round(ramp * fs))
nTot = nSamples + (nRamp * 2)
timeAll = arange(0., nTot) / fs
timeRamp = arange(0., nRamp)
snd = zeros((nTot, 2))
if channel == Right:
snd
Re: [PyQt] pyqt4 amd multiprocessing
Indeed, QProcess is meant to run external python scripts, which allows python
to run one script per available core.
e.g.p = QProcess('myscript.py data' )p.start()
Look at the following files for a more an in-depth example:
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/process_pool.pyhttp://code.google.com/p/ice-cache-explorer/source/browse/trunk/loader_process.pyhttp://code.google.com/p/ice-cache-explorer/source/browse/trunk/iceloader.py
I'm using the H5py module for dealing with HDF5 files, it has nice support for
numpy and easy to use. You can also use pytable which is also a popular choice.
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:46:43 -0400
Subject: Re: [PyQt] pyqt4 amd multiprocessing
From: jmr...@gmail.com
To: mab...@hotmail.com
CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com
sound interesting but, how do you use QProcess, what I have read about
it is that this class is for executing external applications (yes,
processes), how to use QProcess in a pool. Could you tell me more
about HDF5, for example, how do you use it with python. My dataset
could be very big (or what I think is big), for example, a serie of
1500 dicom images so I think that HDF5 could be useful to me.
On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André mab...@hotmail.com wrote:
As an alternative to python multiprocessing module, I'm using QProcess for
loading and exporting files with my own pool manager. It's pretty darn fast
and works well, my data sets are managed with HDF5 which takes care of
handling huge data sets pretty easily. If you want to pass data back to the
UI you can use QSharedMemory to give the GUI access to your data.
Check out my project for a QProcess example.
http://code.google.com/p/ice-cache-explorer/
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:10:20 -0400
From: jmr...@gmail.com
To: sam.carca...@gmail.com
CC: pyqt@riverbankcomputing.com
Subject: Re: [PyQt] pyqt4 amd multiprocessing
I am very interested in the answer because I have plan to use
multiprocessing module to do my hard work (because of the python GIL),
my future design is something like Samuele say, a pool of process
ready to do some hard job (using multiprocessing) and the GUI with
pyqt4 and vtk, data pass back and forth between the gui and the worker
processes (data will be vtk datasets)
On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno sam.carca...@gmail.com
wrote:
Hi,
I would like to use the multiprocessing module to create a set of sounds
(as numpy arrays) in parallel.
The parameters of the sounds (duration, volume, etc..) are controlled by
a pyqt4 GUI, and the sounds
need to be passed back to the GUI for further processing and
presentation. The sounds are generated
in a separate module from the GUI. I'm attaching a minimal example with
a file that creates the gui and
a second file that contains the functions for generating the sounds.
I've had mixed success so far, it seems to work on Kubuntu Natty -
Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3
but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 -
PyQt 4.7.2 with the following error:
Traceback (most recent call last):
File test_gui.py, line 20, in onClickButton1
self.doTrial()
File test_gui.py, line 32, in doTrial
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,
channel, fs, maxLevel)
File /media/ntfsShared/mptest/simple_test/stim.py, line 45, in
complexTone
pool.join()
File /usr/lib/python2.6/multiprocessing/pool.py, line 342, in join
p.join()
File /usr/lib/python2.6/multiprocessing/process.py, line 119, in join
res = self._popen.wait(timeout)
File /usr/lib/python2.6/multiprocessing/forking.py, line 117, in wait
return self.poll(0)
File /usr/lib/python2.6/multiprocessing/forking.py, line 106, in poll
pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call
Am I doing something fundamentally unsafe by using multiprocessing in
this way with a pyqt4 GUI?
What is the best approach to use multiprocessing together with pyqt4?
Thanks for any suggestions!
Sam
###-GUI--File 1
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
from stim import* #import module for stimulus generation
class Example(QtGui.QWidget):
def __init__(self):
super(Example, self).__init__()
self.setGeometry(300, 300, 250, 150)
self.setWindowTitle('Multiprocessing Test')
self.button1 = QtGui.QPushButton('Button 1', self)
QtCore.QObject.connect(self.button1,
QtCore.SIGNAL('clicked()'),
self.onClickButton1)
def onClickButton1(self):
self.doTrial()
def doTrial(self):
F0 = 200
Re: [PyQt] pyqt4 amd multiprocessing
a dummy question, is possible, simple and fast to send a big array of
integer, floats, etc, lets say, a 512x512x1500 array of unsigned int
using hdf5 from one process to another?
On Mon, May 9, 2011 at 10:16 AM, Belzile Marc-André mab...@hotmail.com wrote:
Indeed, QProcess is meant to run external python scripts, which allows
python to run one script per available core.
e.g.
p = QProcess('myscript.py data' )
p.start()
Look at the following files for a more an in-depth example:
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/process_pool.py
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/loader_process.py
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/iceloader.py
I'm using the H5py module for dealing with HDF5 files, it has nice support
for numpy and easy to use. You can also use pytable which is also a popular
choice.
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:46:43 -0400
Subject: Re: [PyQt] pyqt4 amd multiprocessing
From: jmr...@gmail.com
To: mab...@hotmail.com
CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com
sound interesting but, how do you use QProcess, what I have read about
it is that this class is for executing external applications (yes,
processes), how to use QProcess in a pool. Could you tell me more
about HDF5, for example, how do you use it with python. My dataset
could be very big (or what I think is big), for example, a serie of
1500 dicom images so I think that HDF5 could be useful to me.
On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André mab...@hotmail.com
wrote:
As an alternative to python multiprocessing module, I'm using QProcess
for
loading and exporting files with my own pool manager. It's pretty darn
fast
and works well, my data sets are managed with HDF5 which takes care of
handling huge data sets pretty easily. If you want to pass data back to
the
UI you can use QSharedMemory to give the GUI access to your data.
Check out my project for a QProcess example.
http://code.google.com/p/ice-cache-explorer/
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:10:20 -0400
From: jmr...@gmail.com
To: sam.carca...@gmail.com
CC: pyqt@riverbankcomputing.com
Subject: Re: [PyQt] pyqt4 amd multiprocessing
I am very interested in the answer because I have plan to use
multiprocessing module to do my hard work (because of the python GIL),
my future design is something like Samuele say, a pool of process
ready to do some hard job (using multiprocessing) and the GUI with
pyqt4 and vtk, data pass back and forth between the gui and the worker
processes (data will be vtk datasets)
On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno
sam.carca...@gmail.com
wrote:
Hi,
I would like to use the multiprocessing module to create a set of
sounds
(as numpy arrays) in parallel.
The parameters of the sounds (duration, volume, etc..) are controlled
by
a pyqt4 GUI, and the sounds
need to be passed back to the GUI for further processing and
presentation. The sounds are generated
in a separate module from the GUI. I'm attaching a minimal example
with
a file that creates the gui and
a second file that contains the functions for generating the sounds.
I've had mixed success so far, it seems to work on Kubuntu Natty -
Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3
but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 -
PyQt 4.7.2 with the following error:
Traceback (most recent call last):
File test_gui.py, line 20, in onClickButton1
self.doTrial()
File test_gui.py, line 32, in doTrial
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,
channel, fs, maxLevel)
File /media/ntfsShared/mptest/simple_test/stim.py, line 45, in
complexTone
pool.join()
File /usr/lib/python2.6/multiprocessing/pool.py, line 342, in join
p.join()
File /usr/lib/python2.6/multiprocessing/process.py, line 119, in
join
res = self._popen.wait(timeout)
File /usr/lib/python2.6/multiprocessing/forking.py, line 117, in
wait
return self.poll(0)
File /usr/lib/python2.6/multiprocessing/forking.py, line 106, in
poll
pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call
Am I doing something fundamentally unsafe by using multiprocessing in
this way with a pyqt4 GUI?
What is the best approach to use multiprocessing together with pyqt4?
Thanks for any suggestions!
Sam
###-GUI--File 1
from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
from stim import* #import module for stimulus generation
class Example(QtGui.QWidget):
def __init__(self):
super(Example, self).__init__()
self.setGeometry(300, 300, 250, 150)
self.setWindowTitle('Multiprocessing Test
Re: [PyQt] pyqt4 amd multiprocessing
I have never tried it but you can save a file in process A and open it from
another process, that's what I do. The H5py or pytables forum/list should be
able to give you a better answer.
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 10:26:23 -0400
Subject: Re: [PyQt] pyqt4 amd multiprocessing
From: jmr...@gmail.com
To: mab...@hotmail.com
CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com
a dummy question, is possible, simple and fast to send a big array of
integer, floats, etc, lets say, a 512x512x1500 array of unsigned int
using hdf5 from one process to another?
On Mon, May 9, 2011 at 10:16 AM, Belzile Marc-André mab...@hotmail.com
wrote:
Indeed, QProcess is meant to run external python scripts, which allows
python to run one script per available core.
e.g.
p = QProcess('myscript.py data' )
p.start()
Look at the following files for a more an in-depth example:
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/process_pool.py
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/loader_process.py
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/iceloader.py
I'm using the H5py module for dealing with HDF5 files, it has nice support
for numpy and easy to use. You can also use pytable which is also a popular
choice.
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:46:43 -0400
Subject: Re: [PyQt] pyqt4 amd multiprocessing
From: jmr...@gmail.com
To: mab...@hotmail.com
CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com
sound interesting but, how do you use QProcess, what I have read about
it is that this class is for executing external applications (yes,
processes), how to use QProcess in a pool. Could you tell me more
about HDF5, for example, how do you use it with python. My dataset
could be very big (or what I think is big), for example, a serie of
1500 dicom images so I think that HDF5 could be useful to me.
On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André mab...@hotmail.com
wrote:
As an alternative to python multiprocessing module, I'm using QProcess
for
loading and exporting files with my own pool manager. It's pretty darn
fast
and works well, my data sets are managed with HDF5 which takes care of
handling huge data sets pretty easily. If you want to pass data back to
the
UI you can use QSharedMemory to give the GUI access to your data.
Check out my project for a QProcess example.
http://code.google.com/p/ice-cache-explorer/
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:10:20 -0400
From: jmr...@gmail.com
To: sam.carca...@gmail.com
CC: pyqt@riverbankcomputing.com
Subject: Re: [PyQt] pyqt4 amd multiprocessing
I am very interested in the answer because I have plan to use
multiprocessing module to do my hard work (because of the python GIL),
my future design is something like Samuele say, a pool of process
ready to do some hard job (using multiprocessing) and the GUI with
pyqt4 and vtk, data pass back and forth between the gui and the worker
processes (data will be vtk datasets)
On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno
sam.carca...@gmail.com
wrote:
Hi,
I would like to use the multiprocessing module to create a set of
sounds
(as numpy arrays) in parallel.
The parameters of the sounds (duration, volume, etc..) are controlled
by
a pyqt4 GUI, and the sounds
need to be passed back to the GUI for further processing and
presentation. The sounds are generated
in a separate module from the GUI. I'm attaching a minimal example
with
a file that creates the gui and
a second file that contains the functions for generating the sounds.
I've had mixed success so far, it seems to work on Kubuntu Natty -
Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3
but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2 -
PyQt 4.7.2 with the following error:
Traceback (most recent call last):
File test_gui.py, line 20, in onClickButton1
self.doTrial()
File test_gui.py, line 32, in doTrial
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,
channel, fs, maxLevel)
File /media/ntfsShared/mptest/simple_test/stim.py, line 45, in
complexTone
pool.join()
File /usr/lib/python2.6/multiprocessing/pool.py, line 342, in join
p.join()
File /usr/lib/python2.6/multiprocessing/process.py, line 119, in
join
res = self._popen.wait(timeout)
File /usr/lib/python2.6/multiprocessing/forking.py, line 117, in
wait
return self.poll(0)
File /usr/lib/python2.6/multiprocessing/forking.py, line 106, in
poll
pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call
Am I doing something fundamentally unsafe
Re: [PyQt] pyqt4 amd multiprocessing
my problem is that I am using vtk and gdcm for loading files but the
python GIL is in the F$@#$... middle and my GUI is freezing because
vtk and gdcm doesn't release the GIL (a patch exist but is for vtk
5.4.2 and I am using the git version) so I decided to load the dicom
files in another process and send the data just read to the main
process (the GUI) for visualization.
PS: by the way, your project looks interesting, how is the memory
consumption and the performance of visualization. Why you didn't use
vtk for visualization?
On Mon, May 9, 2011 at 10:43 AM, Belzile Marc-André mab...@hotmail.com wrote:
I have never tried it but you can save a file in process A and open it from
another process, that's what I do. The H5py or pytables forum/list should be
able to give you a better answer.
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 10:26:23 -0400
Subject: Re: [PyQt] pyqt4 amd multiprocessing
From: jmr...@gmail.com
To: mab...@hotmail.com
CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com
a dummy question, is possible, simple and fast to send a big array of
integer, floats, etc, lets say, a 512x512x1500 array of unsigned int
using hdf5 from one process to another?
On Mon, May 9, 2011 at 10:16 AM, Belzile Marc-André mab...@hotmail.com
wrote:
Indeed, QProcess is meant to run external python scripts, which allows
python to run one script per available core.
e.g.
p = QProcess('myscript.py data' )
p.start()
Look at the following files for a more an in-depth example:
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/process_pool.py
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/loader_process.py
http://code.google.com/p/ice-cache-explorer/source/browse/trunk/iceloader.py
I'm using the H5py module for dealing with HDF5 files, it has nice
support
for numpy and easy to use. You can also use pytable which is also a
popular
choice.
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:46:43 -0400
Subject: Re: [PyQt] pyqt4 amd multiprocessing
From: jmr...@gmail.com
To: mab...@hotmail.com
CC: sam.carca...@gmail.com; pyqt@riverbankcomputing.com
sound interesting but, how do you use QProcess, what I have read about
it is that this class is for executing external applications (yes,
processes), how to use QProcess in a pool. Could you tell me more
about HDF5, for example, how do you use it with python. My dataset
could be very big (or what I think is big), for example, a serie of
1500 dicom images so I think that HDF5 could be useful to me.
On Mon, May 9, 2011 at 9:24 AM, Belzile Marc-André mab...@hotmail.com
wrote:
As an alternative to python multiprocessing module, I'm using
QProcess
for
loading and exporting files with my own pool manager. It's pretty
darn
fast
and works well, my data sets are managed with HDF5 which takes care
of
handling huge data sets pretty easily. If you want to pass data back
to
the
UI you can use QSharedMemory to give the GUI access to your data.
Check out my project for a QProcess example.
http://code.google.com/p/ice-cache-explorer/
-mab
The world goin' one way, people another! - Poot
Date: Mon, 9 May 2011 09:10:20 -0400
From: jmr...@gmail.com
To: sam.carca...@gmail.com
CC: pyqt@riverbankcomputing.com
Subject: Re: [PyQt] pyqt4 amd multiprocessing
I am very interested in the answer because I have plan to use
multiprocessing module to do my hard work (because of the python
GIL),
my future design is something like Samuele say, a pool of process
ready to do some hard job (using multiprocessing) and the GUI with
pyqt4 and vtk, data pass back and forth between the gui and the
worker
processes (data will be vtk datasets)
On Mon, May 9, 2011 at 6:26 AM, Samuele Carcagno
sam.carca...@gmail.com
wrote:
Hi,
I would like to use the multiprocessing module to create a set of
sounds
(as numpy arrays) in parallel.
The parameters of the sounds (duration, volume, etc..) are
controlled
by
a pyqt4 GUI, and the sounds
need to be passed back to the GUI for further processing and
presentation. The sounds are generated
in a separate module from the GUI. I'm attaching a minimal example
with
a file that creates the gui and
a second file that contains the functions for generating the
sounds.
I've had mixed success so far, it seems to work on Kubuntu Natty -
Python 2.7.1+ - Qt 4.7.2 - PyQt 4.8.3
but sometimes it fails on Kubuntu Lucid - Python 2.6.5 - Qt 4.6.2
-
PyQt 4.7.2 with the following error:
Traceback (most recent call last):
File test_gui.py, line 20, in onClickButton1
self.doTrial()
File test_gui.py, line 32, in doTrial
x = complexTone(F0, lowHarm, highHarm, level, duration, ramp,
channel, fs
