[Numpy-discussion] Simple way to launch python processes?
I would like to launch python modules or functions (I don't know which is easier to do, modules or functions) in separate Terminal windows so I can see the output from each as they execute. I need to be able to pass each module or function a set of parameters. I would like to do this from a python script already running in a Terminal window. In other words, I'd start up a master script and it would launch, say, three processes using another module or a function with different parameter values for each launch and each would run independently in its own Terminal window so stdout from each process would go to it's own respective window. When the process terminated the window would remain open. I've begun to look at subprocess modules, etc., but that's pretty confusing. I can do what I say above manually, but it's gotten clumsy as I want to run eventually in 12 cores. I have a Mac Pro running Mac OS X 10.6. If there is a better forum to ask this question, please let me know. Thanks for any advice. -- Lou Pecora, my views are my own.___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Simple way to launch python processes?
Maybe try stackoverflow, since this isn't really a numpy question. To run a command like python myscript.py arg1 arg2 in a separate process, you can do: p = subprocess.Popen(python myscript.py arg1 arg2.split()) You can launch many of these, and if you want to know if a process p is over, you can call p.poll(). I'm sure there are other (and better) options though. -=- Olivier 2011/12/7 Lou Pecora lou_boog2...@yahoo.com I would like to launch python modules or functions (I don't know which is easier to do, modules or functions) in separate Terminal windows so I can see the output from each as they execute. I need to be able to pass each module or function a set of parameters. I would like to do this from a python script already running in a Terminal window. In other words, I'd start up a master script and it would launch, say, three processes using another module or a function with different parameter values for each launch and each would run independently in its own Terminal window so stdout from each process would go to it's own respective window. When the process terminated the window would remain open. I've begun to look at subprocess modules, etc., but that's pretty confusing. I can do what I say above manually, but it's gotten clumsy as I want to run eventually in 12 cores. I have a Mac Pro running Mac OS X 10.6. If there is a better forum to ask this question, please let me know. Thanks for any advice. -- Lou Pecora, my views are my own. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Simple way to launch python processes?
You should consider the powerful multiprocessing package. Have a look on this
piece of code:
import glob
import os
import multiprocessing as multi
import subprocess as sub
import time
NPROC = 4
Python = '/Library/Frameworks/EPD64.framework/Versions/Current/bin/python'
Xterm = '/usr/X11/bin/xterm '
coord = []
Size = '100x10'
XPos = 810
YPos = 170
XOffset = 0
YOffset = 0
for i in range(NPROC):
if i % 2 == 0:
coord.append(Size + '+' + str(YPos) + '+' + str(YOffset))
else:
coord.append(Size + '+' + str(XPos) + '+' + str(YOffset))
YOffset = YOffset + YPos
def CompareColourRef(Champ):
BaseChamp = os.path.basename(Champ)
NameProc = int(multi.current_process().name[-1]) - 1
print 'Processing', BaseChamp, 'on processor', NameProc+1
os.putenv('ADAM_USER', DirWrk + 'adam_' + str(NameProc+1))
Command = Xterm + '-geometry ' + '' + coord[NameProc] + ' -T Proc' +
str(NameProc+1) + ' ' + BaseChamp + ' ' + ' -e ' + Python + ' ' + DirSrc + \
'CompareColourRef.py ' + BaseChamp + ' 21 | tee ' + DirLog +
BaseChamp + '.log'
Process = sub.Popen([Command], shell=True)
Process.wait()
print BaseChamp, 'processed on processor', NameProc+1
return
pool = multi.Pool(processes=NPROC)
Champs = glob.glob(DirImg + '*/*')
results = pool.map_async(CompareColourRef, Champs)
pool.close()
while results._number_left 0:
print Waiting for, results._number_left, 'tasks to complete'
time.sleep(15)
pool.join()
print 'Process completed'
exit(0)
Cheers
Jean-Baptiste
Le 7 déc. 2011 à 15:43, Olivier Delalleau a écrit :
Maybe try stackoverflow, since this isn't really a numpy question.
To run a command like python myscript.py arg1 arg2 in a separate process,
you can do:
p = subprocess.Popen(python myscript.py arg1 arg2.split())
You can launch many of these, and if you want to know if a process p is over,
you can call p.poll().
I'm sure there are other (and better) options though.
-=- Olivier
2011/12/7 Lou Pecora lou_boog2...@yahoo.com
I would like to launch python modules or functions (I don't know which is
easier to do, modules or functions) in separate Terminal windows so I can see
the output from each as they execute. I need to be able to pass each module
or function a set of parameters. I would like to do this from a python
script already running in a Terminal window. In other words, I'd start up a
master script and it would launch, say, three processes using another
module or a function with different parameter values for each launch and each
would run independently in its own Terminal window so stdout from each
process would go to it's own respective window. When the process terminated
the window would remain open.
I've begun to look at subprocess modules, etc., but that's pretty confusing.
I can do what I say above manually, but it's gotten clumsy as I want to run
eventually in 12 cores.
I have a Mac Pro running Mac OS X 10.6.
If there is a better forum to ask this question, please let me know.
Thanks for any advice.
-- Lou Pecora, my views are my own.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Simple way to launch python processes?
From: Olivier Delalleau sh...@keba.be To: Discussion of Numerical Python numpy-discussion@scipy.org Sent: Wednesday, December 7, 2011 3:43 PM Subject: Re: [Numpy-discussion] Simple way to launch python processes? Maybe try stackoverflow, since this isn't really a numpy question. To run a command like python myscript.py arg1 arg2 in a separate process, you can do: p = subprocess.Popen(python myscript.py arg1 arg2.split()) You can launch many of these, and if you want to know if a process p is over, you can call p.poll(). I'm sure there are other (and better) options though. -=- Olivier Thank you. -- Lou Pecora, my views are my own. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Simple way to launch python processes?
From: Jean-Baptiste Marquette marqu...@iap.fr
To: Discussion of Numerical Python numpy-discussion@scipy.org
Sent: Wednesday, December 7, 2011 4:23 PM
Subject: Re: [Numpy-discussion] Simple way to launch python processes?
You should consider the powerful multiprocessing package. Have a look on this
piece of code:
importglob
importos
import multiprocessing as multi
import subprocess as sub
importtime
NPROC = 4
Python = '/Library/Frameworks/EPD64.framework/Versions/Current/bin/python'
Xterm = '/usr/X11/bin/xterm '
coord = []
Size = '100x10'
XPos = 810
YPos = 170
XOffset = 0
YOffset = 0
for i in range(NPROC):
if i % 2 == 0:
coord.append(Size + '+' + str(YPos) + '+' + str(YOffset))
else:
coord.append(Size + '+' + str(XPos) + '+' + str(YOffset))
YOffset = YOffset + YPos
def CompareColourRef(Champ):
BaseChamp = os.path.basename(Champ)
NameProc = int(multi.current_process().name[-1]) - 1
print 'Processing', BaseChamp, 'on processor', NameProc+1
os.putenv('ADAM_USER', DirWrk + 'adam_' + str(NameProc+1))
Command = Xterm + '-geometry ' + '' + coord[NameProc] + ' -T Proc' +
str(NameProc+1) + ' ' + BaseChamp + ' ' + ' -e ' + Python + ' ' + DirSrc + \
'CompareColourRef.py ' + BaseChamp + ' 21 | tee' + DirLog + BaseChamp
+ '.log'
Process = sub.Popen([Command], shell=True)
Process.wait()
print BaseChamp, 'processed on processor', NameProc+1
return
pool = multi.Pool(processes=NPROC)
Champs = glob.glob(DirImg + '*/*')
results = pool.map_async(CompareColourRef, Champs)
pool.close()
while results._number_left 0:
printWaiting for, results._number_left, 'tasks to complete'
time.sleep(15)
pool.join()
print'Process completed'
exit(0)
Cheers
Jean-Baptiste
--
Wow. I will have to digest that, but thank you.
-- Lou Pecora, my views are my own.
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
