Afraid I cannot replicate a problem with singleton behavior in the 1.8 series:
11:31:52 /home/common/openmpi/v1.8/orte/test/mpi$ ./hello foo bar
Hello, World, I am 0 of 1 [0 local peers]: get_cpubind: 0 bitmap 0-23
OMPI_MCA_orte_default_hostfile=/home/common/hosts
OMPI_COMMAND=./hello
OMPI_ARGV=foo bar
OMPI_NUM_APP_CTX=1
OMPI_FIRST_RANKS=0
OMPI_APP_CTX_NUM_PROCS=1
OMPI_MCA_orte_ess_num_procs=1
You can see that the OMPI_ARGV envar (which is the spot you flagged) is
correctly being set and there is no segfault. Not sure what your program may be
doing, though, so I'm not sure I've really tested your scenario.
On Sep 29, 2014, at 10:55 AM, Ralph Castain <r...@open-mpi.org> wrote:
> Okay, so regression-test.py is calling MPI_Init as a singleton, correct? Just
> trying to fully understand the scenario
>
> Singletons are certainly allowed, if that's the scenario
>
> On Sep 29, 2014, at 10:51 AM, Amos Anderson <amos.ander...@protabit.com>
> wrote:
>
>> I'm not calling mpirun in this case because this particular calculation
>> doesn't use more than one processor. What I'm doing on my command line is
>> this:
>>
>> /home/user/myapp/tools/python/bin/python test/regression/regression-test.py
>> test/regression/regression-jobs
>>
>> and internally I check for rank/size. This command is executed in the
>> context of a souped up LD_LIBRARY_PATH. You can see the variable argv in
>> opal_argv_join is ending up with the last argument on my command line.
>>
>> I suppose your question implies that mpirun is mandatory for executing
>> anything compiled with OpenMPI > 1.6 ?
>>
>>
>>
>> On Sep 29, 2014, at 10:28 AM, Ralph Castain <r...@open-mpi.org> wrote:
>>
>>> Can you pass us the actual mpirun command line being executed? Especially
>>> need to see the argv being passed to your application.
>>>
>>>
>>> On Sep 27, 2014, at 7:09 PM, Amos Anderson <amos.ander...@protabit.com>
>>> wrote:
>>>
>>>> FWIW, I've confirmed that the segfault also happens with OpenMPI 1.7.5.
>>>> Also, I have some gdb output (from 1.7.5) for your perusal, including a
>>>> printout of some of the variables' values.
>>>>
>>>>
>>>>
>>>> Starting program: /home/user/myapp/tools/python/bin/python
>>>> test/regression/regression-test.py test/regression/regression-jobs
>>>> [Thread debugging using libthread_db enabled]
>>>>
>>>> Program received signal SIGSEGV, Segmentation fault.
>>>> 0x00002aaaabc8df1e in opal_argv_join (argv=0xa39398, delimiter=32) at
>>>> argv.c:299
>>>> 299 str_len += strlen(*p) + 1;
>>>> (gdb) where
>>>> #0 0x00002aaaabc8df1e in opal_argv_join (argv=0xa39398, delimiter=32) at
>>>> argv.c:299
>>>> #1 0x00002aaaaab2ce4e in ompi_mpi_init (argc=2, argv=0xa39390,
>>>> requested=0, provided=0x7fffffffba98) at runtime/ompi_mpi_init.c:450
>>>> #2 0x00002aaaaab63e39 in PMPI_Init (argc=0x7fffffffbb8c,
>>>> argv=0x7fffffffbb80) at pinit.c:84
>>>> #3 0x00002aaab7b965d6 in boost::mpi::environment::environment
>>>> (this=0xa3a1d0, argc=@0x7fffffffbb8c, argv=@0x7fffffffbb80,
>>>> abort_on_exception=true)
>>>> at ../tools/boost/libs/mpi/src/environment.cpp:98
>>>> #4 0x00002aaabc7b311d in boost::mpi::python::mpi_init (python_argv=...,
>>>> abort_on_exception=true) at
>>>> ../tools/boost/libs/mpi/src/python/py_environment.cpp:60
>>>> #5 0x00002aaabc7b33fb in boost::mpi::python::export_environment () at
>>>> ../tools/boost/libs/mpi/src/python/py_environment.cpp:94
>>>> #6 0x00002aaabc7d5ab5 in boost::mpi::python::init_module_mpi () at
>>>> ../tools/boost/libs/mpi/src/python/module.cpp:44
>>>> #7 0x00002aaab792a2f2 in
>>>> boost::detail::function::void_function_ref_invoker0<void (*)(),
>>>> void>::invoke (function_obj_ptr=...)
>>>> at ../tools/boost/boost/function/function_template.hpp:188
>>>> #8 0x00002aaab7929e6b in boost::function0<void>::operator()
>>>> (this=0x7fffffffc110) at
>>>> ../tools/boost/boost/function/function_template.hpp:767
>>>> #9 0x00002aaab7928f11 in boost::python::handle_exception_impl (f=...) at
>>>> ../tools/boost/libs/python/src/errors.cpp:25
>>>> #10 0x00002aaab792a54f in boost::python::handle_exception<void (*)()>
>>>> (f=0x2aaabc7d5746 <boost::mpi::python::init_module_mpi()>) at
>>>> ../tools/boost/boost/python/errors.hpp:29
>>>> #11 0x00002aaab792a1d9 in boost::python::detail::(anonymous
>>>> namespace)::init_module_in_scope (m=0x2aaabc617f68,
>>>> init_function=0x2aaabc7d5746 <boost::mpi::python::init_module_mpi()>)
>>>> at ../tools/boost/libs/python/src/module.cpp:24
>>>> #12 0x00002aaab792a26c in boost::python::detail::init_module
>>>> (name=0x2aaabc7f7f4d "mpi", init_function=0x2aaabc7d5746
>>>> <boost::mpi::python::init_module_mpi()>)
>>>> at ../tools/boost/libs/python/src/module.cpp:59
>>>> #13 0x00002aaabc7d5b2b in boost::mpi::python::initmpi () at
>>>> ../tools/boost/libs/mpi/src/python/module.cpp:34
>>>> #14 0x00002aaaab27e095 in _PyImport_LoadDynamicModule (name=0xac9435
>>>> "mpi", pathname=0xb08c60 "/home/user/myapp/lib/mpi.so", fp=0xaca450) at
>>>> ./Python/importdl.c:53
>>>> #15 0x00002aaaab279fd4 in load_module (name=0xac9435 "mpi", fp=0xaca450,
>>>> pathname=0xb08c60 "/home/user/myapp/lib/mpi.so", type=3, loader=0x0) at
>>>> Python/import.c:1915
>>>> #16 0x00002aaaab27c2e8 in import_submodule (mod=0x2aaaab533a20,
>>>> subname=0xac9435 "mpi", fullname=0xac9435 "mpi") at Python/import.c:2700
>>>> #17 0x00002aaaab27b8fa in load_next (mod=0x2aaab0f075a8,
>>>> altmod=0x2aaaab533a20, p_name=0x7fffffffc3f8, buf=0xac9430 "util.mpi",
>>>> p_buflen=0x7fffffffc408)
>>>> at Python/import.c:2519
>>>> #18 0x00002aaaab27a98d in import_module_level (name=0x0, globals=0xe95a70,
>>>> locals=0xe95a70, fromlist=0x2aaaab533a20, level=-1) at Python/import.c:2224
>>>> #19 0x00002aaaab27aeda in PyImport_ImportModuleLevel (name=0x2aaab0f00964
>>>> "mpi", globals=0xe95a70, locals=0xe95a70, fromlist=0x2aaaab533a20,
>>>> level=-1) at Python/import.c:2288
>>>> #20 0x00002aaaab2419c4 in builtin___import__ (self=0x0,
>>>> args=0x2aaabc6211f8, kwds=0x0) at Python/bltinmodule.c:49
>>>> #21 0x00002aaaab1b19c7 in PyCFunction_Call (func=0x2aaaabf85510,
>>>> arg=0x2aaabc6211f8, kw=0x0) at Objects/methodobject.c:85
>>>> #22 0x00002aaaab14d673 in PyObject_Call (func=0x2aaaabf85510,
>>>> arg=0x2aaabc6211f8, kw=0x0) at Objects/abstract.c:2529
>>>> #23 0x00002aaaab25ad03 in PyEval_CallObjectWithKeywords
>>>> (func=0x2aaaabf85510, arg=0x2aaabc6211f8, kw=0x0) at Python/ceval.c:3890
>>>> #24 0x00002aaaab2543e5 in PyEval_EvalFrameEx (f=0xe8aef0, throwflag=0) at
>>>> Python/ceval.c:2333
>>>> #25 0x00002aaaab258b7e in PyEval_EvalCodeEx (co=0x2aaabc61ce00,
>>>> globals=0xe95a70, locals=0xe95a70, args=0x0, argcount=0, kws=0x0,
>>>> kwcount=0, defs=0x0, defcount=0,
>>>> closure=0x0) at Python/ceval.c:3253
>>>> #26 0x00002aaaab24b5ce in PyEval_EvalCode (co=0x2aaabc61ce00,
>>>> globals=0xe95a70, locals=0xe95a70) at Python/ceval.c:667
>>>> #27 0x00002aaaab2779e2 in PyImport_ExecCodeModuleEx (name=0xaa9080
>>>> "util.myappMPI", co=0x2aaabc61ce00, pathname=0xe7d380
>>>> "/home/user/myapp/src/util/myappMPI.pyc")
>>>> at Python/import.c:709
>>>> #28 0x00002aaaab278629 in load_source_module (name=0xaa9080
>>>> "util.myappMPI", pathname=0xe7d380
>>>> "/home/user/myapp/src/util/myappMPI.pyc", fp=0x76eb00)
>>>> at Python/import.c:1099
>>>> #29 0x00002aaaab279fa0 in load_module (name=0xaa9080 "util.myappMPI",
>>>> fp=0x76eb00, pathname=0x80fe00 "/home/user/myapp/src/util/myappMPI.py",
>>>> type=1, loader=0x0)
>>>> at Python/import.c:1906
>>>> #30 0x00002aaaab27c2e8 in import_submodule (mod=0x2aaab0f075a8,
>>>> subname=0xaa9085 "myappMPI", fullname=0xaa9080 "util.myappMPI") at
>>>> Python/import.c:2700
>>>> #31 0x00002aaaab27b860 in load_next (mod=0x2aaab0f075a8,
>>>> altmod=0x2aaaab533a20, p_name=0x7fffffffcd98, buf=0xaa9080
>>>> "util.myappMPI", p_buflen=0x7fffffffcda8)
>>>> at Python/import.c:2515
>>>> #32 0x00002aaaab27a98d in import_module_level (name=0x0, globals=0x7a3c70,
>>>> locals=0x7a3c70, fromlist=0x2aaaaf6f53e0, level=-1) at Python/import.c:2224
>>>> #33 0x00002aaaab27aeda in PyImport_ImportModuleLevel (name=0x2aaaaf6e8854
>>>> "myappMPI", globals=0x7a3c70, locals=0x7a3c70, fromlist=0x2aaaaf6f53e0,
>>>> level=-1)
>>>> at Python/import.c:2288
>>>> #34 0x00002aaaab2419c4 in builtin___import__ (self=0x0,
>>>> args=0x2aaab0a9f7d0, kwds=0x0) at Python/bltinmodule.c:49
>>>> #35 0x00002aaaab1b19c7 in PyCFunction_Call (func=0x2aaaabf85510,
>>>> arg=0x2aaab0a9f7d0, kw=0x0) at Objects/methodobject.c:85
>>>> #36 0x00002aaaab14d673 in PyObject_Call (func=0x2aaaabf85510,
>>>> arg=0x2aaab0a9f7d0, kw=0x0) at Objects/abstract.c:2529
>>>> #37 0x00002aaaab25ad03 in PyEval_CallObjectWithKeywords
>>>> (func=0x2aaaabf85510, arg=0x2aaab0a9f7d0, kw=0x0) at Python/ceval.c:3890
>>>> #38 0x00002aaaab2543e5 in PyEval_EvalFrameEx (f=0x79d000, throwflag=0) at
>>>> Python/ceval.c:2333
>>>> #39 0x00002aaaab258b7e in PyEval_EvalCodeEx (co=0x2aaab0253880,
>>>> globals=0x7a3c70, locals=0x7a3c70, args=0x0, argcount=0, kws=0x0,
>>>> kwcount=0, defs=0x0, defcount=0,
>>>> closure=0x0) at Python/ceval.c:3253
>>>> #40 0x00002aaaab24b5ce in PyEval_EvalCode (co=0x2aaab0253880,
>>>> globals=0x7a3c70, locals=0x7a3c70) at Python/ceval.c:667
>>>> #41 0x00002aaaab2779e2 in PyImport_ExecCodeModuleEx (name=0x7153a0 "util",
>>>> co=0x2aaab0253880, pathname=0x7e98a0
>>>> "/home/user/myapp/src/util/__init__.pyc")
>>>> at Python/import.c:709
>>>> #42 0x00002aaaab278629 in load_source_module (name=0x7153a0 "util",
>>>> pathname=0x7e98a0 "/home/user/myapp/src/util/__init__.pyc", fp=0x6fe020)
>>>> at Python/import.c:1099
>>>> #43 0x00002aaaab279fa0 in load_module (name=0x7153a0 "util", fp=0x6fe020,
>>>> pathname=0x755e40 "/home/user/myapp/src/util/__init__.py", type=1,
>>>> loader=0x0)
>>>> at Python/import.c:1906
>>>> ---Type <return> to continue, or q <return> to quit---
>>>> #44 0x00002aaaab2788ef in load_package (name=0x7153a0 "util",
>>>> pathname=0x703390 "/home/user/myapp/src/util") at Python/import.c:1166
>>>> #45 0x00002aaaab279fea in load_module (name=0x7153a0 "util", fp=0x0,
>>>> pathname=0x703390 "/home/user/myapp/src/util", type=5, loader=0x0) at
>>>> Python/import.c:1920
>>>> #46 0x00002aaaab27c2e8 in import_submodule (mod=0x2aaaab533a20,
>>>> subname=0x7153a0 "util", fullname=0x7153a0 "util") at Python/import.c:2700
>>>> #47 0x00002aaaab27b860 in load_next (mod=0x2aaaab533a20,
>>>> altmod=0x2aaaab533a20, p_name=0x7fffffffd818, buf=0x7153a0 "util",
>>>> p_buflen=0x7fffffffd828) at Python/import.c:2515
>>>> #48 0x00002aaaab27a98d in import_module_level (name=0x2aaaac0e0f59
>>>> "myappSubmission", globals=0x6443c0, locals=0x6443c0,
>>>> fromlist=0x2aaaac0f1760, level=-1)
>>>> at Python/import.c:2224
>>>> #49 0x00002aaaab27aeda in PyImport_ImportModuleLevel (name=0x2aaaac0e0f54
>>>> "util.myappSubmission", globals=0x6443c0, locals=0x6443c0,
>>>> fromlist=0x2aaaac0f1760, level=-1)
>>>> at Python/import.c:2288
>>>> #50 0x00002aaaab2419c4 in builtin___import__ (self=0x0,
>>>> args=0x2aaaac0dbb00, kwds=0x0) at Python/bltinmodule.c:49
>>>> #51 0x00002aaaab1b19c7 in PyCFunction_Call (func=0x2aaaabf85510,
>>>> arg=0x2aaaac0dbb00, kw=0x0) at Objects/methodobject.c:85
>>>> #52 0x00002aaaab14d673 in PyObject_Call (func=0x2aaaabf85510,
>>>> arg=0x2aaaac0dbb00, kw=0x0) at Objects/abstract.c:2529
>>>> #53 0x00002aaaab25ad03 in PyEval_CallObjectWithKeywords
>>>> (func=0x2aaaabf85510, arg=0x2aaaac0dbb00, kw=0x0) at Python/ceval.c:3890
>>>> #54 0x00002aaaab2543e5 in PyEval_EvalFrameEx (f=0x71a1a0, throwflag=0) at
>>>> Python/ceval.c:2333
>>>> #55 0x00002aaaab258b7e in PyEval_EvalCodeEx (co=0x2aaaac0dd720,
>>>> globals=0x6443c0, locals=0x6443c0, args=0x0, argcount=0, kws=0x0,
>>>> kwcount=0, defs=0x0, defcount=0,
>>>> closure=0x0) at Python/ceval.c:3253
>>>> #56 0x00002aaaab24b5ce in PyEval_EvalCode (co=0x2aaaac0dd720,
>>>> globals=0x6443c0, locals=0x6443c0) at Python/ceval.c:667
>>>> #57 0x00002aaaab28d492 in run_mod (mod=0x720960, filename=0x7fffffffe6bf
>>>> "test/regression/regression-test.py", globals=0x6443c0, locals=0x6443c0,
>>>> flags=0x7fffffffe220,
>>>> arena=0x6707d0) at Python/pythonrun.c:1370
>>>> #58 0x00002aaaab28d41c in PyRun_FileExFlags (fp=0x6cb7c0,
>>>> filename=0x7fffffffe6bf "test/regression/regression-test.py", start=257,
>>>> globals=0x6443c0, locals=0x6443c0,
>>>> closeit=1, flags=0x7fffffffe220) at Python/pythonrun.c:1356
>>>> #59 0x00002aaaab28bbfe in PyRun_SimpleFileExFlags (fp=0x6cb7c0,
>>>> filename=0x7fffffffe6bf "test/regression/regression-test.py", closeit=1,
>>>> flags=0x7fffffffe220)
>>>> at Python/pythonrun.c:948
>>>> #60 0x00002aaaab28b1be in PyRun_AnyFileExFlags (fp=0x6cb7c0,
>>>> filename=0x7fffffffe6bf "test/regression/regression-test.py", closeit=1,
>>>> flags=0x7fffffffe220)
>>>> at Python/pythonrun.c:752
>>>> #61 0x00002aaaab2a7497 in Py_Main (argc=3, argv=0x7fffffffe3a8) at
>>>> Modules/main.c:640
>>>> #62 0x00000000004006f3 in main (argc=3, argv=0x7fffffffe3a8) at
>>>> ./Modules/python.c:23
>>>> (gdb) p argv
>>>> $1 = (char **) 0xa39398
>>>> (gdb) p *argv
>>>> $2 = 0xa26390 "test/regression/regression-jobs"
>>>> (gdb) p **argv
>>>> $3 = 116 't'
>>>> (gdb) p p
>>>> $4 = (char **) 0xa393a0
>>>> (gdb) p *p
>>>> $5 = 0x20 <Address 0x20 out of bounds>
>>>> (gdb) p str_len
>>>> $6 = 32
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sep 26, 2014, at 5:19 PM, Amos Anderson <amos.ander...@protabit.com>
>>>> wrote:
>>>>
>>>>> Hello all --
>>>>>
>>>>> I'm trying to get a working configuration for my application and I can
>>>>> get OpenMPI 1.6.5 to work, while OpenMPI 1.8.2 segfaults.
>>>>>
>>>>>
>>>>> Here's how I compile OpenMPI:
>>>>>
>>>>> OPENMPI = openmpi-1.8.2
>>>>> FLAGS = --enable-static
>>>>> cd $(OPENMPI) ; ./configure $(FLAGS) --with-tm=/opt/torque-2.5.9/
>>>>> --prefix=$(CURDIR)
>>>>>
>>>>>
>>>>> I'm able to compile openmpi successfully, and I use a bjam instruction
>>>>> like this to compile my program (which uses boost python boost_1_55_0):
>>>>> using mpi : ../tools/openmpi/bin/mpic++ ;
>>>>>
>>>>> and I run my program in a Torque pbs script like this:
>>>>>
>>>>> /bin/rm -rf jobname.nodes
>>>>> for i in `cat ${PBS_NODEFILE} | sort -u`
>>>>> do
>>>>> echo $i slots \= `grep $i ${PBS_NODEFILE} | wc -l` >> jobname.nodes
>>>>> done
>>>>> /home/user/myapp/tools/openmpi/bin/mpirun -np 2 -hostfile jobname.nodes
>>>>> /home/user/myapp/myapp.exe
>>>>>
>>>>>
>>>>>
>>>>> which also compiles just fine. But when I run my program I get the
>>>>> segfault I printed below. When I switch to:
>>>>> OPENMPI = openmpi-1.6.5
>>>>>
>>>>> then everything works as expected. (As a side question, do I need both
>>>>> -hostfile and --with-tm? I asked this question earlier today on this
>>>>> list). That is, I believe that I'm using the exact same setup in both
>>>>> cases, and 1.6.5 works while 1.8.2 fails. Any suggestions what I might be
>>>>> doing wrong?
>>>>>
>>>>> I suppose if I have a working setup I can give up even if it's with an
>>>>> older version... but this could be evidence of something I'll have to
>>>>> confront eventually.
>>>>>
>>>>> Thanks for any advice!
>>>>> Amos.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [local:27921] *** Process received signal ***
>>>>> [local:27921] Signal: Segmentation fault (11)
>>>>> [local:27921] Signal code: Address not mapped (1)
>>>>> [local:27921] Failing at address: 0x40
>>>>> [local:27921] [ 0] /lib64/libpthread.so.0[0x322180e4c0]
>>>>> [local:27921] [ 1] /lib64/libc.so.6(strlen+0x30)[0x3220c78d80]
>>>>> [local:27921] [ 2]
>>>>> /home/user/myapp/tools/openmpi/lib/libopen-pal.so.6(opal_argv_join+0x95)[0x2b87f5c4e175]
>>>>> [local:27921] [ 3]
>>>>> /home/user/myapp/tools/openmpi/lib/libmpi.so.1(ompi_mpi_init+0x82d)[0x2b87f3c9ec0d]
>>>>> [local:27921] [ 4]
>>>>> /home/user/myapp/tools/openmpi/lib/libmpi.so.1(MPI_Init+0xf0)[0x2b87f3cbc310]
>>>>> [local:27921] [ 5]
>>>>> /home/user/myapp/lib/libboost_mpi.so.1.55.0(_ZN5boost3mpi11environmentC1ERiRPPcb+0x36)[0x2b87f3795826]
>>>>> [local:27921] [ 6]
>>>>> /home/user/myapp/lib/mpi.so(_ZN5boost3mpi6python8mpi_initENS_6python4listEb+0x314)[0x2b87f30bc7b4]
>>>>> [local:27921] [ 7]
>>>>> /home/user/myapp/lib/mpi.so(_ZN5boost3mpi6python18export_environmentEv+0xcc6)[0x2b87f30bd5f6]
>>>>> [local:27921] [ 8]
>>>>> /home/user/myapp/lib/mpi.so(_ZN5boost3mpi6python15init_module_mpiEv+0x547)[0x2b87f30d4967]
>>>>> [local:27921] [ 9]
>>>>> /home/user/myapp/lib/libboost_python.so.1.55.0(_ZN5boost6python21handle_exception_implENS_9function0IvEE+0x530)[0x2b87f3558430]
>>>>> [local:27921] [10]
>>>>> /home/user/myapp/lib/libboost_python.so.1.55.0(_ZN5boost6python16handle_exceptionIPFvvEEEbT_+0x38)[0x2b87f3559798]
>>>>> [local:27921] [11]
>>>>> /home/user/myapp/lib/libboost_python.so.1.55.0(_ZN5boost6python6detail11init_moduleEPKcPFvvE+0x63)[0x2b87f3559463]
>>>>> [local:27921] [12]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(_PyImport_LoadDynamicModule+0xc2)[0x2b87e8c79282]
>>>>> [local:27921] [13]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0[0x2b87e8c771a9]
>>>>> [local:27921] [14]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0[0x2b87e8c776c1]
>>>>> [local:27921] [15]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyImport_ImportModuleLevel+0x1b7)[0x2b87e8c77977]
>>>>> [local:27921] [16]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0[0x2b87e8c57bcd]
>>>>> [local:27921] [17]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b87e8bb7ae8]
>>>>> [local:27921] [18]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x2b87e8c58216]
>>>>> [local:27921] [19]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x381c)[0x2b87e8c5c79c]
>>>>> [local:27921] [20]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x8c9)[0x2b87e8c60c89]
>>>>> [local:27921] [21]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x2b87e8c60d02]
>>>>> [local:27921] [22]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyImport_ExecCodeModuleEx+0xc2)[0x2b87e8c74432]
>>>>> [local:27921] [23]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0[0x2b87e8c769f0]
>>>>> [local:27921] [24]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0[0x2b87e8c771a9]
>>>>> [local:27921] [25]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0[0x2b87e8c77642]
>>>>> [local:27921] [26]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyImport_ImportModuleLevel+0x1b7)[0x2b87e8c77977]
>>>>> [local:27921] [27]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0[0x2b87e8c57bcd]
>>>>> [local:27921] [28]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyObject_Call+0x68)[0x2b87e8bb7ae8]
>>>>> [local:27921] [29]
>>>>> /home/user/myapp/tools/python/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x56)[0x2b87e8c58216]
>>>>> [local:27921] *** End of error message ***
>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> us...@open-mpi.org
>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>> Link to this post:
>>>> http://www.open-mpi.org/community/lists/users/2014/09/25396.php
>>>
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/users/2014/09/25401.php
>>
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post:
>> http://www.open-mpi.org/community/lists/users/2014/09/25403.php
>
