Re: [Users] problem on running bbh expample

2016-12-23 Thread Erik Schnetter
Cory

This indicates a severe problem. The most likely reason is that there
are two processes started, but they don't "find" each other, and then
both run independently thinking that there are the only process.

This can be caused by a mixup of MPI implementation. If you have two
MPI implementations installed, then you might be using one of them to
compile, but you might be using the "mpirun" command of the other one.
You will need to correct this for things to work. It's difficult to
tell from the outside what went wrong. The output from Cactus when it
configured should/might have stated where the MPI implementation is
located (e.g. "/usr/local"), and if you type "which mpirun", it should
tell you which you are using (maybe "/usr").

-erik


On Fri, Dec 23, 2016 at 11:07 AM, Cory Chu  wrote:
> Erik
>
> Actually, even with --procs=2 --num-threads=1
> I will get:
> The environment variable CACTUS_NUM_PROCS is set to 2, but there are 1 MPI
> processes.
>
> However, with only --procs=2 can really start a simulation, and terminate
> due to  TAT/Slab limitation
>
> Is there any configuration corresponding to
> "but there are 1 MPI processes. This may indicate a severe problem with the
> MPI startup mechanism."
>
> I'm using
> optionlist   ubuntu.cfg
> submitscript  generic.sub
> runscript  debian.run
>
> Cory
>
>
> On Fri, Dec 23, 2016 at 9:44 PM, Erik Schnetter 
> wrote:
>>
>> Cory
>>
>> Using "--procs=4 --num-threads=1" might help. You do not need to
>> change the settings in the ".ini" file; only the command line
>> parameters determine how many MPI processes are used.
>>
>> -erik
>>
>> On Fri, Dec 23, 2016 at 8:12 AM, Cory Chu  wrote:
>> > Hi Ian
>> >
>> > I'am running on VirtualBox set whit 2 Processors and 8GB RAM
>> >
>> > When i try with this submit command
>> >
>> > create-submit mybbh --parfile par/BBHLowRes.par --num-threads=1
>> > --procs=2
>> >
>> > I get following message in .err file
>> >
>> > WARNING level 0 from host kpc-VirtualBox process 0
>> >   while executing schedule bin (none), routine (no thorn)::(no routine)
>> >   in thorn Carpet, file
>> >
>> > /home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/Carpet/src/SetupGH.cc:153:
>> >   -> The environment variable CACTUS_NUM_PROCS is set to 2, but there
>> > are 1
>> > MPI processes. This may indicate a severe problem with the MPI startup
>> > mechanism.
>> > cactus_sim:
>> >
>> > /home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275:
>> > int Carpet::Abort(const cGH*, int): Assertion `0' failed.
>> > Rank 0 with PID 15985 received signal 6
>> > Writing backtrace to BBHLowRes/backtrace.0.txt
>> > Aborted (core dumped)
>> >
>> >
>> > i have change these setting in .ini file
>> > ppn =2
>> > max-num-threads =2
>> > num-threads =2
>> >
>> >
>> >
>> > On Fri, Dec 23, 2016 at 6:28 PM, Ian Hinder 
>> > wrote:
>> >>
>> >>
>> >> On 23 Dec 2016, at 10:52, Cory Chu  wrote:
>> >>
>> >> Erik
>> >>
>> >> Should I simply change the .ini file in simfactory/mdb/machines
>> >> with
>> >> ppn = 2
>> >> max-num-threads = 2
>> >> num-threads = 2
>> >>
>> >> and adding --procs = 2 on running command ?
>> >>
>> >> sorry i just can not find a detail documentation on this part
>> >> after i change the above setting
>> >> the problem did not solved
>> >>
>> >>
>> >> Hi Cory,
>> >>
>> >> Can you give the command line you are using to submit the simulation?
>> >> You
>> >> need to ensure that the number of MPI processes you use is larger.  You
>> >> could do this by running fewer threads per process, or if you are
>> >> already
>> >> running one thread per process, by increasing the number of processes
>> >> by
>> >> increasing the number of threads.  That might mean reconfiguring the
>> >> machine
>> >> definition to allow more cores to be used than you actually have.  What
>> >> machine is it; i.e. how many processors and cores do you have?
>> >>
>> >> --
>> >> Ian Hinder
>> >> http://members.aei.mpg.de/ianhin
>> >>
>> >
>>
>>
>>
>> --
>> Erik Schnetter 
>> http://www.perimeterinstitute.ca/personal/eschnetter/
>
>



-- 
Erik Schnetter 
http://www.perimeterinstitute.ca/personal/eschnetter/
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


Re: [Users] problem on running bbh expample

2016-12-23 Thread Cory Chu
Erik

Actually, even with --procs=2 --num-threads=1
I will get:
The environment variable CACTUS_NUM_PROCS is set to 2, but there are 1 MPI
processes.

However, with only --procs=2 can really start a simulation, and terminate
due to  TAT/Slab limitation

Is there any configuration corresponding to
"but there are 1 MPI processes. This may indicate a severe problem with the
MPI startup mechanism."

I'm using
optionlist   ubuntu.cfg
submitscript  generic.sub
runscript  debian.run

Cory


On Fri, Dec 23, 2016 at 9:44 PM, Erik Schnetter 
wrote:

> Cory
>
> Using "--procs=4 --num-threads=1" might help. You do not need to
> change the settings in the ".ini" file; only the command line
> parameters determine how many MPI processes are used.
>
> -erik
>
> On Fri, Dec 23, 2016 at 8:12 AM, Cory Chu  wrote:
> > Hi Ian
> >
> > I'am running on VirtualBox set whit 2 Processors and 8GB RAM
> >
> > When i try with this submit command
> >
> > create-submit mybbh --parfile par/BBHLowRes.par --num-threads=1 --procs=2
> >
> > I get following message in .err file
> >
> > WARNING level 0 from host kpc-VirtualBox process 0
> >   while executing schedule bin (none), routine (no thorn)::(no routine)
> >   in thorn Carpet, file
> > /home/kuangtly/programming/ET/forfun/Cactus/arrangements/
> Carpet/Carpet/src/SetupGH.cc:153:
> >   -> The environment variable CACTUS_NUM_PROCS is set to 2, but there
> are 1
> > MPI processes. This may indicate a severe problem with the MPI startup
> > mechanism.
> > cactus_sim:
> > /home/kuangtly/programming/ET/forfun/Cactus/arrangements/
> Carpet/Carpet/src/helpers.cc:275:
> > int Carpet::Abort(const cGH*, int): Assertion `0' failed.
> > Rank 0 with PID 15985 received signal 6
> > Writing backtrace to BBHLowRes/backtrace.0.txt
> > Aborted (core dumped)
> >
> >
> > i have change these setting in .ini file
> > ppn =2
> > max-num-threads =2
> > num-threads =2
> >
> >
> >
> > On Fri, Dec 23, 2016 at 6:28 PM, Ian Hinder 
> wrote:
> >>
> >>
> >> On 23 Dec 2016, at 10:52, Cory Chu  wrote:
> >>
> >> Erik
> >>
> >> Should I simply change the .ini file in simfactory/mdb/machines
> >> with
> >> ppn = 2
> >> max-num-threads = 2
> >> num-threads = 2
> >>
> >> and adding --procs = 2 on running command ?
> >>
> >> sorry i just can not find a detail documentation on this part
> >> after i change the above setting
> >> the problem did not solved
> >>
> >>
> >> Hi Cory,
> >>
> >> Can you give the command line you are using to submit the simulation?
> You
> >> need to ensure that the number of MPI processes you use is larger.  You
> >> could do this by running fewer threads per process, or if you are
> already
> >> running one thread per process, by increasing the number of processes by
> >> increasing the number of threads.  That might mean reconfiguring the
> machine
> >> definition to allow more cores to be used than you actually have.  What
> >> machine is it; i.e. how many processors and cores do you have?
> >>
> >> --
> >> Ian Hinder
> >> http://members.aei.mpg.de/ianhin
> >>
> >
>
>
>
> --
> Erik Schnetter 
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


Re: [Users] problem on running bbh expample

2016-12-23 Thread Erik Schnetter
Cory

Using "--procs=4 --num-threads=1" might help. You do not need to
change the settings in the ".ini" file; only the command line
parameters determine how many MPI processes are used.

-erik

On Fri, Dec 23, 2016 at 8:12 AM, Cory Chu  wrote:
> Hi Ian
>
> I'am running on VirtualBox set whit 2 Processors and 8GB RAM
>
> When i try with this submit command
>
> create-submit mybbh --parfile par/BBHLowRes.par --num-threads=1 --procs=2
>
> I get following message in .err file
>
> WARNING level 0 from host kpc-VirtualBox process 0
>   while executing schedule bin (none), routine (no thorn)::(no routine)
>   in thorn Carpet, file
> /home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/Carpet/src/SetupGH.cc:153:
>   -> The environment variable CACTUS_NUM_PROCS is set to 2, but there are 1
> MPI processes. This may indicate a severe problem with the MPI startup
> mechanism.
> cactus_sim:
> /home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275:
> int Carpet::Abort(const cGH*, int): Assertion `0' failed.
> Rank 0 with PID 15985 received signal 6
> Writing backtrace to BBHLowRes/backtrace.0.txt
> Aborted (core dumped)
>
>
> i have change these setting in .ini file
> ppn =2
> max-num-threads =2
> num-threads =2
>
>
>
> On Fri, Dec 23, 2016 at 6:28 PM, Ian Hinder  wrote:
>>
>>
>> On 23 Dec 2016, at 10:52, Cory Chu  wrote:
>>
>> Erik
>>
>> Should I simply change the .ini file in simfactory/mdb/machines
>> with
>> ppn = 2
>> max-num-threads = 2
>> num-threads = 2
>>
>> and adding --procs = 2 on running command ?
>>
>> sorry i just can not find a detail documentation on this part
>> after i change the above setting
>> the problem did not solved
>>
>>
>> Hi Cory,
>>
>> Can you give the command line you are using to submit the simulation?  You
>> need to ensure that the number of MPI processes you use is larger.  You
>> could do this by running fewer threads per process, or if you are already
>> running one thread per process, by increasing the number of processes by
>> increasing the number of threads.  That might mean reconfiguring the machine
>> definition to allow more cores to be used than you actually have.  What
>> machine is it; i.e. how many processors and cores do you have?
>>
>> --
>> Ian Hinder
>> http://members.aei.mpg.de/ianhin
>>
>



-- 
Erik Schnetter 
http://www.perimeterinstitute.ca/personal/eschnetter/
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


Re: [Users] problem on running bbh expample

2016-12-23 Thread Cory Chu
Hi Ian

I'am running on VirtualBox set whit 2 Processors and 8GB RAM

When i try with this submit command

create-submit mybbh --parfile par/BBHLowRes.par --num-threads=1 --procs=2

I get following message in .err file

WARNING level 0 from host kpc-VirtualBox process 0
  while executing schedule bin (none), routine (no thorn)::(no routine)
  in thorn Carpet, file
/home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/Carpet/src/SetupGH.cc:153:
  -> The environment variable CACTUS_NUM_PROCS is set to 2, but there are 1
MPI processes. This may indicate a severe problem with the MPI startup
mechanism.
cactus_sim:
/home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275:
int Carpet::Abort(const cGH*, int): Assertion `0' failed.
Rank 0 with PID 15985 received signal 6
Writing backtrace to BBHLowRes/backtrace.0.txt
Aborted (core dumped)


i have change these setting in .ini file
ppn =2
max-num-threads =2
num-threads =2



On Fri, Dec 23, 2016 at 6:28 PM, Ian Hinder  wrote:

>
> On 23 Dec 2016, at 10:52, Cory Chu  wrote:
>
> Erik
>
> Should I simply change the .ini file in simfactory/mdb/machines
> with
> ppn = 2
> max-num-threads = 2
> num-threads = 2
>
> and adding --procs = 2 on running command ?
>
> sorry i just can not find a detail documentation on this part
> after i change the above setting
> the problem did not solved
>
>
> Hi Cory,
>
> Can you give the command line you are using to submit the simulation?  You
> need to ensure that the number of MPI processes you use is larger.  You
> could do this by running fewer threads per process, or if you are already
> running one thread per process, by increasing the number of processes by
> increasing the number of threads.  That might mean reconfiguring the
> machine definition to allow more cores to be used than you actually have.
> What machine is it; i.e. how many processors and cores do you have?
>
> --
> Ian Hinder
> http://members.aei.mpg.de/ianhin
>
>
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


Re: [Users] problem on running bbh expample

2016-12-23 Thread Ian Hinder

On 23 Dec 2016, at 10:52, Cory Chu  wrote:

> Erik
> 
> Should I simply change the .ini file in simfactory/mdb/machines
> with
> ppn = 2
> max-num-threads = 2
> num-threads = 2
> 
> and adding --procs = 2 on running command ?
> 
> sorry i just can not find a detail documentation on this part
> after i change the above setting
> the problem did not solved 

Hi Cory,

Can you give the command line you are using to submit the simulation?  You need 
to ensure that the number of MPI processes you use is larger.  You could do 
this by running fewer threads per process, or if you are already running one 
thread per process, by increasing the number of processes by increasing the 
number of threads.  That might mean reconfiguring the machine definition to 
allow more cores to be used than you actually have.  What machine is it; i.e. 
how many processors and cores do you have?

-- 
Ian Hinder
http://members.aei.mpg.de/ianhin

___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


Re: [Users] problem on running bbh expample

2016-12-23 Thread Cory Chu
Erik

Should I simply change the .ini file in simfactory/mdb/machines
with
ppn = 2
max-num-threads = 2
num-threads = 2

and adding --procs = 2 on running command ?

sorry i just can not find a detail documentation on this part
after i change the above setting
the problem did not solved

Cory


On Fri, Dec 23, 2016 at 5:56 AM, Erik Schnetter 
wrote:

> Cory
>
> The root of the problem is this:
>
> On Thu, Dec 22, 2016 at 3:50 PM, Cory Chu  wrote:
>
>> TAT/Slab can only be used if there is a single local component per MPI
>> process
>
>
> The thorn TAT/Slab has certain limitations, and these require that a
> certain minimum number of MPI processes is used, so that certain operations
> can occur simultaneously. If you increase the number of MPI processes the
> error should go away.
>
> -erik
>
> --
> Erik Schnetter 
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


Re: [Users] problem on running bbh expample

2016-12-22 Thread Erik Schnetter
Cory

The root of the problem is this:

On Thu, Dec 22, 2016 at 3:50 PM, Cory Chu  wrote:

> TAT/Slab can only be used if there is a single local component per MPI
> process


The thorn TAT/Slab has certain limitations, and these require that a
certain minimum number of MPI processes is used, so that certain operations
can occur simultaneously. If you increase the number of MPI processes the
error should go away.

-erik

-- 
Erik Schnetter 
http://www.perimeterinstitute.ca/personal/eschnetter/
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users


[Users] problem on running bbh expample

2016-12-22 Thread Cory Chu
Hi,

I'm using Payne-Gaposchkin release
to run the BBHLowRes.par  example

basically, follow this instruction https://arxiv.org/abs/1305.5299


however, it seems that the simlution terminated too early.
this is the plot of data in BH_diagnostics.ahX.gp i got

‚Äč
i noticed that in the output of "sim list-sim"
it says "Warning: job status is U"

and this is the from .err file in output-  (partially)

.

WARNING level 1 from host kpc-VirtualBox process 0
  while executing schedule bin CCTK_PARAMCHECK, routine
ML_BSSN_Helper::ML_BSSN_ParamCheck
  in thorn ML_BSSN_Helper, file
/home/kuangtly/programming/ET/forfun/Cactus/arrangements/McLachlan/ML_BSSN_Helper/src/Parameters.c:204:
  -> Parameter ML_BSSN::ShiftAdvectionCoeff is outdated; please update the
parameter file. Instead of using this parameter, you should set
ML_BSSN::advectShift.
WARNING level 1 from host kpc-VirtualBox process 0
  while executing schedule bin (none), routine (no thorn)::(no routine)
  in thorn CarpetRegrid2, file
/home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/CarpetRegrid2/src/regrid.cc:389:
  -> Could not enforce grid structure properties in this round, starting
another round
WARNING level 0 from host kpc-VirtualBox process 0
  while executing schedule bin BoundaryConditions, routine
RotatingSymmetry180::Rot180_ApplyBC
  in thorn RotatingSymmetry180, file
/home/kuangtly/programming/ET/forfun/Cactus/arrangements/CactusNumerical/RotatingSymmetry180/src/rotatingsymmetry180.c:447:
  -> TAT/Slab can only be used if there is a single local component per MPI
process
cactus_sim:
/home/kuangtly/programming/ET/forfun/Cactus/arrangements/Carpet/Carpet/src/helpers.cc:275:
int Carpet::Abort(const cGH*, int): Assertion `0' failed.
Rank 0 with PID 15757 received signal 6
Writing backtrace to mybbh/backtrace.0.txt
Aborted (core dumped)
=


I can't figure out what's the problem, even where to get the error message
correctly.
I have try to change the initial data given by "two puncture", and it turns
out that those BH can't move too far as above, although they really changed
the trajectory.


Environment:
virtualBox
8G RAM assigned
Ubuntu 16.04.1



Could anyone give me some hint~


Thanks a lot !
Cory
___
Users mailing list
Users@einsteintoolkit.org
http://lists.einsteintoolkit.org/mailman/listinfo/users