Hello Karima, summarizing the discussion in the call today:
* for --cores 4 --num-threads 2 the issue is that simulation factory counts only "physical" cores while the OS reports "logical" cores including the hyperthreading which we originally chose to ignore * for the other error --cores 2 --num-threads 1 "The domain size for map 0 scaled for convergence level 0 with convergence factor 2 is not integer" would seem to be that the coarse grid spacing "dx" does not evenly divide the grid size xmax - xmin. If the grid spacing is indeed correct, as you indicated in the call, then best would be to provide the exact parameter file that you ran. Yours, Roland > Dear Dr. Ronald and Dr. Peter, > I have a 4 core laptop, and I am using 'Simfactory' command to run the par > file. > I have tested both of your suggestions. The addition of "--cores=2 > --num-threads=1" gives me the attached error while the "--cores=4 > --num-threads=2" > option gives, "Error: Too many nodes specified: nodes=2 (maxnodes is 1) > > Aborting Simfactory". > > > Thanks > > Best, > > KS > > On Tue, Jan 19, 2021 at 11:29 PM <[email protected]> wrote: > > > Send Users mailing list submissions to > > [email protected] > > > > To subscribe or unsubscribe via the World Wide Web, visit > > > > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$ > > > > or, via email, send a message with subject or body 'help' to > > [email protected] > > > > You can reach the person managing the list at > > [email protected] > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Users digest..." > > > > > > Today's Topics: > > > > 1. TAT/ Slab Error (KARIMA SHAHZAD) > > 2. Re: TAT/ Slab Error (Peter Diener) > > 3. Agenda for Thursday's Meeting ([email protected]) > > 4. Re: TAT/ Slab Error (Roland Haas) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Mon, 18 Jan 2021 17:41:04 +0000 > > From: KARIMA SHAHZAD <[email protected]> > > Subject: [Users] TAT/ Slab Error > > To: "[email protected]" <[email protected]> > > Message-ID: > > < > > db7pr07mb5435b609daf0209b60980c8e8c...@db7pr07mb5435.eurprd07.prod.outlook.com > > > > > > > Content-Type: text/plain; charset="iso-8859-1" > > > > Hello there, > > > > Here is the error attached I encountered while evolving black holes > > binary. The work-around I guess is to increase the number of processes (if > > I am not wrong), but I am not quite sure how to do that. I need some help > > to get around this issue. > > > > Thanks in advance > > > > Best regards, > > KS > > > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: > > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.html__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqqXuJnIl$ > > > > -------------- next part -------------- > > A non-text attachment was scrubbed... > > Name: bbhHr.err > > Type: application/octet-stream > > Size: 1075 bytes > > Desc: bbhHr.err > > Url : > > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/64c8b97d/attachment-0001.obj__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqre9Jkw5$ > > > > > > ------------------------------ > > > > Message: 2 > > Date: Mon, 18 Jan 2021 14:19:02 -0600 (CST) > > From: Peter Diener <[email protected]> > > Subject: Re: [Users] TAT/ Slab Error > > To: KARIMA SHAHZAD <[email protected]> > > Cc: "[email protected]" <[email protected]> > > Message-ID: <[email protected]> > > Content-Type: text/plain; charset="iso-8859-1" > > > > Hi Karima, > > > > Yes, that parameter file requires at least 2 mpi processes to run. > > > > You seem to have submitted using simfactory and ended up running on 1 > > mpi process awith 2 openmp threads. If you instead do something like > > > > simfactory/bin/sim run bbhHr --cores=2 --num-threads=1 > > > > if you're on a workstation that doesn't use a queueing system or > > > > simfactory/bin/sim submit bbhHr --cores=2 --num-threads=1 > > > > if you're on a cluster. This should give you 2 mpi processes running on > > 1 openmp thread each. You might have to add other options to simfactory > > if you did so in your original run. > > > > Cheers, > > > > Peter > > > > On Monday 2021-01-18 11:41, KARIMA SHAHZAD wrote: > > > > >Date: Mon, 18 Jan 2021 11:41:04 > > >From: KARIMA SHAHZAD <[email protected]> > > >To: "[email protected]" <[email protected]> > > >Subject: [Users] TAT/ Slab Error > > > > > >Hello there,? > > > > > >Here is the error attached I encountered while evolving black holes > > binary. > > >The work-around I guess is to increase the number of processes (if I am > > not > > >wrong), but I am not quite sure how to do that. I need some help to get > > >around this issue. > > > > > >Thanks in advance > > > > > >Best regards, > > >KS > > > > > > > > > > > > > ------------------------------ > > > > Message: 3 > > Date: Mon, 18 Jan 2021 15:18:01 -0600 > > From: [email protected] > > Subject: [Users] Agenda for Thursday's Meeting > > To: [email protected] > > Message-ID: <E1l1buT-009sDh-Ev@ekohaes8> > > Content-Type: text/plain; charset="ANSI_X3.4-1968" > > > > Please update the Wiki with agenda items for Thursday's meeting. Thanks! > > > > https://urldefense.com/v3/__https://docs.einsteintoolkit.org/et-docs/meeting_agenda__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqhEBI4Ov$ > > > > > > --The Maintainers > > > > > > ------------------------------ > > > > Message: 4 > > Date: Mon, 18 Jan 2021 16:17:32 -0600 > > From: Roland Haas <[email protected]> > > Subject: Re: [Users] TAT/ Slab Error > > To: KARIMA SHAHZAD <[email protected]> > > Cc: "[email protected]" <[email protected]> > > Message-ID: <[email protected]> > > Content-Type: text/plain; charset="us-ascii" > > > > Hello Karima, > > > > > Here is the error attached I encountered while evolving black holes > > > binary. The work-around I guess is to increase the number of > > > processes (if I am not wrong), but I am not quite sure how to do > > > that. I need some help to get around this issue. > > Assuming you use simulation factory you need to pass options to make > > sure it creates multiple MPI processes. The auto-detection code tends > > to choose a number of threads equal to the the number of cores on your > > laptop and and a single MPI rank. > > > > Eg if you have a 4 core laptop then you have to use: > > > > ./simfactory/bin/sim submit testrun01 --cores 4 --num-threads 2 > > --parfile ... > > > > which uses a total of 4 cores and starts 2 threads per MPI rank so that > > you end up with 4 / 2 = 2 MPI ranks each of which uses 2 threads. > > > > Similarly on clusters, the basic idea is to set --num-threads so that > > there are more than 1 MPI rank started (ie --num-threads is larger than > > the value for --cores or --procs [which are synonyms for each other]). > > > > If not using simulation factory you have to manually use mpirun and > > OMP_NUM_THREADS. Eg: > > > > export OMP_NUM_THREADS=2 > > > > mpirun -np 2 /home/karima/simulations/bbhHr/SIMFACTORY/exe/cactus_sim > > -L 3 /home/karima/simulations/bbhHr/output-0000/BBHHigherRes.par > > > > which starts 2 MPI ranks each will use 2 OpenMP threads. > > > > Yours, > > Roland > > > > -- > > My email is as private as my paper mail. I therefore support encrypting > > and signing email messages. Get my PGP key from > > https://urldefense.com/v3/__http://pgp.mit.edu__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqglVIuVi$ > > . > > -------------- next part -------------- > > A non-text attachment was scrubbed... > > Name: not available > > Type: application/pgp-signature > > Size: 833 bytes > > Desc: OpenPGP digital signature > > Url : > > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/pipermail/users/attachments/20210118/78ab7576/attachment-0001.bin__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqifXAHiE$ > > > > > > ------------------------------ > > > > _______________________________________________ > > Users mailing list > > [email protected] > > https://urldefense.com/v3/__http://lists.einsteintoolkit.org/mailman/listinfo/users__;!!DZ3fjg!qvVdUF1VaSq1y2DEvNkCQdSQOMn9alZKqAjw5plhvkzwHLGuAxOF8-TFqvI47uvX$ > > > > > > > > End of Users Digest, Vol 130, Issue 18 > > ************************************** > > -- My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from http://pgp.mit.edu .
pgpyqUCfyrZiY.pgp
Description: OpenPGP digital signature
_______________________________________________ Users mailing list [email protected] http://lists.einsteintoolkit.org/mailman/listinfo/users
