Dear Professor Schnetter, dear Professor Haas, thank you very much for your answer. The code I am using is the geodesic thorn written by Ian Hinder and Daniel Gerlicher which I am trying to parallelize. I have tried removing the interpolation entirely and the code ran faster with a factor of about 30 on 96 cores, but I have yet to benchmark it to find out where the time really goes. The problem is that I need to interpolate the fields and their derivatives in every timestep and I thought I could save time if the interpolation was done locally instead of communicating the particle coordinates in every step. So far, however, I have only used CarpetInterp and not CarpetInterp2.
I would be very interested to participate in the telecon. Kind regards, Cosima Breu > Hello Cosima, Erik, > > for code that finds out the "owner" of a particular location you can > look at the code in CarpetInterp itself. I had written a small helper > thorn that does this while at GeorgiaTech. You can find it here: > > https://github.com/rhaas80/MapPoints.git > > It provides a function MapPoints (in interface.ccl) that returns (among > other information) the MPI rank of the owning process: > > CCTK_INT FUNCTION MapPoints \ > (CCTK_POINTER_TO_CONST IN cctkGH, \ > CCTK_INT IN N_dims, \ > CCTK_INT IN param_table_handle, \ > CCTK_INT IN coord_system_handle, \ > CCTK_INT IN N_interp_points, \ > CCTK_INT IN interp_coords_type_code, \ > CCTK_POINTER_TO_CONST ARRAY IN coords_list, \ > CCTK_POINTER ARRAY OUT procs, \ > CCTK_POINTER ARRAY OUT rlev) > > Yours, > Roland > >> Cosima >> >> This information is available from Carpet. Carpet maintains a "grid >> hierarchy" gh, and if you use AMR, there will be a single gh that you >> can >> look up in Carpet, probably as "Carpet::vhh.at(0)" (this is C++ code). >> This >> grid hierarchy contains the information you want -- you can look up >> which >> regions exist, to which process they are assigned, and you can also look >> up >> on which region (and thus on which process) a particular point is >> located. >> >> Having said this -- are you sure that this will save a noticeable amount >> of >> time compared to letting Carpet handle the interpolation? Did you >> benchmark >> your current implementation? If you leave out the interpolation in your >> current implementation, how much time do you save? >> >> I'd be happy to discuss further. A good venue would e.g. be one of the >> Einstein Toolkit telecons that we have on Mondays. (I might not attend >> next >> Monday because I'll be at SC16.) >> >> -erik >> >> >> On Thu, Nov 10, 2016 at 8:54 AM, <[email protected]> >> wrote: >> >> > Dear users of the Einstein Toolkit, >> > >> > I have a question: I would like to integrate the geodesic equations >> for a >> > large quantity of particles in parallel (e.g. one million). To save >> > computational time, I want to make sure that the arrays containing the >> > particle data are handled by the same process that takes care of the >> > corresponding grid patch on which the particles are moving and only >> > communicate particle data if a particle moves to another grid patch. >> > >> > Since I only want a process to communicate with the processes that >> contain >> > the adjacent grid patches, can I access information on which process >> owns >> > which grid patch and if yes, how? >> > >> > So far I have only found functions that return the upper and lower >> bounds >> > for the local process, but each process would need to know where all >> the >> > rest of the grid is. >> > >> > Kind regards, >> > >> > Cosima Breu >> > >> > _______________________________________________ >> > Users mailing list >> > [email protected] >> > http://lists.einsteintoolkit.org/mailman/listinfo/users >> > >> >> >> > > > > -- > My email is as private as my paper mail. I therefore support encrypting > and signing email messages. Get my PGP key from http://keys.gnupg.net. > _______________________________________________ > Users mailing list > [email protected] > http://lists.einsteintoolkit.org/mailman/listinfo/users > _______________________________________________ Users mailing list [email protected] http://lists.einsteintoolkit.org/mailman/listinfo/users
