Kilian - APEX+OTF should work fine with distributed runs. Are you running the job from a directory where you have write permission? Here are some tips -
- OTF requires a "unification" process at the end of the run. Ideally, I would use HPX to do that, but I haven't implemented that. Instead, I use the filesystem. So, each rank will write three (or so) "hidden" files in the current working directory, and rank/locality 0 will read them, unify them, and write out a "master" mapping for the other ranks. - By default, APEX will write the OTF2_archive directory of OTF2 files in the current working directory. - If you want to change the location of either of those two locations, set the APEX_OTF2_ARCHIVE_PATH environment variable before your run, and the files will be written there. The default value for APEX_OTF2_ARCHIVE_PATH is "." (current working directory, relative). Let me know if that works, and/or if you have any other problems. Thanks - Kevin > On Mar 8, 2019, at 4:44 AM, Kilian Werner <[email protected]> wrote: > > Thank you very much Kevin! > > It works flawlessly and is a very handy profiling tool. > > It is inteded to work within distributed settings to right? > As soon as I run the application with mpirun e.g. with multiple localities > apex complains: > > [OTF2] src/otf2_archive_int.c:1108: error: Unknown error code: Couldn't > create directories on root. > OTF2 Error: INVALID, Unknown error code > > > While I do not have root access while running the application, I do have > writing rights for the working directory (thus the directory where the > OTF2_archive is put). > > Am I missing some additional steps for running apex otf2 support on multiple > locations, is it not inteded to support that or is the problem probably > located within my toolchain and environment? > > Thanks, > > Kilian > > On Tue, 26 Feb 2019 10:13:12 -0800 > Kevin Huck <[email protected]> wrote: >> Kilian - >> You don't need TAU to generate OTF2 traces directly from APEX. Here is what >> you need to generate them: >> Get and build the OTF2 library (straightforward configure/make process): >> https://www.vi-hps.org/projects/score-p/ >> Configure HPX with APEX enabled and OTF2 support enabled: >> cmake <usual hpx options...> -DHPX_WITH_APEX=TRUE -DAPEX_WITH_OTF2=TRUE >> -DOTF2_ROOT=/path/to/otf2 ... >> If you want to make sure you have the most recent APEX, you can also add: >> -DHPX_WITH_APEX_NO_UPDATE=TRUE -DHPX_WITH_APEX_TAG=develop >> ...but I think the current HPX master uses APEX 2.1.2, which is up to date. >> After building HPX and your application, set the following environment >> variable before executing: >> APEX_OTF2=1 >> You should see an OTF_archive directory after the run, you can load the >> APEX.otf2 file in vampir. >> Let me know if you run into any problems - >> Kevin >>> On Feb 26, 2019, at 5:16 AM, Hartmut Kaiser <[email protected]> >>> wrote: >>> Kilian, >>> I am cc'ing Kevin Huck to make sure he sees this. He is Mr. APEX and will be >>> able to give you all the information you need. >>> Regards Hartmut >>> --------------- >>> http://stellar.cct.lsu.edu >>> https://github.com/STEllAR-GROUP/hpx >>>> -----Original Message----- >>>> From: [email protected] <hpx-users- >>>> [email protected]> On Behalf Of Kilian Werner >>>> Sent: Tuesday, February 26, 2019 3:03 AM >>>> To: [email protected] >>>> Subject: [hpx-users] HPX Profiling (TAU?) >>>> Dear list! >>>> I am trying to optimize my HPX project and was looking for a way to >>>> observe per-thread traces of the execution (in a vampir trace like >>>> fashion). >>>> I am already successfully utilizing APEX Support, however to my >>>> understanding this only yields total times/percentages. >>>> I did find references to HPX supporting TAU through APEX, however the >>>> HPX_WITH_TAU flag was removed in >>>> https://github.com/STEllAR-GROUP/hpx/pull/2380 >>>> Was TAU support dropped or did the process of building hpx with tau >>>> change? If so, how would I go about building it. >>>> If it was dropped are there any other ways to trace HPX in a vampir trace >>>> / gantt-chart like fashion? >>>> Best regards, >>>> Kilian Werner >>>> _______________________________________________ >>>> hpx-users mailing list >>>> [email protected] >>>> https://mail.cct.lsu.edu/mailman/listinfo/hpx-users >> -- >> Kevin Huck, PhD >> Research Associate / Computer Scientist >> OACISS - Oregon Advanced Computing Institute for Science and Society >> University of Oregon >> [email protected] >> http://tau.uoregon.edu > -- Kevin Huck, PhD Research Associate / Computer Scientist OACISS - Oregon Advanced Computing Institute for Science and Society University of Oregon [email protected] http://tau.uoregon.edu _______________________________________________ hpx-users mailing list [email protected] https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
