On Wed, 25 Sep 2019, Mark Adams via petsc-dev wrote: > On Wed, Sep 25, 2019 at 12:44 PM Balay, Satish <[email protected]> wrote: > > > Can you retry with updated balay/fix-mpiexec-shell-escape branch? > > > > > > current mpiexec interface/code in petsc is messy. > > > > Its primarily needed for the test suite. But then - you can't easily > > run the test suite on machines like summit. > > > > Also - it assumes mpiexec provided supports '-n 1'. However if one > > provides non-standard mpiexec such as --with-mpiexec="jsrun -g 1" - > > what is the appropriate thing here? > > > > jsrun does take -n. It just has other args. I am trying to check if it > requires other args. I thought it did but let me check.
https://www.olcf.ornl.gov/for-users/system-user-guides/summitdev-quickstart-guide/ -n --nrs Number of resource sets Beta2 Change (October 17): -n was be replaced by -nnodes So its not the same functionality as 'mpiexec -n' Either way - please try the above branch Satish > > > > > > And then configure needs to run some binaries for some checks - here > > perhaps '-n 1' doesn't matter. [MPICH defaults to 1, OpenMPI defaults > > to ncore]. So perhaps mpiexec is required for this purpose on summit? > > > > And then there is this code to escape spaces in path - for > > windows. [but we have to make sure this is not in code-path for user > > specified --with-mpiexec="jsrun -g 1" > > > > Satish > > > > On Wed, 25 Sep 2019, Mark Adams via petsc-dev wrote: > > > > > No luck, > > > > > > On Wed, Sep 25, 2019 at 10:01 AM Balay, Satish <[email protected]> > > wrote: > > > > > > > Mark, > > > > > > > > Can you try the fix in branch balay/fix-mpiexec-shell-escape and see > > if it > > > > works? > > > > > > > > Satish > > > > > > > > On Wed, 25 Sep 2019, Balay, Satish via petsc-dev wrote: > > > > > > > > > Mark, > > > > > > > > > > Can you send configure.log from mark/fix-cuda-with-gamg-pintocpu > > branch? > > > > > > > > > > Satish > > > > > > > > > > On Wed, 25 Sep 2019, Mark Adams via petsc-dev wrote: > > > > > > > > > > > I double checked that a clean build of your (master) branch has > > this > > > > error > > > > > > by my branch (mark/fix-cuda-with-gamg-pintocpu), which may include > > > > stuff > > > > > > from Barry that is not yet in master, works. > > > > > > > > > > > > On Wed, Sep 25, 2019 at 5:26 AM Karl Rupp via petsc-dev < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > On 9/25/19 11:12 AM, Mark Adams via petsc-dev wrote: > > > > > > > > I am using karlrupp/fix-cuda-streams, merged with master, and I > > > > get this > > > > > > > > error: > > > > > > > > > > > > > > > > Could not execute "['jsrun -g\\ 1 -c\\ 1 -a\\ 1 > > --oversubscribe -n > > > > 1 > > > > > > > > printenv']": > > > > > > > > Error, invalid argument: 1 > > > > > > > > > > > > > > > > My branch mark/fix-cuda-with-gamg-pintocpu seems to work but I > > did > > > > edit > > > > > > > > the jsrun command but Karl's branch still fails. (SUMMIT was > > down > > > > today > > > > > > > > so there could have been updates). > > > > > > > > > > > > > > > > Any suggestions? > > > > > > > > > > > > > > Looks very much like a systems issue to me. > > > > > > > > > > > > > > Best regards, > > > > > > > Karli > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
