Dave, Thank you for your detailed report and testing, that is indeed very helpful. We will definitely have to do something. Here is what I think would be potentially doable.
a) if we detect a Lustre file system without flock support, we can printout an error message. Completely disabling MPI I/O is on the ompio architecture not possible at the moment, since the Lustre component can disqualify itself, but the generic Unix FS component would kick in in that case, and still continue execution. To be more precise, the query function of the Lustre component has no way to return anything than "I am interested to run" or "I am not interested to run" b) I can add an MCA parameter that would allow the Lustre component to abort execution of the job entirely. While this parameter would probably be by default set to 'false', a system administrator could configure it to be set to 'true' an particular platform. I will discuss this also with a couple of other people in the next couple of days. Thanks Edgar > -----Original Message----- > From: users [mailto:users-boun...@lists.open-mpi.org] On Behalf Of Dave > Love > Sent: Monday, October 15, 2018 4:22 AM > To: Open MPI Users <users@lists.open-mpi.org> > Subject: Re: [OMPI users] ompio on Lustre > > For what it's worth, I found the following from running ROMIO's tests with > OMPIO on Lustre mounted without flock (or localflock). I used 48 processes > on two nodes with Lustre for tests which don't require a specific number. > > OMPIO fails tests atomicity, misc, and error on ext4; it additionally fails > noncontig_coll2, fp, shared_fp, and ordered_fp on Lustre/noflock. > > On Lustre/noflock, ROMIO fails on atomicity, i_noncontig, noncontig, > shared_fp, ordered_fp, and error. > > Please can OMPIO be changed to fail in the same way as ROMIO (with a clear > message) for the operations it can't support without flock. > Otherwise it looks as if you can potentially get invalid data, or at least > waste > time debugging other errors. > > I'd debug the common failure on the "error" test, but ptrace is disabled on > the > system. > > In case anyone else is in the same boat and can't get mounts changed, I > suggested staging data to and from a PVFS2^WOrangeFS ephemeral > filesystem on jobs' TMPDIR local mounts if they will fit. Of course other > libraries will potentially corrupt data on nolock mounts. > _______________________________________________ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users _______________________________________________ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
