Re: [Beowulf] PMIX and Julia?
On Monday, 19 November 2018 6:35:56 AM AEDT John Hearns via Beowulf wrote: > What I am really asking is will pmix be totally necessary when running on > near-exascale systems, or am I missing something? My thoughts are should > the Julia world be looking at mpix adaptations? If someone with a clue > about pmix could enlighten me I would be grateful. My (limited) understanding of this is that PMI* is an MPI wire-up protocol, in other words a mechanism for the MPI ranks to discover each other, set up communications and also talk to the resource scheduler (if present). There's a handy little description of PMI (v2 in this case) here: https://wiki.mpich.org/mpich/index.php/PMI_v2_API The PMIx website (along with standards documents etc) here: https://pmix.org/ My instinct is that it might be better for Julia to sit on MPI and let it handle this for it, rather than have to know about PMI2/PMIx itself.. All the best! Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC ___ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
[Beowulf] PMIX and Julia?
I am really going to show my ignorance here. I just flicked through the excellent OpenMPI BoF slides from SC18 https://www.open-mpi.org/papers/sc-2018/Open-MPI-SC18-BOF.pdf This reminded me of something I have at the back of my mind. Clearly pmix is intimately associated with MPI. On the other hand I am a fan of the Julia language, which has its own concepts of parallelism. However for a cluster spanning across multiple machines Julia currently assumes passwordless ssh will be used https://docs.julialang.org/en/v1/manual/parallel-computing/index.html#Starting-and-managing-worker-processes-1 Oooh - a bit of googling reveals that MPI can be used to communicate with the processes https://github.com/JuliaParallel/MPI.jl/blob/master/test/test_cman_mpi.jl What I am really asking is will pmix be totally necessary when running on near-exascale systems, or am I missing something? My thoughts are should the Julia world be looking at mpix adaptations? If someone with a clue about pmix could enlighten me I would be grateful. ___ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
