Hi all, Just wondering if there has been any developments regarding Phi cards?
We have just installed a small 10-node cluster with two MICs per node, and are wondering how best to use the cards. I intend to try the cross-compilation to install slurm on the cards, and then have a separate queue, similar to that described in the discussion on linkedin below. But I think the users would also like to be able to run offload as well, so that's a bit of an issue. I have an additional question: the users have asked if a portion of the host memory can be ``reserved'' for comms with the MICs, with the rest available for whoever runs on the host. Is that possible?? Thanks, Paddy On Tue, Feb 18, 2014 at 01:39:44AM -0800, Olli-Pekka Lehto wrote: > > Getting the daemon running on the Phi is certainly possible and we tried this > a year or > so ago. The real challenge lies in being able to run offload, host-native, > symmetric and > Phi-native mode programs nicely on the same set of nodes. It is something > that would > really be needed in order to maximize utilization of a Phi-cluster. > > Furthermore, having a simple way to maintain affinity of Phi cards with their > associated > hosts when doing symmetric runs would be very useful. It's sort of possible > with the topology > plugin but a bit clunky. > > Also it would be nice to have a more lightweight daemon in order to conserve > the precious > resources of the Phi as was presented in the Slurm User Group presentation. I > expect that > this would be a more significant undertaking, however(?) > > I'm wondering if people have been working on these kind of things? > > Olli-Pekka > > On Feb 18, 2014, at 1:05 PM, Ralph Castain <[email protected]> wrote: > > > > > I know others have direct-launched processes onto the Phi before, both with > > Slurm and just using rsh/ssh. The OpenMPI user mailing list archive talks > > about the ssh method (search for "phi" and you'll see the chatter) > > > > http://www.open-mpi.org/community/lists/users/ > > > > and the folks at Bright talk about how they did it with Slurm here: > > > > https://www.linkedin.com/groups/Yes-we-run-SLURM-inside-4501392.S.5792769036550955008 > > > > Ralph > > > > On Feb 17, 2014, at 5:46 PM, Christopher Samuel <[email protected]> > > wrote: > > > >> > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA1 > >> > >> Hi all, > >> > >> At the Slurm User Group in Oakland last year it was mentioned that > >> there was intended to be support for a lightweight Slurm daemon on > >> Xeon Phi (MIC) cards. > >> > >> I had a quick look in the git master last night but couldn't spot > >> anything related, is this still the intention? > >> > >> Olli-Pekka Lehto from CSC is running a Xeon Phi workshop at VLSCI at > >> the moment and it's of interest to a number of us. > >> > >> We're going to run a hack day on Wednesday and we'll see if we can > >> build an LDAP enabled Xeon Phi stack, if we can then we we'll see if > >> we can get standard Slurm going too. Nothing like having lofty goals! > >> > >> All the best! > >> Chris > >> - -- > >> Christopher Samuel Senior Systems Administrator > >> VLSCI - Victorian Life Sciences Computation Initiative > >> Email: [email protected] Phone: +61 (0)3 903 55545 > >> http://www.vlsci.org.au/ http://twitter.com/vlsci > >> > >> -----BEGIN PGP SIGNATURE----- > >> Version: GnuPG v1.4.14 (GNU/Linux) > >> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > >> > >> iEYEARECAAYFAlMCuvIACgkQO2KABBYQAh+pwgCcCLPvoUJamArfmpxY5igcJm3I > >> 0p0AnjF51qUgZfoZtIsKTDLCK+pJe+bf > >> =7HO3 > >> -----END PGP SIGNATURE----- > -- Paddy Doyle Trinity Centre for High Performance Computing, Lloyd Building, Trinity College Dublin, Dublin 2, Ireland. Phone: +353-1-896-3725 http://www.tchpc.tcd.ie/
