Re: MPI implementations in squeeze
Am Sonntag, den 21.03.2010, 12:49 +0100 schrieb Thibaut Paumard: I am about to upload a new package which build-depends on mpi-default- dev (it will need to be checked by a sponsor and then go through NEW). It it ok to do so in the next couple of days or should I wait for the new mpi-default-dev? I do not think it does any harm to upload it. If it hits the archive before the new mpi-defaults, it will be binNMUed; if it hits it later, there is no problem. But you'd might wait for the outcome of the other thread you started, in case your rules file needs to be updated slightly. ;) Best regards, Manuel -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1269368025.16192.27.ca...@ce170155.zmb.uni-duisburg-essen.de
Re: MPI implementations in squeeze
Hi, I am about to upload a new package which build-depends on mpi-default- dev (it will need to be checked by a sponsor and then go through NEW). It it ok to do so in the next couple of days or should I wait for the new mpi-default-dev? Regards, Thibaut. -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/98447058-9033-4e2b-8da5-d630e363e...@free.fr
Re: MPI implementations in squeeze
tags 563705 +patch thanks Hi and apologies for the long delay... On Fri, 2010-02-26 at 14:55 -0500, Adam C Powell IV wrote: On Fri, 2010-02-26 at 20:22 +0100, Lucas Nussbaum wrote: On 26/02/10 at 10:46 -0500, Adam C Powell IV wrote: On Thu, 2010-02-25 at 20:49 +0100, Lucas Nussbaum wrote: On 25/02/10 at 14:22 -0500, Adam C Powell IV wrote: On Thu, 2010-02-25 at 18:10 +0100, Lucas Nussbaum wrote: There is not much progress so far with respect to changing mpi-defaults to use MPICH2 instead of LAM on the architectures where Open MPI is not available yet. This needs a round of binNMUs. Marc Brockschmidt said he will look at the request to debian-release in the next few days, so this might resolve soon as well. Something to consider: this will break a lot of packages which use FORTRAN until 563705 is fixed, and then that will require mods to packages. I understand that bug as: if mpich2 or openmpi don't do the right thing when calling mpif77/mpif90, then symlinks are needed. Is there a proof that either of them doesn't do the right thing? Wouldn't it be more appropriate to fix them to do the right thing? (Those are honest questions -- I don't know anything about fortran) As discussed before (including in the bug), when there are mixed FORTRAN and C++ symbols, it's not clear whether to use mpif77/90 or mpic++. Also, it's a big convenience: a lot of packages make multiple executables and/or libraries, some of which use MPI and some don't. Pointing them to -lmpi -lmpi++ -lmpif77 for the MPI execs/lib directories seems easier than telling them to use mpicc and friends for some targets and gcc for others. I'm not sure I buy that, since mpicc friends also hide include paths, which are not handled with alternatives currently. Are you sure? % update-alternatives --display mpi mpi - auto mode link currently points to /usr/lib/openmpi/include /usr/lib/openmpi/include - priority 40 slave libmpi++.so: /usr/lib/openmpi/lib/libmpi_cxx.so [And a bunch of other slaves] It sounds more like a way to break packages by getting them linked with the wrong version of MPI. Do you know of packages doing that already? I've used this in most of my packages: petsc, hypre, libmesh, the new netgen and med-fichier under development (pending togl and updated hdf5 respectively), and salomé under development. Why would this break packages, if they just build-depend on mpi-default-dev? If the mpicc/mpif77 etc. alternatives work, why not the includes and libs as well? OK. Since it's harmless anyway, could you prepare and test patches for openmpi and mpich2? Since it would be a slave alternative, it doesn't require any alternatives transition. Will do, thanks. I'll also need to patch (at least my) packages which use this, will get to that in short order. Short order turned out to be somewhat long order, but here are the patches for mpich2 and openmpi. -Adam -- GPG fingerprint: D54D 1AEE B11C CE9B A02B C5DD 526F 01E8 564E E4B6 Engineering consulting with open source tools http://www.opennovation.com/ --- mpich2-1.2.1/debian/libmpich2-dev.postinst~ 2010-02-28 08:04:31.0 -0500 +++ mpich2-1.2.1/debian/libmpich2-dev.postinst 2010-03-09 10:56:08.0 -0500 @@ -19,6 +19,8 @@ --install /usr/include/mpi mpi /usr/include/mpich2 40 \ --slave /usr/lib/libmpi.so libmpi.so /usr/lib/libmpich.so \ --slave /usr/lib/libmpi++.so libmpi++.so /usr/lib/libmpichcxx.so \ + --slave /usr/lib/libmpif77.so libmpif77.so /usr/lib/libfmpich.so \ + --slave /usr/lib/libmpif90.so libmpif90.so /usr/lib/libmpichf90.so \ --slave /usr/bin/mpicc mpicc /usr/bin/mpicc.mpich2 \ --slave /usr/bin/mpic++ mpic++ /usr/bin/mpic++.mpich2 \ --slave /usr/bin/mpicxx mpicxx /usr/bin/mpicxx.mpich2 \ --- mpich2-1.2.1/debian/changelog~ 2010-02-28 08:04:31.0 -0500 +++ mpich2-1.2.1/debian/changelog 2010-03-09 11:04:13.0 -0500 @@ -1,3 +1,11 @@ +mpich2 (1.2.1-3) UNRELEASED; urgency=low + + [Adam C. Powell, IV] + * Added slave alternatives symlinks for MPI FORTRAN libraries +(closes: #563705). + + -- Adam C. Powell, IV hazel...@debian.org Tue, 09 Mar 2010 11:04:13 -0500 + mpich2 (1.2.1-2) unstable; urgency=low * Remove bashism in postinst introduced in 1.2.1-1. (fixes piuparts failure) --- openmpi-1.4.1/debian/libopenmpi-dev.postinst~ 2010-02-28 08:04:26.0 -0500 +++ openmpi-1.4.1/debian/libopenmpi-dev.postinst 2010-03-09 08:16:16.0 -0500 @@ -6,6 +6,8 @@ --install /usr/include/mpi mpi /usr/lib/openmpi/include 40 \ --slave /usr/lib/libmpi.so libmpi.so /usr/lib/openmpi/lib/libmpi.so \ --slave /usr/lib/libmpi++.so libmpi++.so /usr/lib/openmpi/lib/libmpi_cxx.so \ + --slave /usr/lib/libmpif77.so
Re: MPI implementations in squeeze
On Thu, 2010-02-25 at 20:49 +0100, Lucas Nussbaum wrote: On 25/02/10 at 14:22 -0500, Adam C Powell IV wrote: On Thu, 2010-02-25 at 18:10 +0100, Lucas Nussbaum wrote: There is not much progress so far with respect to changing mpi-defaults to use MPICH2 instead of LAM on the architectures where Open MPI is not available yet. This needs a round of binNMUs. Marc Brockschmidt said he will look at the request to debian-release in the next few days, so this might resolve soon as well. Something to consider: this will break a lot of packages which use FORTRAN until 563705 is fixed, and then that will require mods to packages. I understand that bug as: if mpich2 or openmpi don't do the right thing when calling mpif77/mpif90, then symlinks are needed. Is there a proof that either of them doesn't do the right thing? Wouldn't it be more appropriate to fix them to do the right thing? (Those are honest questions -- I don't know anything about fortran) As discussed before (including in the bug), when there are mixed FORTRAN and C++ symbols, it's not clear whether to use mpif77/90 or mpic++. Also, it's a big convenience: a lot of packages make multiple executables and/or libraries, some of which use MPI and some don't. Pointing them to -lmpi -lmpi++ -lmpif77 for the MPI execs/lib directories seems easier than telling them to use mpicc and friends for some targets and gcc for others. I'm not sure I buy that, since mpicc friends also hide include paths, which are not handled with alternatives currently. Are you sure? % update-alternatives --display mpi mpi - auto mode link currently points to /usr/lib/openmpi/include /usr/lib/openmpi/include - priority 40 slave libmpi++.so: /usr/lib/openmpi/lib/libmpi_cxx.so [And a bunch of other slaves] It sounds more like a way to break packages by getting them linked with the wrong version of MPI. Do you know of packages doing that already? I've used this in most of my packages: petsc, hypre, libmesh, the new netgen and med-fichier under development (pending togl and updated hdf5 respectively), and salomé under development. Why would this break packages, if they just build-depend on mpi-default-dev? If the mpicc/mpif77 etc. alternatives work, why not the includes and libs as well? And for something like med-fichier, wouldn't CC=mpicc and friends unnecessarily link a lot of non-MPI executables and libs to MPI libraries, causing memory bloat? Finally, are you sure mpif77 c++-object.o c-object.o f77-object.o -o executable will bring in all of the necessary libraries? I've never tried it, but I'd want to make sure it works for at least OpenMPI and MPICH2 before committing to use it. And we have libmpi.so and libmpi++.so symlinks, why not libmpif77.so? :) Well, maybe it might be a better idea to drop those links ;) Fair enough. :-) -Adam -- GPG fingerprint: D54D 1AEE B11C CE9B A02B C5DD 526F 01E8 564E E4B6 Engineering consulting with open source tools http://www.opennovation.com/ signature.asc Description: This is a digitally signed message part
Re: MPI implementations in squeeze
On 26/02/10 at 10:46 -0500, Adam C Powell IV wrote: On Thu, 2010-02-25 at 20:49 +0100, Lucas Nussbaum wrote: On 25/02/10 at 14:22 -0500, Adam C Powell IV wrote: On Thu, 2010-02-25 at 18:10 +0100, Lucas Nussbaum wrote: There is not much progress so far with respect to changing mpi-defaults to use MPICH2 instead of LAM on the architectures where Open MPI is not available yet. This needs a round of binNMUs. Marc Brockschmidt said he will look at the request to debian-release in the next few days, so this might resolve soon as well. Something to consider: this will break a lot of packages which use FORTRAN until 563705 is fixed, and then that will require mods to packages. I understand that bug as: if mpich2 or openmpi don't do the right thing when calling mpif77/mpif90, then symlinks are needed. Is there a proof that either of them doesn't do the right thing? Wouldn't it be more appropriate to fix them to do the right thing? (Those are honest questions -- I don't know anything about fortran) As discussed before (including in the bug), when there are mixed FORTRAN and C++ symbols, it's not clear whether to use mpif77/90 or mpic++. Also, it's a big convenience: a lot of packages make multiple executables and/or libraries, some of which use MPI and some don't. Pointing them to -lmpi -lmpi++ -lmpif77 for the MPI execs/lib directories seems easier than telling them to use mpicc and friends for some targets and gcc for others. I'm not sure I buy that, since mpicc friends also hide include paths, which are not handled with alternatives currently. Are you sure? % update-alternatives --display mpi mpi - auto mode link currently points to /usr/lib/openmpi/include /usr/lib/openmpi/include - priority 40 slave libmpi++.so: /usr/lib/openmpi/lib/libmpi_cxx.so [And a bunch of other slaves] It sounds more like a way to break packages by getting them linked with the wrong version of MPI. Do you know of packages doing that already? I've used this in most of my packages: petsc, hypre, libmesh, the new netgen and med-fichier under development (pending togl and updated hdf5 respectively), and salomé under development. Why would this break packages, if they just build-depend on mpi-default-dev? If the mpicc/mpif77 etc. alternatives work, why not the includes and libs as well? OK. Since it's harmless anyway, could you prepare and test patches for openmpi and mpich2? Since it would be a slave alternative, it doesn't require any alternatives transition. -- | Lucas Nussbaum | lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ | | jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100226192228.gb5...@xanadu.blop.info
Re: MPI implementations in squeeze
Hi all, this is a short status update on this topic. Am Montag, den 09.11.2009, 16:47 -0800 schrieb Nicholas Breen: * should we start filing wishlist bugs asking packagers not to build against MPICH (1) and LAM? I (finally) got around to file bugs against all packages build-depending on LAM and/or MPICH. To my surprise, some maintainers already did the transition, which I'm very happy about, and taking at as a sign that this is a welcomed decision. There were only 10 bugs left, not included the reverse deps of hdf5. You can get an overview via [1]. There is not much progress so far with respect to changing mpi-defaults to use MPICH2 instead of LAM on the architectures where Open MPI is not available yet. This needs a round of binNMUs. Marc Brockschmidt said he will look at the request to debian-release in the next few days, so this might resolve soon as well. I'd be very happy if we could get this transition done before the release, let's see if that works out! Best regards, Manuel [1] http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=old-mpi-eol;users=debian-scie...@lists.debian.org -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1267111959.10806.15.ca...@ce170155.zmb.uni-duisburg-essen.de
Re: MPI implementations in squeeze
On Thu, 2010-02-25 at 18:10 +0100, Lucas Nussbaum wrote: There is not much progress so far with respect to changing mpi-defaults to use MPICH2 instead of LAM on the architectures where Open MPI is not available yet. This needs a round of binNMUs. Marc Brockschmidt said he will look at the request to debian-release in the next few days, so this might resolve soon as well. Something to consider: this will break a lot of packages which use FORTRAN until 563705 is fixed, and then that will require mods to packages. I understand that bug as: if mpich2 or openmpi don't do the right thing when calling mpif77/mpif90, then symlinks are needed. Is there a proof that either of them doesn't do the right thing? Wouldn't it be more appropriate to fix them to do the right thing? (Those are honest questions -- I don't know anything about fortran) As discussed before (including in the bug), when there are mixed FORTRAN and C++ symbols, it's not clear whether to use mpif77/90 or mpic++. Also, it's a big convenience: a lot of packages make multiple executables and/or libraries, some of which use MPI and some don't. Pointing them to -lmpi -lmpi++ -lmpif77 for the MPI execs/lib directories seems easier than telling them to use mpicc and friends for some targets and gcc for others. And we have libmpi.so and libmpi++.so symlinks, why not libmpif77.so? :) -Adam -- GPG fingerprint: D54D 1AEE B11C CE9B A02B C5DD 526F 01E8 564E E4B6 Engineering consulting with open source tools http://www.opennovation.com/ signature.asc Description: This is a digitally signed message part
Re: MPI implementations in squeeze
On Mon, 2010-01-04 at 15:46 +0100, Manuel Prinz wrote: Am Montag, den 28.12.2009, 14:08 +0100 schrieb Lucas Nussbaum: OK, let's do that. We can always revisit this decision after the squeeze release, but it's important to have something sane for squeeze. Sylvestre, can you take the lead on that, as you proposed? The required steps, I think are: - change mpi-default to use OpenMPI + MPICH2 I took care of that. Changes are in the Git repository. I will do some additional testing and coordinate the upload and binNMUs with the release team. Should I CC d...@l.d.o or d-...@l.a.d.o? Hi Manuel and MPI keepers, were you planning to upload a new version of mpi-default that removes LAM? I think it's LAM that's screwing up the gerris mpi builds. Drew - identify the packages that currently use mpi-default need binNMUs or full source uploads, and file bugs with severity: serious - identify packages that don't use mpi-default yet, but use one of the MPI implementations directly, and file bugs: severity: serious if they use Mpich1 or LAM severity: normal if they use OpenMPI or MPICH2, asking them to switch to mpi-default This sounds good, I will start filing bugs during the next days. Best regards Manuel -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
Am Sonntag, den 03.01.2010, 23:26 +0100 schrieb Michael Banck: If one still wants to have a non-MPI package as well, is there a preferred suffix for the mpi package using the default? foo-mpi? There is no policy on that (yet) but your suggestion seems very reasonable. Best regards Manuel -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On Mon, Nov 30, 2009 at 10:11:49PM -0800, Nicholas Breen wrote: If you just want one MPI package, it should be enough to Build-Depend on mpi-default-dev and perform a build of your package with whatever flags are necessary -- which ideally should be something simple like configure --enable-mpi CC=mpicc. If one still wants to have a non-MPI package as well, is there a preferred suffix for the mpi package using the default? foo-mpi? Michael -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On Tue, 2009-12-01 at 11:31 +0100, Manuel Prinz wrote: Am Montag, den 30.11.2009, 22:11 -0800 schrieb Nicholas Breen: On Tue, Dec 01, 2009 at 03:49:20PM +1100, Drew Parsons wrote: From the point of view of a client package, what is now best practice for working with MPI? Is there a website/wiki/document explaining how to set up an MPI-using package, or is it just a matter of RTFM from mpi-default-dev? No particular documentation that I'm aware of, though it's a good thought But adding MPI support is usually as easy as Nicholas pointed out in his email. If one only wants to build an MPI version, build-depending on mpi-default-dev is best; if you want to build one package per MPI implementation, you have to do that yourself. Nicholas' gromacs package is a good example for that! I'll probably keep it simple and just build once against mpi-default-dev. If users have a strong need for other MPIs then they can file a bug request. Thanks all for the tips, hopefully won't be too tricky to get gerris mpi-packaged. Drew -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
Am Montag, den 30.11.2009, 22:11 -0800 schrieb Nicholas Breen: On Tue, Dec 01, 2009 at 03:49:20PM +1100, Drew Parsons wrote: From the point of view of a client package, what is now best practice for working with MPI? Is there a website/wiki/document explaining how to set up an MPI-using package, or is it just a matter of RTFM from mpi-default-dev? For context, gerris includes MPI support which is currently switched off. I would like to create an additional gerris-mpi package which has MPI switched on. No particular documentation that I'm aware of, though it's a good thought I'm also not aware of that and really might be reasonable to write such a document. It seems to me that there has been no or not much cooperation of maintainers of MPI packages, which is changing at the moment and works out well. I'm really happy about that! So chances for MPI guidelines are good. But adding MPI support is usually as easy as Nicholas pointed out in his email. If one only wants to build an MPI version, build-depending on mpi-default-dev is best; if you want to build one package per MPI implementation, you have to do that yourself. Nicholas' gromacs package is a good example for that! Best regards Manuel -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
Am Montag, den 09.11.2009, 16:47 -0800 schrieb Nicholas Breen: * should we start filing wishlist bugs asking packagers not to build against MPICH (1) and LAM? If noone objects by the end of the week, I will file the bugs then. The number of affected packages is really low and the severity wishlist, so I guess we do not need to go through the usual MBF hassle, or do we? Best regards Manuel -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On 01/12/09 at 11:33 +0100, Manuel Prinz wrote: Am Montag, den 09.11.2009, 16:47 -0800 schrieb Nicholas Breen: * should we start filing wishlist bugs asking packagers not to build against MPICH (1) and LAM? If noone objects by the end of the week, I will file the bugs then. The number of affected packages is really low and the severity wishlist, so I guess we do not need to go through the usual MBF hassle, or do we? So the plan would be to use OpenMPI on all architectures where it is supported, and MPICH2 on the other ones, using mpi-default? What do we do with LAM and mpich1? Target their removal for squeeze? While I would be fine with that, there are other possible plans: (2) use MPICH2 as default everywhere (it is supported on all release archs). Clearly, the package didn't receive a lot of testing yet, but this is likely to be only a temporary problem. (3) encourage people to provide packages for both MPICH2 and OpenMPI (where possible, or only MPICH2 if OpenMPI not possible). -- | Lucas Nussbaum | lu...@lucas-nussbaum.net http://www.lucas-nussbaum.net/ | | jabber: lu...@nussbaum.fr GPG: 1024D/023B3F4F | -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
Le mardi 01 décembre 2009 à 21:20 +0100, Lucas Nussbaum a écrit : On 01/12/09 at 11:33 +0100, Manuel Prinz wrote: Am Montag, den 09.11.2009, 16:47 -0800 schrieb Nicholas Breen: * should we start filing wishlist bugs asking packagers not to build against MPICH (1) and LAM? If noone objects by the end of the week, I will file the bugs then. The number of affected packages is really low and the severity wishlist, so I guess we do not need to go through the usual MBF hassle, or do we? So the plan would be to use OpenMPI on all architectures where it is supported, and MPICH2 on the other ones, using mpi-default? To me, this would be the best solution. We could do the third proposal (build for both MPICH2 OpenMPI) but, at least to me, it complexifies the build process to have to handle many different MPI implementations for a low gain for the lambda user. Sylvestre -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On Tue, Dec 01, 2009 at 11:31:02AM +0100, Manuel Prinz wrote: Am Montag, den 30.11.2009, 22:11 -0800 schrieb Nicholas Breen: On Tue, Dec 01, 2009 at 03:49:20PM +1100, Drew Parsons wrote: From the point of view of a client package, what is now best practice for working with MPI? Is there a website/wiki/document explaining how to set up an MPI-using package, or is it just a matter of RTFM from mpi-default-dev? For context, gerris includes MPI support which is currently switched off. I would like to create an additional gerris-mpi package which has MPI switched on. No particular documentation that I'm aware of, though it's a good thought I'm also not aware of that and really might be reasonable to write such a document. It seems to me that there has been no or not much cooperation of maintainers of MPI packages, which is changing at the moment and works out well. I'm really happy about that! So chances for MPI guidelines are good. But adding MPI support is usually as easy as Nicholas pointed out in his email. If one only wants to build an MPI version, build-depending on mpi-default-dev is best; if you want to build one package per MPI implementation, you have to do that yourself. Nicholas' gromacs package is a good example for that! Nice to see things are going on. A policy document would be nice indeed, with possibly use cases. Note that currently also hdf5 builds for different MPI flavors with some specific hacks. -- Francesco P. Lovergine -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On Wed, 2009-12-02 at 00:48 +0100, Manuel Prinz wrote: Am Dienstag, den 01.12.2009, 21:20 +0100 schrieb Lucas Nussbaum: While I would be fine with that, there are other possible plans: (2) use MPICH2 as default everywhere (it is supported on all release archs). Clearly, the package didn't receive a lot of testing yet, but this is likely to be only a temporary problem. I do not care that much. We chose Open MPI because it was the most mature implementation at that time. I do not know much about MPICH2, and am open to use MPICH2 as the default. I'd like to go with Open MPI, but as one of the Open MPI maintainers in Debian, my view is biased. As a data point from the real world, the Australian Supercomputing Facility currently offers Open MPI but not MPICH2 [1]. Mind you the clusters are just x86_64 so no arcane architecture. They will install new software if users request and justify the need, so they might potentially build MPICH2 if someone asked, though I imagine they'd first strongly encourage the user to try Open MPI first. [1] http://nf.nci.org.au/facilities/software/software.php?software=MPIsite=ANUfrom_site=ANU -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
I understand the final state of the modern Debian infrastructure for MPI is now in place (modulo the questions around deprecating MPICH and LAM). That is, the disruptive Open MPI transition is now complete, correct? From the point of view of a client package, what is now best practice for working with MPI? Is there a website/wiki/document explaining how to set up an MPI-using package, or is it just a matter of RTFM from mpi-default-dev? For context, gerris includes MPI support which is currently switched off. I would like to create an additional gerris-mpi package which has MPI switched on. Thanks, Drew On Mon, 2009-11-09 at 16:47 -0800, Nicholas Breen wrote: With the recent upload of MPICH2, we now have four separate MPI implementations in the archive: two with active upstreams and maintainers (MPICH2, OpenMPI) and two on terminal life support (MPICH, LAM). There's been some preliminary discussion before about dropping the latter two [1], though legacy support is an issue as well [2]. Currently, the MPI situation is fairly messy: 18 source packages depend on mpi-default-dev (OpenMPI or LAM, depending on architecture), but another 18 depend on various permutations of the implementations directly [3], with no particular consistency. From a package maintainer standpoint, I'd like to see us start reducing the number of implementations to build client packages against, even if we maintain all the MPI implementations themselves (perhaps moving them to the 'oldlibs' section). What I'm wondering is: * in mpi-defaults, should MPICH2 replace LAM for architectures not supported by OpenMPI? * should we start filing wishlist bugs asking packagers not to build against MPICH (1) and LAM? * is it too late in the release cycle to propose this as a release goal? should squeeze+1 be the target instead? squeeze+2? This is orthogonal to solving #552429, which will need action before the squeeze release in any case. -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On Tue, Dec 01, 2009 at 03:49:20PM +1100, Drew Parsons wrote: I understand the final state of the modern Debian infrastructure for MPI is now in place (modulo the questions around deprecating MPICH and LAM). That is, the disruptive Open MPI transition is now complete, correct? Yes, so far as I know. There might be the occasional need for a binNMU as MPICH2 settles in and if mpi-default-dev switches to MPICH2 instead of LAM as the backup, but those shouldn't interfere with testing migration. From the point of view of a client package, what is now best practice for working with MPI? Is there a website/wiki/document explaining how to set up an MPI-using package, or is it just a matter of RTFM from mpi-default-dev? For context, gerris includes MPI support which is currently switched off. I would like to create an additional gerris-mpi package which has MPI switched on. No particular documentation that I'm aware of, though it's a good thought If you just want one MPI package, it should be enough to Build-Depend on mpi-default-dev and perform a build of your package with whatever flags are necessary -- which ideally should be something simple like configure --enable-mpi CC=mpicc. -- Nicholas Breen nbr...@ofb.net -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
Gee, I should stop posting past a certain hour of the day… ;) On Tue, 2009-11-10 at 18:08 -0500, Adam C Powell IV wrote: The .so alternatives symlinks only require that the libraries be API-compatible, which they are (or if not, it's a bug, since they're supposed to follow the MPI standard). That's why these links, and plain .so links in general, are in the -dev packages, not the shlib packages. It should be possible to compile any MPI program's source code against any implementation by linking -lmpi -lmpi++ etc. Then the resulting binary, shlib etc. includes the soname specific to the library it linked with, e.g. libopen-rte.so.0 . So if it's in a Debian package, the resulting binary depends on the ABI-correct library package, e.g. libopenmpi1 . You're of course right. I somehow mixed several things up and got quite confused. Open MPI indeed does use libmpi.so.0 for the name, all other implementations don't. I probably mixed libmpi.so and libmpi.so.0 here, among other things. (Though this is not really an issue, it might be nice to build it as libopenmpi.so.0 or something.) If this still doesn't make sense, the libtool online documentation is pretty clear. Now, awake and conscious, it makes perfect sense. The libtool doc is a good read, nevertheless. Sorry for the noise! You're nitpicking, as you called it, was very welcome! :) Best regards Manuel -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On Tue, Nov 10, 2009 at 09:19:11AM -0500, Adam C Powell IV wrote: On Mon, 2009-11-09 at 16:47 -0800, Nicholas Breen wrote: * is it too late in the release cycle to propose this as a release goal? should squeeze+1 be the target instead? squeeze+2? I think it's too late for squeeze, but squeeze+1 should definitely be doable. How many packages are affected? Squeeze freeze is several months away still. I am not sure about release-goal freeze, but even then, it could be considered an inofficial release goal among debian-science at least. Michael -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
On Tue, 2009-11-10 at 16:27 +0100, Michael Banck wrote: On Tue, Nov 10, 2009 at 09:19:11AM -0500, Adam C Powell IV wrote: On Mon, 2009-11-09 at 16:47 -0800, Nicholas Breen wrote: * is it too late in the release cycle to propose this as a release goal? should squeeze+1 be the target instead? squeeze+2? I think it's too late for squeeze, but squeeze+1 should definitely be doable. How many packages are affected? I think your previous email best answered that question. Squeeze freeze is several months away still. I am not sure about release-goal freeze, but even then, it could be considered an inofficial release goal among debian-science at least. Oh, right. For some reason my mental calendar still has late '09 as the release goal date. I'm all for it then, at least unofficially. The other issue would be rebuilding all of the mpi-defaults packages at least on lam architectures when/if we decide to switch to mpich2. Count my vote in support of such a switch, which might be worth as much as $0.03-.04 since I wrote the original mpi-defaults. :-) -Adam -- GPG fingerprint: D54D 1AEE B11C CE9B A02B C5DD 526F 01E8 564E E4B6 Engineering consulting with open source tools http://www.opennovation.com/ signature.asc Description: This is a digitally signed message part
Re: MPI implementations in squeeze
Am Dienstag, den 10.11.2009, 09:19 -0500 schrieb Adam C Powell IV: * in mpi-defaults, should MPICH2 replace LAM for architectures not supported by OpenMPI? I think that would make a lot of sense since LAM is end-of-life. Filed as bug #555653, so we won't forget about that. We should wait with the fix when the dust has settled. (Both here and with the big transitions going on at the moment.) Best regards Manuel -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: MPI implementations in squeeze
I generally agree, just a quick nit-pick/clarification: On Tue, 2009-11-10 at 20:56 +0100, Manuel Prinz wrote: * Alternatives need to be fixed. Besides what the bugs that Nicholas referenced say, there are two other issues with those: First, the priorities do not match the reality (Open MPI being the default / recommended implementation to use), and second, that mpi.so is also in the alternatives, which is wrong in every respect and has bother me for a very long time now. The implementations are not ABI-compatible in any way, and we really need to get rid of that. The .so alternatives symlinks only require that the libraries be API-compatible, which they are (or if not, it's a bug, since they're supposed to follow the MPI standard). That's why these links, and plain .so links in general, are in the -dev packages, not the shlib packages. It should be possible to compile any MPI program's source code against any implementation by linking -lmpi -lmpi++ etc. Then the resulting binary, shlib etc. includes the soname specific to the library it linked with, e.g. libopen-rte.so.0 . So if it's in a Debian package, the resulting binary depends on the ABI-correct library package, e.g. libopenmpi1 . If this still doesn't make sense, the libtool online documentation is pretty clear. -Adam -- GPG fingerprint: D54D 1AEE B11C CE9B A02B C5DD 526F 01E8 564E E4B6 Engineering consulting with open source tools http://www.opennovation.com/ signature.asc Description: This is a digitally signed message part
Re: MPI implementations in squeeze
On Mon, Nov 09, 2009 at 04:47:27PM -0800, Nicholas Breen wrote: [3] build-deps on: lam4-devlibmpich1.0-dev libopenmpi-dev ... tree-puzzle yes no no Just uploaded tree-puzzle builded agianst libopenmpi-dev. Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-science-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
