Bug#658281: Bug#657949: Cannot install libhdf5-mpi-dev and libnetcdf-dev
Hi Francesco, Do you recommend that we build the next NetCDF from 4.1.1 or should we use the 4.1.3 from experimental as the base? Regards Alastair On 2012-02-07 13:17, Francesco P. Lovergine wrote: On Tue, Feb 07, 2012 at 09:28:00AM +, Alastair McKinstry wrote: On 2012-02-02 01:43, Steve M. Robbins wrote: Hi, I'd like to contribute towards a solution for this. I'm forwarding to debian-devel to get some others' ideas. Naively, I don't understand why netcdf can't offer multiple variants, just as hdf5 does. Or, at least, one package libnetcdf-mpi-dev that links with the default MPI implementation. I am not involved in the netcdf. You should report a bug on this package. I'm prepared to do so, but I'd first like to get agreement that netcdf is where the problem lies. Netcdf maintainers, please chime in! I think we can no longer live in the status quo (see all the blockers of #631019), so something has to give. Even if it is painful, perhaps Debian could pioneer something and pass patches back to upstream? Thoughts? -Steve As of now, I have several packages (eg ADIOS, CDO) that used to build against netcdf and libhdf5-mpi-dev that don't. Without fixes to netCDF (I appreciate what Francesco says about netcdf upstream not giving the libraries proper names), there needs to be a regression: either the packages build with netcdf but no MPI, or MPI but no netcdf. The problem is the following: with latest update to hdf5, the chain of dependencies changed, so that now libnetcdf6 depends on the pure serial version of hdf5, while the previous one depended on serial or parallel: Version: 1:4.1.1-6+b1 Depends: libc6 (= 2.7), libcurl3-gnutls (= 7.16.2), libgcc1 (= 1:4.1.1), libgfortran3 (= 4.3), libhdf5-7 (= 1.8.7), libquadmath0 (= 4.6), libstdc++6 (= 4.4.0) Version: 1:4.1.1-6 Depends: libc6 (= 2.7), libcurl3-gnutls (= 7.16.2-1), libgcc1 (= 1:4.1.1), libgfortran3 (= 4.3), libhdf5-serial-1.8.4 | libhdf5-1.8.4, libquadmath0 (= 4.6), libstdc++6 (= 4.4.0) So at least at packaging level, that should be fixed to follow the previous criteria. That said, indeed NetCDF provides nc_create_par and nc_open_par in both serial and parallel versions, but needs to be built with --enable-parallel to take advantage of parallel I/O in HDF5, else it works in pure serial mode. -- Alastair McKinstry , alast...@sceal.ie , mckins...@debian.org http://blog.sceal.ie Anyone who believes exponential growth can go on forever in a finite world is either a madman or an economist - Kenneth Boulter, Economist. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#658281: Bug#657949: Cannot install libhdf5-mpi-dev and libnetcdf-dev
On Wed, Feb 08, 2012 at 02:32:07PM +, Alastair McKinstry wrote: Hi Francesco, Do you recommend that we build the next NetCDF from 4.1.1 or should we use the 4.1.3 from experimental as the base? Regards Alastair AFAIK Sylvestre is going to reset the dependencies chain in hdf5 to avoid that kind of problem. About 4.1.3 in experimental, it still needs a bit of work, and I'm going to split in separate packages current netcdf 4.1.1 before, in order to have a decent organization of all solibs to have a smooth migration to 4.1.3. You have free access to the git repository, so a branch can be prepared for having a parallel flavor too. -- Francesco P. Lovergine -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#658281: Bug#657949: Cannot install libhdf5-mpi-dev and libnetcdf-dev
On 2012-02-02 01:43, Steve M. Robbins wrote:
Hi,
I'd like to contribute towards a solution for this. I'm forwarding to
debian-devel to get some others' ideas.
Naively, I don't understand why netcdf can't offer multiple variants,
just as hdf5 does. Or, at least, one package libnetcdf-mpi-dev that
links with the default MPI implementation.
I am not involved in the netcdf. You should report a bug on this
package.
I'm prepared to do so, but I'd first like to get agreement that
netcdf is where the problem lies. Netcdf maintainers, please
chime in!
I think we can no longer live in the status quo (see all the blockers
of #631019), so something has to give. Even if it is painful, perhaps
Debian could pioneer something and pass patches back to upstream?
Thoughts?
-Steve
As of now, I have several packages (eg ADIOS, CDO) that used to build
against netcdf and libhdf5-mpi-dev
that don't. Without fixes to netCDF (I appreciate what Francesco says
about netcdf upstream
not giving the libraries proper names), there needs to be a regression:
either the packages
build with netcdf but no MPI, or MPI but no netcdf.
I could split the package, and provide two versions, eg. adios-mpi and
adios-serial, but this to me is going backwards.
In an increasingly parallel world, we need binaries that will run in
parallel when its available. eg. detect
an MPI environmnent, and if so, use the parallel version of libraries.
Do others think this is the way to go, or that way lies madness?
That is, can we work out the details of what would be needed for
automatic parallelism,
what we can do and what upstream changes might be needed?
e.g. we might add some shim code at the start of programs that do:
if (mpi_detected || ($ENV{NETCDF_SERIAL_WANTED})
dlopen(netcdf_mpi_version)
else
dlopen(netcdf_serial_version)
(Some netCDF programs, even running under MPI, might run in serial mode
in order to use
features such as compression that don't work in parallel netcdf).
We need to come up with:
(1) A bigger picture of where we want Debian to go (may involve upstream
changes)
(2) A plan for that we can do for the next release.
Regards
Alastair
--
Alastair McKinstry , alast...@sceal.ie , mckins...@debian.org
http://blog.sceal.ie
Anyone who believes exponential growth can go on forever in a finite world
is either a madman or an economist - Kenneth Boulter, Economist.
--
To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#658281: Bug#657949: Cannot install libhdf5-mpi-dev and libnetcdf-dev
On Tue, Feb 07, 2012 at 09:28:00AM +, Alastair McKinstry wrote: On 2012-02-02 01:43, Steve M. Robbins wrote: Hi, I'd like to contribute towards a solution for this. I'm forwarding to debian-devel to get some others' ideas. Naively, I don't understand why netcdf can't offer multiple variants, just as hdf5 does. Or, at least, one package libnetcdf-mpi-dev that links with the default MPI implementation. I am not involved in the netcdf. You should report a bug on this package. I'm prepared to do so, but I'd first like to get agreement that netcdf is where the problem lies. Netcdf maintainers, please chime in! I think we can no longer live in the status quo (see all the blockers of #631019), so something has to give. Even if it is painful, perhaps Debian could pioneer something and pass patches back to upstream? Thoughts? -Steve As of now, I have several packages (eg ADIOS, CDO) that used to build against netcdf and libhdf5-mpi-dev that don't. Without fixes to netCDF (I appreciate what Francesco says about netcdf upstream not giving the libraries proper names), there needs to be a regression: either the packages build with netcdf but no MPI, or MPI but no netcdf. The problem is the following: with latest update to hdf5, the chain of dependencies changed, so that now libnetcdf6 depends on the pure serial version of hdf5, while the previous one depended on serial or parallel: Version: 1:4.1.1-6+b1 Depends: libc6 (= 2.7), libcurl3-gnutls (= 7.16.2), libgcc1 (= 1:4.1.1), libgfortran3 (= 4.3), libhdf5-7 (= 1.8.7), libquadmath0 (= 4.6), libstdc++6 (= 4.4.0) Version: 1:4.1.1-6 Depends: libc6 (= 2.7), libcurl3-gnutls (= 7.16.2-1), libgcc1 (= 1:4.1.1), libgfortran3 (= 4.3), libhdf5-serial-1.8.4 | libhdf5-1.8.4, libquadmath0 (= 4.6), libstdc++6 (= 4.4.0) So at least at packaging level, that should be fixed to follow the previous criteria. That said, indeed NetCDF provides nc_create_par and nc_open_par in both serial and parallel versions, but needs to be built with --enable-parallel to take advantage of parallel I/O in HDF5, else it works in pure serial mode. -- Francesco P. Lovergine -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
