[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Matt Satish On Wed, 10 Apr 2013, Barry Smith wrote: Shouldn't this be fixed by now in PETSc-dev? Begin forwarded message: From: Satish Balay balay at mcs.anl.gov Subject: Re: [petsc-users] cannot find 'libz.a' when configuring Date: April 10, 2013 5:21:38 PM CDT To: PETSc users list petsc-users at mcs.anl.gov Reply-To: PETSc users list petsc-users at mcs.anl.gov try using configure option LIBS=-L/opt/zlib-1.2.7/lib -lz' Satish On Wed, 10 Apr 2013, Seungbum Koo wrote: Hi. I tried to add hdf5 when configuring. HDF5-1.8.9 is currently installed with szip-2.1 and zlib-1.2.7. It stops with message *** UNABLE to CONFIGURE with GIVEN OPTIONS(see configure.log for details): --- Compression library [libz.a or equivalent] not found *** What I don't understand is that 'libz.a' file exists in '/opt/zlib-1.2.7/lib'. What should I do? Seungbum -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- next part -- An HTML attachment was scrubbed... URL: http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130411/c3976651/attachment.html
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
Entire package time :-( On Apr 11, 2013, at 7:02 AM, Matthew Knepley knepley at gmail.com wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Matt Satish On Wed, 10 Apr 2013, Barry Smith wrote: Shouldn't this be fixed by now in PETSc-dev? Begin forwarded message: From: Satish Balay balay at mcs.anl.gov Subject: Re: [petsc-users] cannot find 'libz.a' when configuring Date: April 10, 2013 5:21:38 PM CDT To: PETSc users list petsc-users at mcs.anl.gov Reply-To: PETSc users list petsc-users at mcs.anl.gov try using configure option LIBS=-L/opt/zlib-1.2.7/lib -lz' Satish On Wed, 10 Apr 2013, Seungbum Koo wrote: Hi. I tried to add hdf5 when configuring. HDF5-1.8.9 is currently installed with szip-2.1 and zlib-1.2.7. It stops with message *** UNABLE to CONFIGURE with GIVEN OPTIONS(see configure.log for details): --- Compression library [libz.a or equivalent] not found *** What I don't understand is that 'libz.a' file exists in '/opt/zlib-1.2.7/lib'. What should I do? Seungbum -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, 11 Apr 2013, Matthew Knepley wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Currently we handle 'mandatory' dependencies properly - and optional depencencies for --download-pakcage somewhat - but not for --with-package-dir For hdf5 - handling just zlib.py is not sufficient. Do we add szip.py, zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use these as optionaly when testing for user provided --with-hdf5? Which package currently implements this usage [to check for this optionaly dependency - and give the correct error]? i.e In this case - when the user specified --with-hdf5-dir option. We had to print an error.: * please provide --with-zlib-dir and --with-szip-dir options. [For a different user - with the same usage - there would be no error] For netcdf above one would have kerbros.py, curl.py and perhaps more? Satish
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Currently we handle 'mandatory' dependencies properly - and optional depencencies for --download-pakcage somewhat - but not for --with-package-dir For hdf5 - handling just zlib.py is not sufficient. Do we add szip.py, zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use these as optionaly when testing for user provided --with-hdf5? Which package currently implements this usage [to check for this optionaly dependency - and give the correct error]? i.e In this case - when the user specified --with-hdf5-dir option. We had to print an error.: * please provide --with-zlib-dir and --with-szip-dir options. [For a different user - with the same usage - there would be no error] For netcdf above one would have kerbros.py, curl.py and perhaps more? We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Matt Satish -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- next part -- An HTML attachment was scrubbed... URL: http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130411/70648a1f/attachment.html
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Currently we handle 'mandatory' dependencies properly - and optional depencencies for --download-pakcage somewhat - but not for --with-package-dir For hdf5 - handling just zlib.py is not sufficient. Do we add szip.py, zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use these as optionaly when testing for user provided --with-hdf5? Which package currently implements this usage [to check for this optionaly dependency - and give the correct error]? i.e In this case - when the user specified --with-hdf5-dir option. We had to print an error.: * please provide --with-zlib-dir and --with-szip-dir options. [For a different user - with the same usage - there would be no error] For netcdf above one would have kerbros.py, curl.py and perhaps more? We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This thread started becasue one expected --with-hdf5-dir to be sufficient - and its not. current workarround [becuase of self.needsCompression] is to specify zlib with LIBS otpions - and all other dependencies with --with-hdf5-lib. It works - but if were were satisfactory - this whole thread on petsc-dev wouldn't arise. Satish
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, Apr 11, 2013 at 9:38 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Currently we handle 'mandatory' dependencies properly - and optional depencencies for --download-pakcage somewhat - but not for --with-package-dir For hdf5 - handling just zlib.py is not sufficient. Do we add szip.py, zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use these as optionaly when testing for user provided --with-hdf5? Which package currently implements this usage [to check for this optionaly dependency - and give the correct error]? i.e In this case - when the user specified --with-hdf5-dir option. We had to print an error.: * please provide --with-zlib-dir and --with-szip-dir options. [For a different user - with the same usage - there would be no error] For netcdf above one would have kerbros.py, curl.py and perhaps more? We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). Matt This thread started becasue one expected --with-hdf5-dir to be sufficient - and its not. current workarround [becuase of self.needsCompression] is to specify zlib with LIBS otpions - and all other dependencies with --with-hdf5-lib. It works - but if were were satisfactory - this whole thread on petsc-dev wouldn't arise. Satish -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- next part -- An HTML attachment was scrubbed... URL: http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130411/4adbe000/attachment.html
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:38 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Currently we handle 'mandatory' dependencies properly - and optional depencencies for --download-pakcage somewhat - but not for --with-package-dir For hdf5 - handling just zlib.py is not sufficient. Do we add szip.py, zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use these as optionaly when testing for user provided --with-hdf5? Which package currently implements this usage [to check for this optionaly dependency - and give the correct error]? i.e In this case - when the user specified --with-hdf5-dir option. We had to print an error.: * please provide --with-zlib-dir and --with-szip-dir options. [For a different user - with the same usage - there would be no error] For netcdf above one would have kerbros.py, curl.py and perhaps more? We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Satish Matt This thread started becasue one expected --with-hdf5-dir to be sufficient - and its not. current workarround [becuase of self.needsCompression] is to specify zlib with LIBS otpions - and all other dependencies with --with-hdf5-lib. It works - but if were were satisfactory - this whole thread on petsc-dev wouldn't arise. Satish
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, Apr 11, 2013 at 9:48 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:38 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Currently we handle 'mandatory' dependencies properly - and optional depencencies for --download-pakcage somewhat - but not for --with-package-dir For hdf5 - handling just zlib.py is not sufficient. Do we add szip.py, zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use these as optionaly when testing for user provided --with-hdf5? Which package currently implements this usage [to check for this optionaly dependency - and give the correct error]? i.e In this case - when the user specified --with-hdf5-dir option. We had to print an error.: * please provide --with-zlib-dir and --with-szip-dir options. [For a different user - with the same usage - there would be no error] For netcdf above one would have kerbros.py, curl.py and perhaps more? We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. Matt Satish Matt This thread started becasue one expected --with-hdf5-dir to be sufficient - and its not. current workarround [becuase of self.needsCompression] is to specify zlib with LIBS otpions - and all other dependencies with --with-hdf5-lib. It works - but if were were satisfactory - this whole thread on petsc-dev wouldn't arise. Satish -- What most experimenters take for granted before they begin their experiments
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:48 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:38 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 9:17 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Wed, Apr 10, 2013 at 7:53 PM, Satish Balay balay at mcs.anl.gov wrote: How do we handle these packages with complicated dependencies [they are complicated as petsc dependencies?] In this report hdf5 was built with szip-2.1 and zlib-1.2.7. So how does petsc configure automatically detect this? And I've recently used the following for hdf5/necddf5 on fusion [with the extra implicit depenceny of hdf5 on zlib, and netcdf5 on hdf5 from within configure] '--with-hdf5-lib=-L/soft/hdf5/1.8.6-parallel/lib -lhdf5_hl -lhdf5 -lgpfs', '--with-netcdf-lib=-L/soft/netcdf/4.1.1-parallel/lib -lnetcdff -lnetcdf -L/usr/kerberos/lib64 -lcurl -ldl -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lidn -lssl -lcrypto -lz', Yeah - things work if the user knows the dependencies and the link command for those dependencies - and specify it to petsc configure as above. But I'm not sure how to autodetect this. Also currently -lz is handled in package.py with 'self.needsCompression' similar to 'self.needsMath' with the detection of -lz in libraires.py:checkCompression() - but one can't specify a --with-zlib-lib option this way. I guess this part can be fixed by migrating it a standalone package z.py. [And somehow handle the optional part of this dependency for hdf5] We already have a mechanism for this. Lots of packages depend on other packages. This is just screwed up in the case of libz because someone (maybe me) did not want to write an entire package file for it, and instead copped out with the needsCompression. Currently we handle 'mandatory' dependencies properly - and optional depencencies for --download-pakcage somewhat - but not for --with-package-dir For hdf5 - handling just zlib.py is not sufficient. Do we add szip.py, zlib.py, gpfs.py [and perhaps more] for hdf5? And somehow use these as optionaly when testing for user provided --with-hdf5? Which package currently implements this usage [to check for this optionaly dependency - and give the correct error]? i.e In this case - when the user specified --with-hdf5-dir option. We had to print an error.: * please provide --with-zlib-dir and --with-szip-dir options. [For a different user - with the same usage - there would be no error] For netcdf above one would have kerbros.py, curl.py and perhaps more? We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. In this case - why bother with libz.py and szip.py? Why not tell the user to specify hdf5 completely with --with-hdf5-include and --with-hdf5-lib options? Just printing the link errors on the screen might be sufficient to clue in the user. Currently its hidden inside configure. [And
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, 11 Apr 2013, Satish Balay wrote: We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. In this case - why bother with libz.py and szip.py? Why not tell the user to specify hdf5 completely with --with-hdf5-include and --with-hdf5-lib options? Just printing the link errors on the screen might be sufficient to clue in the user. Currently its hidden inside configure. [And its not clear which of the 10 tests configure did for this case should be printed on the screen as the appropriate message] I might have been the one that advocated self.needsCompression approach thats currently in configure. This is currently used only by hdf5.py. My current thought is to strip it out - and just add it to hdf5.py as: self.liblist = [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] This would be equivalent to the current functionality - and perhaps what was there previously - before we wanted better error messages and detection of zlib. Since this is not possible anyway why bother with the additional complexity? Satish
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Satish Balay wrote: We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. In this case - why bother with libz.py and szip.py? Why not tell the user to specify hdf5 completely with --with-hdf5-include and --with-hdf5-lib options? Just printing the link errors on the screen might be sufficient to clue in the user. Currently its hidden inside configure. [And its not clear which of the 10 tests configure did for this case should be printed on the screen as the appropriate message] I might have been the one that advocated self.needsCompression approach thats currently in configure. This is currently used only by hdf5.py. My current thought is to strip it out - and just add it to hdf5.py as: self.liblist = [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] This would be equivalent to the current functionality - and perhaps what was there previously - before we wanted better error messages and detection of zlib. Since this is not possible anyway why bother with the additional complexity? What is not possible? Detecting zlib? Of course this is possible, and we do it. It is not possible to know without testing what crazy link line is required by HDF5, at least I don't know how to do that. This is no way affects the utility of testing for zlib. Matt Satish -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener -- next part -- An HTML attachment was scrubbed... URL: http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130411/be579e6a/attachment-0001.html
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Satish Balay wrote: We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. In this case - why bother with libz.py and szip.py? Why not tell the user to specify hdf5 completely with --with-hdf5-include and --with-hdf5-lib options? Just printing the link errors on the screen might be sufficient to clue in the user. Currently its hidden inside configure. [And its not clear which of the 10 tests configure did for this case should be printed on the screen as the appropriate message] I might have been the one that advocated self.needsCompression approach thats currently in configure. This is currently used only by hdf5.py. My current thought is to strip it out - and just add it to hdf5.py as: self.liblist = [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] This would be equivalent to the current functionality - and perhaps what was there previously - before we wanted better error messages and detection of zlib. Since this is not possible anyway why bother with the additional complexity? What is not possible? Detecting zlib? Of course this is possible, and we do it. The primary thing we need is detecting hdf5 provided by user requires zlib [and or szip]. This is not poissible. And current code can't detect zlib anyway [outside system locations] so we need libz.py to detect it better. But whats the point of it? [detecting zlib or szip via --with-zlib-dir and --with-szip-dir options?] When use has a mesages: --with-hdf5=foo/bar provides does not work I'd rather respond with: Specify hdf5 and all its dependencies correctly with --with-hdf5-lib,--with-hdf5-include options. It is not possible to know without testing what crazy link line is required by HDF5, at least I don't know how to do that. yes This is no way affects the utility of testing for zlib. But whats the point of testing for zlib - if not for hdf5? And if we can't relate them automatically. Satish
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, 11 Apr 2013, Satish Balay wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Satish Balay wrote: We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. In this case - why bother with libz.py and szip.py? Why not tell the user to specify hdf5 completely with --with-hdf5-include and --with-hdf5-lib options? Just printing the link errors on the screen might be sufficient to clue in the user. Currently its hidden inside configure. [And its not clear which of the 10 tests configure did for this case should be printed on the screen as the appropriate message] I might have been the one that advocated self.needsCompression approach thats currently in configure. This is currently used only by hdf5.py. My current thought is to strip it out - and just add it to hdf5.py as: self.liblist = [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] This would be equivalent to the current functionality - and perhaps what was there previously - before we wanted better error messages and detection of zlib. Since this is not possible anyway why bother with the additional complexity? What is not possible? Detecting zlib? Of course this is possible, and we do it. The primary thing we need is detecting hdf5 provided by user requires zlib [and or szip]. This is not poissible. And current code can't detect zlib anyway [outside system locations] so we need libz.py to detect it better. But whats the point of it? [detecting zlib or szip via --with-zlib-dir and --with-szip-dir options?] When use has a mesages: --with-hdf5=foo/bar provides does not work I'd rather respond with: Specify hdf5 and all its dependencies correctly with --with-hdf5-lib,--with-hdf5-include options. It is not possible to know without testing what crazy link line is required by HDF5, at least I don't know how to do that. yes This is no way affects the utility of testing for zlib. But whats the point of testing for zlib - if not for hdf5? And if we can't relate them automatically. I'll concede that having zlib.py szip.py will be useful if you need them for --download-hdf5 [either as mandatory dependencies or optonal dependencies]. Currently --download-hdf5 isn't built with either. My contention here is - depencency management stuff in configure doesn't help for the optional dependencies of user installed packages that are specified to petsc configure with the option --with-package-dir Satish
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
On Thu, Apr 11, 2013 at 10:39 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Satish Balay wrote: On Thu, 11 Apr 2013, Matthew Knepley wrote: On Thu, Apr 11, 2013 at 10:12 AM, Satish Balay balay at mcs.anl.gov wrote: On Thu, 11 Apr 2013, Satish Balay wrote: We have had this same discussion a million times, and we always eventually get to the same answer and forget. Do we need our own error message for petsc-dev? We cannot possibly guess everything that can go wrong. What we need is a system where the user can specify what he actually wants and we check it. So, zlib and everything else are optional dependencies for hdf5. If something is not specified, it fails, and the user goes back and specifies the right thing. Sure - but specify how? specify everything with --with-hdf5-lib,--with-hdf5-include options? Or somehow he knows that he has to specify --with-zlib-include/-with-zlib-lib options? This. It would be nice if HDF5 would tell us about its dependencies, but if it does not, then the user has to, and this is our system (packages). So we would start adding zlib.py, szip.py and manymore [and support --download-zlib --download-szip]. And if user gets the error: --with-hdf5-dir=/foo/bar does not work. So he/she send us configure.log Then we look at it and respond saying: you need to specify --with-zlib-dir --with-szip-dir options aswell. Does this look right? Yes, that is what I think makes sense. We can always try to add another layer that 'greps' for libz in the hdf5 error message and guesses that the user forgot it, but hopefully before we get to it, a sane storage library appears, or HDF5 gets its act together. In this case - why bother with libz.py and szip.py? Why not tell the user to specify hdf5 completely with --with-hdf5-include and --with-hdf5-lib options? Just printing the link errors on the screen might be sufficient to clue in the user. Currently its hidden inside configure. [And its not clear which of the 10 tests configure did for this case should be printed on the screen as the appropriate message] I might have been the one that advocated self.needsCompression approach thats currently in configure. This is currently used only by hdf5.py. My current thought is to strip it out - and just add it to hdf5.py as: self.liblist = [['libhdf5.a','libhdf5_hl.a'],'['libhdf5.a','libhdf5_hl.a','libz.a'],['libhdf5.a','libhdf5_hl.a','zlib.lib']] This would be equivalent to the current functionality - and perhaps what was there previously - before we wanted better error messages and detection of zlib. Since this is not possible anyway why bother with the additional complexity? What is not possible? Detecting zlib? Of course this is possible, and we do it. The primary thing we need is detecting hdf5 provided by user requires zlib [and or szip]. This is not poissible. And current code can't detect zlib anyway [outside system locations] so we need libz.py to detect it better. But whats the point of it? [detecting zlib or szip via --with-zlib-dir and --with-szip-dir options?] When use has a mesages: --with-hdf5=foo/bar provides does not work I'd rather respond with: Specify hdf5 and all its dependencies correctly with --with-hdf5-lib,--with-hdf5-include options. It is not possible to know without testing what crazy link line is required by HDF5, at least I don't know how to do that. yes This is no way affects the utility of testing for zlib. But whats the point of testing for zlib - if not for hdf5? And if we can't relate them automatically. I'll concede that having zlib.py szip.py will be useful if you need them for --download-hdf5 [either as mandatory dependencies or optonal dependencies]. Currently --download-hdf5 isn't built with either. My contention here is - depencency management stuff in configure doesn't help for the optional dependencies of user installed packages that are specified to petsc configure with the option --with-package-dir Okay, I would like to distinguish between good behavior and HDF5. If we can ask whether the package enables it, we are fine. If there is a simple test we can do, we are fine. What can we do for HDF5? Matt Satish -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
[petsc-dev] Fwd: [petsc-users] cannot find 'libz.a' when configuring
Shouldn't this be fixed by now in PETSc-dev? Begin forwarded message: From: Satish Balay balay at mcs.anl.gov Subject: Re: [petsc-users] cannot find 'libz.a' when configuring Date: April 10, 2013 5:21:38 PM CDT To: PETSc users list petsc-users at mcs.anl.gov Reply-To: PETSc users list petsc-users at mcs.anl.gov try using configure option LIBS=-L/opt/zlib-1.2.7/lib -lz' Satish On Wed, 10 Apr 2013, Seungbum Koo wrote: Hi. I tried to add hdf5 when configuring. HDF5-1.8.9 is currently installed with szip-2.1 and zlib-1.2.7. It stops with message *** UNABLE to CONFIGURE with GIVEN OPTIONS(see configure.log for details): --- Compression library [libz.a or equivalent] not found *** What I don't understand is that 'libz.a' file exists in '/opt/zlib-1.2.7/lib'. What should I do? Seungbum
