Re: Linking mod_ssl with a specific OpenSSL version
On 18.09.2012 15:05, Joe Orton wrote: one minor nit/bikeshed-colour-review: - APR_ADDTO(MOD_SSL_LDADD, [-export-symbols-regex ssl_module]) + APR_ADDTO(MOD_LDFLAGS, [-export-symbols-regex ssl_module]) this is currently equivalent but keeping _LDADD seems better. No strong feelings about that, so switched back in r1387610. Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On 18.09.2012 17:55, Ben Laurie wrote: though I still have to actually install openssl somewhere, which would be irritating if I had to do any serious development. That said, I could probably figure out how to work around that if I needed to... Something like the following will make your OpenSSL build directory behave like an installation directory to httpd's configure: cd /wherever/the/compiled/openssl/sources/are mkdir -p lib/pkgconfig echo prefix=`pwd` lib/pkgconfig/openssl.pc tail +2 openssl.pc lib/pkgconfig/openssl.pc cd lib for l in ../*.a ../*.so*; do ln -s $l; done (by omitting the symlinks for *.so*, it's possible to force the static versions of libcrypto and libssl being used for mod_ssl) Now ... what about the backport? :-) Proposed for 2.4 in r1387613. Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On Sun, Sep 16, 2012 at 08:00:00AM +0200, Kaspar Brand wrote: I have committed an improved version in r1385214 (in particular, more tweaking was required to properly handle support/ab, which can't make use of MOD_CFLAGS etc.). Reviews, further testing and feedback welcome. Looks good to me, thanks Kaspar. one minor nit/bikeshed-colour-review: - APR_ADDTO(MOD_SSL_LDADD, [-export-symbols-regex ssl_module]) + APR_ADDTO(MOD_LDFLAGS, [-export-symbols-regex ssl_module]) this is currently equivalent but keeping _LDADD seems better. Regards, Joe
Re: Linking mod_ssl with a specific OpenSSL version
On Sun, Sep 16, 2012 at 7:24 AM, Kaspar Brand httpd-dev.2...@velox.ch wrote: On 16.09.2012 08:00, Kaspar Brand wrote: I have committed an improved version in r1385214 Um, make that read r1385216. I left out the acinclude.m4 changes in the first attempt, unfortunately. OK, I just checked it out and tested it - it does indeed solve my problem, though I still have to actually install openssl somewhere, which would be irritating if I had to do any serious development. That said, I could probably figure out how to work around that if I needed to... So ... thanks! Now ... what about the backport? :-)
Re: Linking mod_ssl with a specific OpenSSL version
On 23.08.2012 09:22, Kaspar Brand wrote: On 17.08.2012 10:11, Joe Orton wrote: On Thu, Aug 16, 2012 at 08:36:31PM +0200, Kaspar Brand wrote: I wonder if we should add support for module-specific CFLAGS etc., which would always appear before the EXTRA_XXX stuff in the compile and link commands, i.e. in rules.mk we would have: ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS) ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS) ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES) ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS) A particular module could then set its specific MOD_CFLAGS etc. in modules.mk, and these would always have priority over those possibly inserted by other modules. Doing CFLAGS et al like that doesn't generalise brilliantly, because they are per-directory (modules/xxx) not strictly per-module, but it could be done anyway, and that wouldn't matter for mod_ssl. Yeah, probably a good idea. I gave it a try, and so far it seems to work as expected, see the attached patch (against r1358166, to reduce clutter). I have committed an improved version in r1385214 (in particular, more tweaking was required to properly handle support/ab, which can't make use of MOD_CFLAGS etc.). Reviews, further testing and feedback welcome. Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On 16.09.2012 08:00, Kaspar Brand wrote: I have committed an improved version in r1385214 Um, make that read r1385216. I left out the acinclude.m4 changes in the first attempt, unfortunately. Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On 17.08.2012 10:11, Joe Orton wrote:
On Thu, Aug 16, 2012 at 08:36:31PM +0200, Kaspar Brand wrote:
I wonder if we should add support for module-specific CFLAGS etc.,
which would always appear before the EXTRA_XXX stuff in the compile
and link commands, i.e. in rules.mk we would have:
ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS)
$(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS)
ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES)
ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
A particular module could then set its specific MOD_CFLAGS etc. in
modules.mk, and these would always have priority over those possibly
inserted by other modules.
Doing CFLAGS et al like that doesn't generalise brilliantly, because
they are per-directory (modules/xxx) not strictly per-module, but it
could be done anyway, and that wouldn't matter for mod_ssl. Yeah,
probably a good idea.
I gave it a try, and so far it seems to work as expected, see the
attached patch (against r1358166, to reduce clutter). Right now the
following modules are affected:
- modules/cache: mod_socache_dc (--with-distcache)
- modules/filters: mod_deflate (--with-z), mod_xml2enc (--with-libxml2),
mod_proxy_html (--with-libxml2)
- modules/lua: mod_lua (--with-lua, --enable-luajit)
- modules/ssl: mod_ssl (--with-ssl)
I.e. there's currently only a potential clash in modules/filters.
It should be possible to override LDFLAGS truly per-module by tweaking
the SH_LINK line generated in modules.mk.
I didn't pursue this option for the time being, as it would currently
only be of potential benefit for mod_deflate and
mod_xml2enc/mod_proxy_html. If you (or other devs) think it's an
important aspect, please let me know and I'll have a closer look.
Feedback and comments about the proposed approach - especially from
build system experts - is very much welcome.
Kaspar
Index: acinclude.m4
===
--- acinclude.m4(revision 1358166)
+++ acinclude.m4(working copy)
@@ -149,12 +149,20 @@
fi
])
+dnl the list of build variabled which are available for customization on a
+dnl per module directory basis (to be inserted into modules.mk with a MOD_
+dnl prefix, i.e. MOD_CFLAGS etc.). Used in APACHE_MODPATH_{INIT,FINISH}.
+define(mod_buildvars, [CFLAGS CXXFLAGS CPPFLAGS LDFLAGS LIBS INCLUDES])
+dnl
dnl APACHE_MODPATH_INIT(modpath)
AC_DEFUN(APACHE_MODPATH_INIT,[
current_dir=$1
modpath_current=modules/$1
modpath_static=
modpath_shared=
+ for var in mod_buildvars; do
+eval MOD_$var=
+ done
test -d $1 || $srcdir/build/mkdir.sh $modpath_current
$modpath_current/modules.mk
])dnl
@@ -163,6 +171,11 @@
echo DISTCLEAN_TARGETS = modules.mk $modpath_current/modules.mk
echo static = $modpath_static $modpath_current/modules.mk
echo shared = $modpath_shared $modpath_current/modules.mk
+ for var in mod_buildvars; do
+if eval val=\\$MOD_$var\; test -n $val; then
+ echo MOD_$var = $val $modpath_current/modules.mk
+fi
+ done
if test ! -z $modpath_static -o ! -z $modpath_shared; then
MODULE_DIRS=$MODULE_DIRS $current_dir
else
@@ -480,7 +493,7 @@
dnl Determine the OpenSSL base directory, if any
AC_MSG_CHECKING([for user-provided OpenSSL base directory])
-AC_ARG_WITH(ssl, APACHE_HELP_STRING(--with-ssl=DIR,OpenSSL base
directory), [
+AC_ARG_WITH(ssl, APACHE_HELP_STRING(--with-ssl=PATH,OpenSSL installation
directory), [
dnl If --with-ssl specifies a directory, we use that directory
if test x$withval != xyes -a x$withval != x; then
dnl This ensures $withval is actually a directory and that it is
absolute
@@ -497,7 +510,6 @@
saved_CPPFLAGS=$CPPFLAGS
saved_LIBS=$LIBS
saved_LDFLAGS=$LDFLAGS
-SSL_LIBS=
dnl Before doing anything else, load in pkg-config variables
if test -n $PKGCONFIG; then
@@ -514,10 +526,10 @@
ap_openssl_found=yes
pkglookup=`$PKGCONFIG --cflags-only-I openssl`
APR_ADDTO(CPPFLAGS, [$pkglookup])
-APR_ADDTO(INCLUDES, [$pkglookup])
+APR_ADDTO(MOD_INCLUDES, [$pkglookup])
pkglookup=`$PKGCONFIG --libs-only-L --libs-only-other openssl`
APR_ADDTO(LDFLAGS, [$pkglookup])
-APR_ADDTO(SSL_LIBS, [$pkglookup])
+APR_ADDTO(MOD_LDFLAGS, [$pkglookup])
fi
PKG_CONFIG_PATH=$saved_PKG_CONFIG_PATH
fi
@@ -525,12 +537,12 @@
dnl fall back to the user-supplied directory if not found via pkg-config
if test x$ap_openssl_base != x -a x$ap_openssl_found = x; then
APR_ADDTO(CPPFLAGS, [-I$ap_openssl_base/include])
- APR_ADDTO(INCLUDES, [-I$ap_openssl_base/include])
+ APR_ADDTO(MOD_INCLUDES, [-I$ap_openssl_base/include])
APR_ADDTO(LDFLAGS, [-L$ap_openssl_base/lib])
- APR_ADDTO(SSL_LIBS,
Re: Linking mod_ssl with a specific OpenSSL version
On Thu, Aug 16, 2012 at 7:36 PM, Kaspar Brand httpd-dev.2...@velox.ch wrote:
On 12.8.12 20:01, Ben Laurie wrote:
On Sun, Aug 12, 2012 at 5:23 PM, Kaspar Brand httpd-dev.2...@velox.ch
wrote:
a workaround is to call configure with
suitable {CPP,LD}FLAGS, i.e.
CPPFLAGS=-I${openssl_build_dir}/include \
LDFLAGS=-L${openssl_build_dir} \
./configure ...
(when using the shared libssl/libcrypto libraries, adding
-Wl,-R${openssl_build_dir} or similar to LDFLAGS might make sense)
I haven't had time to retest it, but I think the problem with this
approach is that the default version of OpenSSL gets included first if
anything else uses /usr/local/{include,lib}. IIRC, this was the basis
of all my problems.
Ok, but then we're talking about a current limitation of the build
system, I think: the compile commands are put together in build/rules.mk
like this:
ALL_CFLAGS = $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(EXTRA_CPPFLAGS)
$(NOTEST_CPPFLAGS) $(CPPFLAGS)
[...]
ALL_INCLUDES = $(INCLUDES) $(EXTRA_INCLUDES)
# Compile commands
BASE_CC = $(CC) $(ALL_CFLAGS) $(ALL_CPPFLAGS) $(ALL_INCLUDES)
Both $EXTRA_CPPFLAGS and $EXTRA_INCLUDES are assembled by configure,
where each module can contribute its stuff (appended with APR_ADDTO,
usually). I.e., if you happen to configure mod_deflate with a zlib
in /usr/local, then you end up with $EXTRA_INCLUDES where mod_deflate's
-I/usr/local precedes -I/path/to/openssl/build/dir (since
modules/filter/config.m4 is run before modules/ssl/config.m4).
Similarly, for linking we have:
ALL_LDFLAGS = $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
[...]
LINK = $(LIBTOOL) --mode=link $(CC) $(ALL_CFLAGS) $(LT_LDFLAGS)
$(ALL_LDFLAGS) -o $@
I wonder if we should add support for module-specific CFLAGS etc.,
which would always appear before the EXTRA_XXX stuff in the compile
and link commands, i.e. in rules.mk we would have:
ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS)
$(NOTEST_CPPFLAGS) $(CPPFLAGS)
ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES)
ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
A particular module could then set its specific MOD_CFLAGS etc. in
modules.mk, and these would always have priority over those possibly
inserted by other modules.
This sounds like it would solve my problem - and seems like about as a
good a solution as we can expect :-)
Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On Thu, Aug 16, 2012 at 08:36:31PM +0200, Kaspar Brand wrote: I wonder if we should add support for module-specific CFLAGS etc., which would always appear before the EXTRA_XXX stuff in the compile and link commands, i.e. in rules.mk we would have: ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS) ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS) $(NOTEST_CPPFLAGS) $(CPPFLAGS) ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES) ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS) A particular module could then set its specific MOD_CFLAGS etc. in modules.mk, and these would always have priority over those possibly inserted by other modules. Doing CFLAGS et al like that doesn't generalise brilliantly, because they are per-directory (modules/xxx) not strictly per-module, but it could be done anyway, and that wouldn't matter for mod_ssl. Yeah, probably a good idea. It should be possible to override LDFLAGS truly per-module by tweaking the SH_LINK line generated in modules.mk. Regards, Joe
Re: Linking mod_ssl with a specific OpenSSL version
On 12.8.12 20:01, Ben Laurie wrote:
On Sun, Aug 12, 2012 at 5:23 PM, Kaspar Brand httpd-dev.2...@velox.ch wrote:
a workaround is to call configure with
suitable {CPP,LD}FLAGS, i.e.
CPPFLAGS=-I${openssl_build_dir}/include \
LDFLAGS=-L${openssl_build_dir} \
./configure ...
(when using the shared libssl/libcrypto libraries, adding
-Wl,-R${openssl_build_dir} or similar to LDFLAGS might make sense)
I haven't had time to retest it, but I think the problem with this
approach is that the default version of OpenSSL gets included first if
anything else uses /usr/local/{include,lib}. IIRC, this was the basis
of all my problems.
Ok, but then we're talking about a current limitation of the build
system, I think: the compile commands are put together in build/rules.mk
like this:
ALL_CFLAGS = $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(EXTRA_CPPFLAGS)
$(NOTEST_CPPFLAGS) $(CPPFLAGS)
[...]
ALL_INCLUDES = $(INCLUDES) $(EXTRA_INCLUDES)
# Compile commands
BASE_CC = $(CC) $(ALL_CFLAGS) $(ALL_CPPFLAGS) $(ALL_INCLUDES)
Both $EXTRA_CPPFLAGS and $EXTRA_INCLUDES are assembled by configure,
where each module can contribute its stuff (appended with APR_ADDTO,
usually). I.e., if you happen to configure mod_deflate with a zlib
in /usr/local, then you end up with $EXTRA_INCLUDES where mod_deflate's
-I/usr/local precedes -I/path/to/openssl/build/dir (since
modules/filter/config.m4 is run before modules/ssl/config.m4).
Similarly, for linking we have:
ALL_LDFLAGS = $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
[...]
LINK = $(LIBTOOL) --mode=link $(CC) $(ALL_CFLAGS) $(LT_LDFLAGS)
$(ALL_LDFLAGS) -o $@
I wonder if we should add support for module-specific CFLAGS etc.,
which would always appear before the EXTRA_XXX stuff in the compile
and link commands, i.e. in rules.mk we would have:
ALL_CFLAGS = $(MOD_CFLAGS) $(EXTRA_CFLAGS) $(NOTEST_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(DEFS) $(INTERNAL_CPPFLAGS) $(MOD_CPPFLAGS) $(EXTRA_CPPFLAGS)
$(NOTEST_CPPFLAGS) $(CPPFLAGS)
ALL_INCLUDES = $(INCLUDES) $(MOD_INCLUDES) $(EXTRA_INCLUDES)
ALL_LDFLAGS = $(MOD_LDFLAGS) $(EXTRA_LDFLAGS) $(NOTEST_LDFLAGS) $(LDFLAGS)
A particular module could then set its specific MOD_CFLAGS etc. in
modules.mk, and these would always have priority over those possibly
inserted by other modules.
Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On 10.08.2012 01:55, William A. Rowe Jr. wrote:
An openssl 'make localinstall' could trivially create the lib, include
trees consisting entirely of symlinks to the origin files in the same
build tree, and create an appropriate openssl.pc file describing the
link steps required to
Specifically, that lib subdirectory would have to be populated like so:
lib
|-- engines
| |-- lib4758cca.so
| |-- ...
| `-- libubsec.so
|-- pkgconfig
| |-- libcrypto.pc
| |-- libssl.pc
| `-- openssl.pc
|-- libcrypto.a
|-- libcrypto.so - libcrypto.so.1.0.0
|-- libcrypto.so.1.0.0
|-- libssl.a
|-- libssl.so - libssl.so.1.0.0
`-- libssl.so.1.0.0
(If pkg-config is in $PATH, but lib/pkgconfig/openssl.pc is missing,
then httpd's configure will pick the default installation of OpenSSL,
and ignore the directory specified through --with-ssl.)
Rather than solving 'for the httpd case', this would allow you to build
most any app or library depending upon openssl in a sensible way, in
order to diagnose openssl's interoperation with any consumer.
This really isn't the repository to solve openssl's contorted build
issues.
I tend to agree, and would be in favor of reverting r1358167. As long as
OpenSSL doesn't provide a Makefile target for creating the lib
subdirectory in the build tree, a workaround is to call configure with
suitable {CPP,LD}FLAGS, i.e.
CPPFLAGS=-I${openssl_build_dir}/include \
LDFLAGS=-L${openssl_build_dir} \
./configure ...
(when using the shared libssl/libcrypto libraries, adding
-Wl,-R${openssl_build_dir} or similar to LDFLAGS might make sense)
One problem with the current version of acinclude.m4 I noticed in the
meantime is that on OS X, the test in line 503
(https://svn.apache.org/viewvc/httpd/httpd/trunk/acinclude.m4?view=markuppathrev=1358167#l503)
is true whenever --with-ssl is missing from the configure arguments (OS
X doesn't have /lib). configure will then add bogus -L strings to
SSL_LIBS and EXTRA_LDFLAGS in build/config_vars.mk... and this breaks
the link commands for the whole tree (libtool: link: require no space
between `-L' and `-o').
Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On Sun, Aug 12, 2012 at 5:23 PM, Kaspar Brand httpd-dev.2...@velox.ch wrote:
On 10.08.2012 01:55, William A. Rowe Jr. wrote:
An openssl 'make localinstall' could trivially create the lib, include
trees consisting entirely of symlinks to the origin files in the same
build tree, and create an appropriate openssl.pc file describing the
link steps required to
Specifically, that lib subdirectory would have to be populated like so:
lib
|-- engines
| |-- lib4758cca.so
| |-- ...
| `-- libubsec.so
|-- pkgconfig
| |-- libcrypto.pc
| |-- libssl.pc
| `-- openssl.pc
|-- libcrypto.a
|-- libcrypto.so - libcrypto.so.1.0.0
|-- libcrypto.so.1.0.0
|-- libssl.a
|-- libssl.so - libssl.so.1.0.0
`-- libssl.so.1.0.0
(If pkg-config is in $PATH, but lib/pkgconfig/openssl.pc is missing,
then httpd's configure will pick the default installation of OpenSSL,
and ignore the directory specified through --with-ssl.)
Rather than solving 'for the httpd case', this would allow you to build
most any app or library depending upon openssl in a sensible way, in
order to diagnose openssl's interoperation with any consumer.
This really isn't the repository to solve openssl's contorted build
issues.
I tend to agree, and would be in favor of reverting r1358167. As long as
OpenSSL doesn't provide a Makefile target for creating the lib
subdirectory in the build tree, a workaround is to call configure with
suitable {CPP,LD}FLAGS, i.e.
CPPFLAGS=-I${openssl_build_dir}/include \
LDFLAGS=-L${openssl_build_dir} \
./configure ...
(when using the shared libssl/libcrypto libraries, adding
-Wl,-R${openssl_build_dir} or similar to LDFLAGS might make sense)
I haven't had time to retest it, but I think the problem with this
approach is that the default version of OpenSSL gets included first if
anything else uses /usr/local/{include,lib}. IIRC, this was the basis
of all my problems.
One problem with the current version of acinclude.m4 I noticed in the
meantime is that on OS X, the test in line 503
(https://svn.apache.org/viewvc/httpd/httpd/trunk/acinclude.m4?view=markuppathrev=1358167#l503)
is true whenever --with-ssl is missing from the configure arguments (OS
X doesn't have /lib). configure will then add bogus -L strings to
SSL_LIBS and EXTRA_LDFLAGS in build/config_vars.mk... and this breaks
the link commands for the whole tree (libtool: link: require no space
between `-L' and `-o').
Kaspar
Re: Linking mod_ssl with a specific OpenSSL version
On 8/8/2012 9:03 AM, Joe Orton wrote: On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote: My thinking was that people should explicitly tell configure that they want to link with the libs in a build directory (so that they don't accidentally use a directory which might only temporarily exist - that's also the primary reason for preferring the static over the shared libs from that dir). This all seems totally crazy to me. Why are we adding complexity to the httpd build system so openssl devs can skip typing the install part of running make install when testing against modified versions of OpenSSL? Joe I concur. If lib devs wish to use a build tree, it should not be that hard to properly craft the right symlinks for an 'install in place' model used for rapid development. I agree with Ben that this is desireable, but httpd is not the place to solve it. openssl.pc can describe such an environment in a useful manner for developers and solve the entire issue in the lib package.
Re: Linking mod_ssl with a specific OpenSSL version
On 8/8/2012 8:56 PM, Ben Laurie wrote: On Wed, Aug 8, 2012 at 5:03 PM, Joe Orton jor...@redhat.com wrote: On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote: My thinking was that people should explicitly tell configure that they want to link with the libs in a build directory (so that they don't accidentally use a directory which might only temporarily exist - that's also the primary reason for preferring the static over the shared libs from that dir). This all seems totally crazy to me. Why are we adding complexity to the httpd build system so openssl devs can skip typing the install part of running make install when testing against modified versions of OpenSSL? Joe IIRC, the issue is that it doesn't build correctly even if you do that (to a non-standard location - and clearly installing to a standard location is nuts). I have no issues installing to a non-standard location, and with a very few hacks, can even eliminate the -R compiled-in search paths and create a build entirely managed by LD_LIBRARY_PATH. It would help if you explained better what bug you see in handling the --with-ssl arguement, and I think we could resolve this easily. If the issue is that openssl.pc is not processed correctly by httpd or not created correctly by openssl.org, then we can fix either defect.
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On 8/8/2012 4:45 AM, Ben Laurie wrote: On Wed, Aug 8, 2012 at 2:47 AM, Guenter Knauf fua...@apache.org wrote: Am 08.08.2012 07:39, schrieb Kaspar Brand: On 06.08.2012 22:08, William A. Rowe Jr. wrote: On 8/5/2012 10:10 PM, Kaspar Brand wrote: On 05.08.2012 14:38, Guenter Knauf wrote: Am 05.08.2012 10:10, schrieb Kaspar Brand: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) what about splitting into two arguments: --with-ssl-include= --with-ssl-lib= this would be equal to what many other configure also use ... That's an option, yes, although the way the proposed option would work doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib (i.e., add those args with -I and -L): --with-ssl-builddir forces mod_ssl to always be linked with the static libraries in that directory. Maybe --with-ssl-static-libdir would be a more appropriate name? Why not simply consume the installed openssl.pc? In this particular case we're talking about using an OpenSSL build directory, i.e. the generated openssl.pc is sitting in a temporary location in that build tree (and when used, would output incorrect includedir/libdir locations). probably a dumb question and I'm missing something, but: why is this necessary at all? Given someone wants to build/link against a development version of OpenSSL why should he ommit the last step of installing OpenSSL to a separate location? Time taken to iterate in development is an important determiner of productivity. Having to do an install for every change introduces significant delay. It also has a tendency to make debugging hard. I agree. An openssl 'make localinstall' could trivially create the lib, include trees consisting entirely of symlinks to the origin files in the same build tree, and create an appropriate openssl.pc file describing the link steps required to Rather than solving 'for the httpd case', this would allow you to build most any app or library depending upon openssl in a sensible way, in order to diagnose openssl's interoperation with any consumer. This really isn't the repository to solve openssl's contorted build issues.
Re: Linking mod_ssl with a specific OpenSSL version
On 06.08.2012 23:36, Rainer Jung wrote: On 05.08.2012 10:10, Kaspar Brand wrote: It's a useful enhancement if mod_ssl can be linked with a specific OpenSSL version in a non-default location, but the current approach has at least one problem, AFAICT: it will only work if the directory pointed to by --with-ssl does not include shared libraries for OpenSSL (by default, OpenSSL only builds libssl.a and libcrypto.a, so the issue might not be obvious at first sight). Why wouldn't it work with shared libs? Because they usually have dependencies themselves that are ignored? Sorry, will only work is inaccurate... what I was (implicitly) assuming is that the OpenSSL build directory would not be part of the standard library search path - i.e., unless LD_LIBRARY_PATH is adapted, httpd won't find the proper libraries at startup. Forcing the linker to prefer the static over the shared libraries in a particular directory (like I tried in the preliminary patch with the -Wl,-Bstatic,-lssl,-lcrypto,-Bdynamic linker options) turns out to be too linker/platform specific, however - I have abandoned this idea meanwhile. I'm not sure what to really do about it... perhaps add the OpenSSL builddir with -R and leave the rest to libtool? I would suggest to use a separate configure argument to support this build option, e.g. --with-ssl-srcdir. We could, but I think if it would be easy to just add the logic to with-ssl to also work with a src directory it would be easier to use. I have no strong opinion on this though. My thinking was that people should explicitly tell configure that they want to link with the libs in a build directory (so that they don't accidentally use a directory which might only temporarily exist - that's also the primary reason for preferring the static over the shared libs from that dir). I do like the idea to give with-ssl priority over pkgconfig. If a user chooses the ssl directory explicitly that should not be overwritten by pkgconfig. Of course one can still use pkgconfig info in the chosen directory to find dependency libs etc. OpenSSL doesn't have any further dependencies, AFAICT (cf. http://cvs.openssl.org/fileview?f=openssl/Makefile.org and the Requires: line it adds to openssl.pc). Are you referring to the --libs-only-other option for pkg-config? Kaspar
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
Am 08.08.2012 07:39, schrieb Kaspar Brand: On 06.08.2012 22:08, William A. Rowe Jr. wrote: On 8/5/2012 10:10 PM, Kaspar Brand wrote: On 05.08.2012 14:38, Guenter Knauf wrote: Am 05.08.2012 10:10, schrieb Kaspar Brand: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) what about splitting into two arguments: --with-ssl-include= --with-ssl-lib= this would be equal to what many other configure also use ... That's an option, yes, although the way the proposed option would work doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib (i.e., add those args with -I and -L): --with-ssl-builddir forces mod_ssl to always be linked with the static libraries in that directory. Maybe --with-ssl-static-libdir would be a more appropriate name? Why not simply consume the installed openssl.pc? In this particular case we're talking about using an OpenSSL build directory, i.e. the generated openssl.pc is sitting in a temporary location in that build tree (and when used, would output incorrect includedir/libdir locations). probably a dumb question and I'm missing something, but: why is this necessary at all? Given someone wants to build/link against a development version of OpenSSL why should he ommit the last step of installing OpenSSL to a separate location? Wouldnt everything work fine using --with-ssl= and building OpenSSL static-only + installing to a separate development location? Gün.
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On Wed, Aug 8, 2012 at 2:47 AM, Guenter Knauf fua...@apache.org wrote: Am 08.08.2012 07:39, schrieb Kaspar Brand: On 06.08.2012 22:08, William A. Rowe Jr. wrote: On 8/5/2012 10:10 PM, Kaspar Brand wrote: On 05.08.2012 14:38, Guenter Knauf wrote: Am 05.08.2012 10:10, schrieb Kaspar Brand: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) what about splitting into two arguments: --with-ssl-include= --with-ssl-lib= this would be equal to what many other configure also use ... That's an option, yes, although the way the proposed option would work doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib (i.e., add those args with -I and -L): --with-ssl-builddir forces mod_ssl to always be linked with the static libraries in that directory. Maybe --with-ssl-static-libdir would be a more appropriate name? Why not simply consume the installed openssl.pc? In this particular case we're talking about using an OpenSSL build directory, i.e. the generated openssl.pc is sitting in a temporary location in that build tree (and when used, would output incorrect includedir/libdir locations). probably a dumb question and I'm missing something, but: why is this necessary at all? Given someone wants to build/link against a development version of OpenSSL why should he ommit the last step of installing OpenSSL to a separate location? Time taken to iterate in development is an important determiner of productivity. Having to do an install for every change introduces significant delay. It also has a tendency to make debugging hard. Wouldnt everything work fine using --with-ssl= and building OpenSSL static-only + installing to a separate development location? Gün.
Re: Linking mod_ssl with a specific OpenSSL version
On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote: My thinking was that people should explicitly tell configure that they want to link with the libs in a build directory (so that they don't accidentally use a directory which might only temporarily exist - that's also the primary reason for preferring the static over the shared libs from that dir). This all seems totally crazy to me. Why are we adding complexity to the httpd build system so openssl devs can skip typing the install part of running make install when testing against modified versions of OpenSSL? Joe
Re: Linking mod_ssl with a specific OpenSSL version
On Wed, 2012-08-08 at 17:03 +0100, Joe Orton wrote: On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote: My thinking was that people should explicitly tell configure that they want to link with the libs in a build directory (so that they don't accidentally use a directory which might only temporarily exist - that's also the primary reason for preferring the static over the shared libs from that dir). This all seems totally crazy to me. Why are we adding complexity to the httpd build system so openssl devs can skip typing the install part of running make install when testing against modified versions of OpenSSL? Joe +1 signature.asc Description: This is a digitally signed message part
Re: Linking mod_ssl with a specific OpenSSL version
On Wed, Aug 8, 2012 at 5:03 PM, Joe Orton jor...@redhat.com wrote: On Wed, Aug 08, 2012 at 08:00:25AM +0200, Kaspar Brand wrote: My thinking was that people should explicitly tell configure that they want to link with the libs in a build directory (so that they don't accidentally use a directory which might only temporarily exist - that's also the primary reason for preferring the static over the shared libs from that dir). This all seems totally crazy to me. Why are we adding complexity to the httpd build system so openssl devs can skip typing the install part of running make install when testing against modified versions of OpenSSL? Joe IIRC, the issue is that it doesn't build correctly even if you do that (to a non-standard location - and clearly installing to a standard location is nuts). The only extra complexity introduced by allowing a dev to skip make install is that the lib is in base instead of base/lib.
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On 06.08.2012 22:08, William A. Rowe Jr. wrote: On 8/5/2012 10:10 PM, Kaspar Brand wrote: On 05.08.2012 14:38, Guenter Knauf wrote: Am 05.08.2012 10:10, schrieb Kaspar Brand: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) what about splitting into two arguments: --with-ssl-include= --with-ssl-lib= this would be equal to what many other configure also use ... That's an option, yes, although the way the proposed option would work doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib (i.e., add those args with -I and -L): --with-ssl-builddir forces mod_ssl to always be linked with the static libraries in that directory. Maybe --with-ssl-static-libdir would be a more appropriate name? Why not simply consume the installed openssl.pc? In this particular case we're talking about using an OpenSSL build directory, i.e. the generated openssl.pc is sitting in a temporary location in that build tree (and when used, would output incorrect includedir/libdir locations). Kaspar
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On 8/5/2012 10:10 PM, Kaspar Brand wrote: On 05.08.2012 14:38, Guenter Knauf wrote: Am 05.08.2012 10:10, schrieb Kaspar Brand: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) what about splitting into two arguments: --with-ssl-include= --with-ssl-lib= this would be equal to what many other configure also use ... That's an option, yes, although the way the proposed option would work doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib (i.e., add those args with -I and -L): --with-ssl-builddir forces mod_ssl to always be linked with the static libraries in that directory. Maybe --with-ssl-static-libdir would be a more appropriate name? Why not simply consume the installed openssl.pc?
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On 05.08.2012 10:10, Kaspar Brand wrote: On 08.07.2012 10:30, Kaspar Brand wrote: On 06.07.2012 14:41, b...@apache.org wrote: Author: ben Date: Fri Jul 6 12:41:10 2012 New Revision: 1358167 URL: http://svn.apache.org/viewvc?rev=1358167view=rev Log: Work correctly with a development version of OpenSSL. I suspect something similar is needed when there are two OpenSSL installations, one in a default location. I had another look at this, since it has been proposed for backporting to 2.4 in the meantime, and still think the following is true: If I'm understanding correctly, then this patch tries to support building against an OpenSSL source tree (or perhaps a build directory where only make libs has been executed)? That is my understanding as well. (should have read make build_libs instead) It's a useful enhancement if mod_ssl can be linked with a specific OpenSSL version in a non-default location, but the current approach has at least one problem, AFAICT: it will only work if the directory pointed to by --with-ssl does not include shared libraries for OpenSSL (by default, OpenSSL only builds libssl.a and libcrypto.a, so the issue might not be obvious at first sight). Why wouldn't it work with shared libs? Because they usually have dependencies themselves that are ignored? I would suggest to use a separate configure argument to support this build option, e.g. --with-ssl-srcdir. We could, but I think if it would be easy to just add the logic to with-ssl to also work with a src directory it would be easier to use. I have no strong opinion on this though. I gave it a try, see the attached work-in-progress patch. While we're at it, I think we should also fix a flaw in the handling of the --with-ssl argument: in http://svn.apache.org/viewvc?view=revisionrevision=730926, acinclude.m4 was modified to always give pkg-config precedence over any argument specified through --with-ssl. While the rationale for this change becomes clear from the commit log, I consider it an unfortunate side effect that pkg-config always trumps any --with-ssl directory argument. My suggestion would be to handle OpenSSL paths in configure arguments like this, instead: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) 2) use --with-ssl for linking against an installed version of OpenSSL 3) use pkg-config to locate OpenSSL Does that sound like a reasonable proposal? Comments welcome, and test feedback would be much appreciated (remember to run buildconf after applying the patch to acinclude.m4, and before calling configure). I do like the idea to give with-ssl priority over pkgconfig. If a user chooses the ssl directory explicitly that should not be overwritten by pkgconfig. Of course one can still use pkgconfig info in the chosen directory to find dependency libs etc. Whether 1) and 2) needs to be a separate priority or simply using the same switch and auto-detect whether it is an installation directory or only a build directory is undecided to me. Regards, Rainer
Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On 08.07.2012 10:30, Kaspar Brand wrote: On 06.07.2012 14:41, b...@apache.org wrote: Author: ben Date: Fri Jul 6 12:41:10 2012 New Revision: 1358167 URL: http://svn.apache.org/viewvc?rev=1358167view=rev Log: Work correctly with a development version of OpenSSL. I suspect something similar is needed when there are two OpenSSL installations, one in a default location. I had another look at this, since it has been proposed for backporting to 2.4 in the meantime, and still think the following is true: If I'm understanding correctly, then this patch tries to support building against an OpenSSL source tree (or perhaps a build directory where only make libs has been executed)? (should have read make build_libs instead) It's a useful enhancement if mod_ssl can be linked with a specific OpenSSL version in a non-default location, but the current approach has at least one problem, AFAICT: it will only work if the directory pointed to by --with-ssl does not include shared libraries for OpenSSL (by default, OpenSSL only builds libssl.a and libcrypto.a, so the issue might not be obvious at first sight). I would suggest to use a separate configure argument to support this build option, e.g. --with-ssl-srcdir. I gave it a try, see the attached work-in-progress patch. While we're at it, I think we should also fix a flaw in the handling of the --with-ssl argument: in http://svn.apache.org/viewvc?view=revisionrevision=730926, acinclude.m4 was modified to always give pkg-config precedence over any argument specified through --with-ssl. While the rationale for this change becomes clear from the commit log, I consider it an unfortunate side effect that pkg-config always trumps any --with-ssl directory argument. My suggestion would be to handle OpenSSL paths in configure arguments like this, instead: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) 2) use --with-ssl for linking against an installed version of OpenSSL 3) use pkg-config to locate OpenSSL Does that sound like a reasonable proposal? Comments welcome, and test feedback would be much appreciated (remember to run buildconf after applying the patch to acinclude.m4, and before calling configure). Kaspar Index: acinclude.m4 === --- acinclude.m4(revision 1369535) +++ acinclude.m4(working copy) @@ -467,86 +467,97 @@ dnl dnl APACHE_CHECK_OPENSSL dnl -dnl Configure for OpenSSL, giving preference to -dnl --with-ssl=path if it was specified. +dnl Configure for OpenSSL, giving preference to the following options: +dnl 1) --with-ssl-builddir=path, for linking against static libraries +dnlin an OpenSSL build directory where at least make build_libs +dnlhas been executed +dnl 2) --with-ssl=path, pointing to a path where an installed version +dnlof OpenSSL can be found +dnl 3) the path as determined by pkg-config dnl AC_DEFUN(APACHE_CHECK_OPENSSL,[ - AC_CACHE_CHECK([for OpenSSL], [ac_cv_openssl], [ + AC_CACHE_VAL([ac_cv_openssl], [ dnl initialise the variables we use ac_cv_openssl=no -ap_openssl_found= ap_openssl_base= -ap_openssl_libs= +saved_CPPFLAGS=$CPPFLAGS +SSL_LIBS= -dnl Determine the OpenSSL base directory, if any -AC_MSG_CHECKING([for user-provided OpenSSL base directory]) -AC_ARG_WITH(ssl, APACHE_HELP_STRING(--with-ssl=DIR,OpenSSL base directory), [ - dnl If --with-ssl specifies a directory, we use that directory - if test x$withval != xyes -a x$withval != x; then -dnl This ensures $withval is actually a directory and that it is absolute +AC_MSG_NOTICE([checking for OpenSSL...]) + +dnl Allow linking against static libraries from an OpenSSL build directory +AC_MSG_CHECKING([for user-provided OpenSSL build directory with static libraries]) +AC_ARG_WITH(ssl-builddir, APACHE_HELP_STRING(--with-ssl-builddir=DIR,OpenSSL build directory with static libraries to link with), [ + if test x$withval != xyes -a -d $withval; then +dnl This ensures $withval is actually a directory +dnl and that it is absolute ap_openssl_base=`cd $withval ; pwd` +if test x$ap_openssl_base != x; then + AC_MSG_RESULT($ap_openssl_base) + CPPFLAGS=-I$ap_openssl_base/include $CPPFLAGS + INCLUDES=-I$ap_openssl_base/include $INCLUDES + if test x$enable_ssl = xstatic; then +APR_ADDTO(LIBS, [$ap_openssl_base/libssl.a $ap_openssl_base/libcrypto.a]) + else +LDFLAGS=-L$ap_openssl_base -Wl,-L$ap_openssl_base $LDFLAGS +dnl force the linker to use libssl.a and libcrypto.a (but only +dnl these, i.e. make sure that we are switching back to dynamic +dnl mode afterwards - from ld(1): affects library searching +dnl for -l options which follow it) +APR_ADDTO(SSL_LIBS,
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On 05.08.2012 10:10, Kaspar Brand wrote: test feedback would be much appreciated (remember to run buildconf after applying the patch to acinclude.m4, and before calling configure). The patch attached to the previous message was missing an important line, unfortunately (sorry to anybody who already played with it). Apply the one below on top of mod_ssl-configure-options.patch, and things should work properly. Kaspar --- acinclude.m4.orig +++ acinclude.m4 @@ -574,6 +574,7 @@ ap_apr_libs=`$apr_config --libs` APR_ADDTO(SSL_LIBS, [$ap_apr_libs]) APR_ADDTO(LIBS, [-lssl -lcrypto $ap_apr_libs]) + APACHE_SUBST(SSL_LIBS) dnl Run library and function checks liberrors=
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
Hi Kaspar, Am 05.08.2012 10:10, schrieb Kaspar Brand: My suggestion would be to handle OpenSSL paths in configure arguments like this, instead: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) what about splitting into two arguments: --with-ssl-include= --with-ssl-lib= this would be equal to what many other configure also use ... just a thought ... Gün.
Re: Linking mod_ssl with a specific OpenSSL version (Re: svn commit: r1358167 - in /httpd/httpd/trunk: acinclude.m4 modules/ssl/ssl_engine_init.c)
On 05.08.2012 14:38, Guenter Knauf wrote: Am 05.08.2012 10:10, schrieb Kaspar Brand: 1) use --with-ssl-builddir for linking with the static OpenSSL libraries in that directory (and ignore --with-ssl in this case) what about splitting into two arguments: --with-ssl-include= --with-ssl-lib= this would be equal to what many other configure also use ... That's an option, yes, although the way the proposed option would work doesn't mirror the typical case of --with-xyz-include/--with-xyz-lib (i.e., add those args with -I and -L): --with-ssl-builddir forces mod_ssl to always be linked with the static libraries in that directory. Maybe --with-ssl-static-libdir would be a more appropriate name? Kaspar
