Re: Feature request: setting env vars for binary wrappers
On Tue, 2006-06-13 at 17:00 -0400, Ralf Wildenhues wrote: [ this is: http://thread.gmane.org/gmane.comp.gnu.libtool.bugs/5319 or http://lists.gnu.org/archive/html/bug-libtool/2006-03/msg00013.html ] Hi Behdad, everyone, Hi again, Sorry for the delay again. So, yeah, replying after two years... I know. Hope you still remember this issue and the patch is not too stale by now... * Behdad Esfahbod wrote on Thu, Mar 09, 2006 at 04:59:38PM CET: On Wed, 1 Mar 2006, Ralf Wildenhues wrote: * Behdad Esfahbod wrote on Wed, Mar 01, 2006 at 05:15:17AM CET: This is a feature request for libtool. Let me describe the problem as I face it, so you may have a workaround for it already: I'm using libtool in the Pango text rendering engine. The Pango library accesses a file in $prefix/etc/pango/pangorc to find its modules at run-time. Now all I want to do is to be able to run the examples in pango/example when Pango is not installed. The easiest way that can be done is to set the PANGO_RC_PATH envvar to a special pangorc file. Another is to pass the --pangorc path-to-pango-rc to the examples if they understand it. What I need from libtool is to let me do one of these things in the wrapper it creates for uninstalled binaries. [ my concerns were: we don't always build a wrapper ATM. ] Sure, a wrapper is created if need be. And this feature can be just another reason to create a wrapper. If we can find an answer to this question to define coherent semantics, I'm all for it. Ok, this is my view of the problem: Running uninstalled programs requires some modifications to the environment. One is to make sure that the uninstalled libraries are linked. Other changes may be needed, on a per application basis. Libtool is free to choose how to implement these. So far it has only supported the library path modification, and implemented that using a wrapper script on some platforms, or using rpath on others (right?) More or less, yes. The same goes with the other modifications. They can be easily implemented using a wrapper. So if there are such modifications requested, a wrapper should be used. Yep. And that's really the easiest solution: as soon as extra arguments or extra environment variables are passed, we always build a wrapper. My current workaround has been making pango-view accept a --pangorc path-to-pangorc parameter and defaulting to ./pangorc. But that opens up a known security problem... So I really want to fix it the right way. I don't quite understand how this fixes any security problems... but here you go with a suggestion (against current CVS HEAD). The attached patch implements two new link flags, -wrapper-arg and -wrapper-env, to prepend arguments to programs, and modify their environment. They will force creating a (shell) wrapper. Here's the hairy part about it: somebody may eventually want to extend the C wrapper that is currently used on w32 to encompass all the functionality that the shell wrapper currently has. And while I don't have current plans about this, I still don't want to add any unnecessary obstackles to it. Fair enough. So whatever we add to the shell wrapper should be doable easily in a C wrapper as well. This led me to add these restrictions: the -wrapper-env works like putenv: you pass an argument `var=value', the variable will be exported, the value will _not_ be interpreted by the shell any more. For now you cannot unset variables (this is to cater for hosts with a shell that does not support `unset'), and, e.g., -wrapper-env 'foo=$bar' will cause the environment variable `foo' to contain the four characters $, b, a, and r, not the contents of the variable $bar. But make variables are expanded, right? Similarly, -wrapper-arg will add exactly one literal argument to the exec. I've put suitable quoting in place for this to work with most kinds of input, and of course a test to this end. What do you think? OK for HEAD right now, or do you think this is too intrusive now? Sounds good to me. Can it finally go in?! I think we should not backport this to 1.5.x, it is a new feature. Cheers, Ralf Cheers, -- behdad http://behdad.org/ Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety. -- Benjamin Franklin, 1759
Re: add cegcc support
The README file explains it: run make check VERBOSE=yes TESTS=tests/demo-shared.test tests/demo-make.test tests/demo-exec.test Also, please run the other half of the tests (the new testsuite) using make check-local and post tests/testsuite.log, please. You can run both with make -k check. Here are the log of the test suite. After it, the output of the 2 failed tests (there are 2 now, after some modifications of the patch) ## -- ## ## libtool 2.2.3a test suite. ## ## -- ## Libtoolize operation. 1: libtoolize macro installation ok 2: libtoolize macro serial update ok 3: libtoolize config files serial update ok 4: diagnose missing LT_CONFIG_LTDL_DIR ok 5: copy ltdl.m4 with shared macro directoryok 6: correctly parse LTDL_INIT from configure.ac ok 7: diagnose missing LTDL_INIT invocation ok 8: upgrading verbatim style aclocal.m4 ok 9: nonrecursive ltdl with AC_CONFIG_MACRO_DIR ok 10: subproject ltdl with non-shared directories ok Testing libtool functions. 11: duplicate members in archive tests skipped (duplicate_members.at:73) 12: duplicate convenience archive names skipped (duplicate_conv.at:55) 13: preserve duplicate convenience deps skipped (duplicate_deps.at:61) 14: inherited_linker_flags ok 15: C convenience archives skipped (convenience.at:63) 16: C++ convenience archivesskipped (convenience.at:103) 17: F77 convenience archivesskipped (convenience.at:110) 18: FC convenience archives skipped (convenience.at:170) 19: Java convenience archives skipped (convenience.at:230) 20: Link order test.skipped (link-order.at:105) 21: Link order of deplibs. skipped (link-order2.at:124) 22: Failure tests ok 23: shlibpath_overrides_runpath skipped (shlibpath.at:66) 24: Runpath in libtool library filesok 25: static linking flags for programs skipped (static.at:177) 26: Export test skipped (export.at:159) 27: sys_lib_search_path ok 28: indirect convenienceskipped (indirect_deps.at:64) 29: indirect uninstalledskipped (indirect_deps.at:113) 30: static library contains static library expected failure (archive-in-archive.at:49) 31: execute modeok 32: localized compiler messages ok DESTDIR tests 33: Simple DESTDIR install skipped (destdir.at:70) 34: DESTDIR with in-package deplibs skipped (destdir.at:127) Support for older m4 interface. 35: AM_PROG_LIBTOOL skipped (old-m4-iface.at:87) 36: AC_WITH_LTDLskipped (old-m4-iface.at:156) Libtool subdir-objects support. 37: C subdir-objectsFAILED (am-subdir.at:80) 38: C++ subdir-objects FAILED (am-subdir.at:148) Libltdl functionality. 39: lt_dlexit unloading libsskipped (lt_dlexit.at:154) 40: lt_dlopenadvise library loading FAILED (lt_dladvise.at:323) 41: enforced lib prefix skipped (need_lib_prefix.at:170) Standalone Libltdl. 42: compiling softlinked libltdlok 43: compiling copied libltdlok 44: installable libltdl ok 45: linking libltdl without autotools skipped (standalone.at:87) Subproject Libltdl. 46: compiling softlinked libltdlok 47: compiling copied libltdlok 48: installable libltdl ok 49: linking libltdl without autotools skipped (subproject.at:117) Nonrecursive Automake Libltdl. 50: compiling softlinked libltdlFAILED (nonrecursive.at:93) 51: compiling copied libltdlFAILED (nonrecursive.at:117) 52: installable libltdl FAILED (nonrecursive.at:143) Recursive Automake Libltdl. 53: compiling softlinked libltdlFAILED (recursive.at:71) 54: compiling copied libltdlFAILED (recursive.at:91) 55: installable libltdl FAILED (recursive.at:113) C++ template tests. 56: simple template testskipped (template.at:92) 57: template test with subdirs skipped (template.at:243) Constructors. 58: C++ static
Re: Feature request: setting env vars for binary wrappers
On Thu, 2008-04-17 at 23:38 +0300, Roumen Petrov wrote: I think that all above is out of libtool scope. It's is exceptional project specific (lets skip cross-compilation environment) and is subject of project regression test suite. The project is responsible to set appropriate test environment before to run regression test. Please let me know when I don't understand request properly. It's not about regression testing at all. It's about making binaries in uninstalled builds work. For example, many GNOME applications need to load their UI from XML files. If you build and not install them and try to run from the build dir, not surprisingly, the XML file is not found at destination, and the program will fail to start. With my proposed additions, programs can for example check for an env var for an alternative prefix, and the Makefile.am can pass that information to libtool to put into the wrapper. Then running from uninstalled build will work. If running uninstalled build is not a goal, why bother about LD_LIBRARY_PATH'ing the uninstalled library path at all? Roumen Cheers, -- behdad http://behdad.org/ Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety. -- Benjamin Franklin, 1759
Re: Feature request: setting env vars for binary wrappers
On Thu, 17 Apr 2008, Behdad Esfahbod wrote:
If running uninstalled build is not a goal, why bother about
LD_LIBRARY_PATH'ing the uninstalled library path at all?
I feel your pain. As a workaround, for my own software I add a small
script which exports environment variables prior to running each test.
This is an extra shim layer that should not have to exist. The script
(attached) has substitutions applied to it prior to use. In the
future, I might include similar replicated code in each test script so
that the tests are completely stand-alone and run a bit faster.
An unfortunate thing is that not all software is configured the same.
For example, my own software supports independent configuration for
the location of different types of files. A single top prefix
environment variable would not directly work with my application while
it might work for others. A single top prefix would require that the
application know the structure of the build tree, which is likely a
bad thing since it may not match the install tree.
It seems to me that the proper place to fix this is at the Automake
level since Automake's weak support for tests is the true cause of the
problem.
Bob
==
Bob Friesenhahn
[EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
# !/bin/sh
# Copyright (C) 2003, 2004 GraphicsMagick Group
#
# This program is covered by multiple licenses, which are described in
# Copyright.txt. You should have received a copy of Copyright.txt with this
# package; otherwise see http://www.graphicsmagick.org/www/Copyright.html.
#
# Execute a program with the environment required to execute it using
# files from the source and build directory. This helps avoid needing to
# install GraphicsMagick before testing it.
#
# Written by Bob Friesenhahn [EMAIL PROTECTED] December 2003
#
top_srcdir='@abs_top_srcdir@'
top_builddir='@abs_top_builddir@'
MAGICK_CODER_MODULE_PATH='@MAGICK_CODER_MODULE_PATH@'
MAGICK_CONFIGURE_SRC_PATH='@MAGICK_CONFIGURE_SRC_PATH@'
MAGICK_CONFIGURE_BUILD_PATH='@MAGICK_CONFIGURE_BUILD_PATH@'
MAGICK_FILTER_MODULE_PATH='@MAGICK_FILTER_MODULE_PATH@'
DIRSEP='@DIRSEP@'
PATH=${top_builddir}/utilities:${PATH}
if test -n $VERBOSE
then
echo $@
fi
env \
LD_LIBRARY_PATH=${top_builddir}/magick/.libs:${LD_LIBRARY_PATH} \
MAGICK_CODER_MODULE_PATH=${MAGICK_CODER_MODULE_PATH} \
MAGICK_CONFIGURE_PATH=${MAGICK_CONFIGURE_BUILD_PATH}${DIRSEP}${MAGICK_CONFIGURE_SRC_PATH}
\
MAGICK_FILTER_MODULE_PATH=${MAGICK_FILTER_MODULE_PATH} \
PATH=${PATH} \
$@
Re: Feature request: setting env vars for binary wrappers
On Thu, 2008-04-17 at 16:40 -0500, Bob Friesenhahn wrote: An unfortunate thing is that not all software is configured the same. For example, my own software supports independent configuration for the location of different types of files. A single top prefix environment variable would not directly work with my application while it might work for others. A single top prefix would require that the application know the structure of the build tree, which is likely a bad thing since it may not match the install tree. It doesn't matter whether your app works with one prefix or not. That's not being proposed here. What is proposed is adding feature that lets application developer pass any env-var or cmd-line arguments that they need to the uninstalled binary. You get to decide what to set. It seems to me that the proper place to fix this is at the Automake level since Automake's weak support for tests is the true cause of the problem. As I said, my use case is not running tests at all. For tests I use automake's TESTS_ENVIRONMENT variable and it works great. For example I have: TESTS_ENVIRONMENT = CAIRO_XFAIL_TESTS=$(XFAIL_TESTS:$(EXEEXT)=) CAIRO_TEST_TARGET=$(TARGETS) CAIRO_TEST_TARGET_EXCLUDE=$(TARGETS_EXCLUDE) My use case is when users (or myself) want to run the pango/pango-view/pango-view binary, by hand. I can put a pango-view.sh by its side, but users won't run it. We went through this all two years ago :). -- behdad http://behdad.org/ Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety. -- Benjamin Franklin, 1759
Re: Feature request: setting env vars for binary wrappers
On Fri, 2008-04-18 at 00:45 +0300, Roumen Petrov wrote:
Behdad Esfahbod wrote:
On Thu, 2008-04-17 at 23:38 +0300, Roumen Petrov wrote:
I think that all above is out of libtool scope.
It's is exceptional project specific (lets skip cross-compilation
environment) and is subject of project regression test suite.
The project is responsible to set appropriate test environment
before
to run regression test.
Please let me know when I don't understand request properly.
It's not about regression testing at all. It's about making
binaries in
uninstalled builds work. For example, many GNOME applications need
to
load their UI from XML files. If you build and not install them and
try
to run from the build dir, not surprisingly, the XML file is not
found
at destination, and the program will fail to start. With my
proposed
additions, programs can for example check for an env var for an
alternative prefix, and the Makefile.am can pass that information to
libtool to put into the wrapper. Then running from uninstalled
build
will work.
If running uninstalled build is not a goal, why bother about
LD_LIBRARY_PATH'ing the uninstalled library path at all?
Roumen
Cheers,
Exactly, libtool do home work and set LD_LIBRARY_PATH
(DYLD_LIBRARY_PATH
, LIBRARY_PATH, PATH and etc. host/platform specific environment
library
search paths) but for application specific environment is
author/project
responsibility. I see that you understand case when a library isn't
installed at specified(system) location. This library will load
dependent libraries from default(system) library search path. So the
wrapper script help correct library to be loaded so the libtool home
work is done.
Sure. The request is: since libtool already has this machinery for a
wrapper in place, can you please expose it to application developers so
they can benefit from it too?.
But if user run directly an application installed in non-default
location the user is responsible to set environment.
I'm not talking about application installed in non-default location.
I'm talking about uninstalled application.
If its a regression/unit test the correct application environment has
to
be set in Makefile{|.in|.am} and the program/library will inherit it.
No, it's not a test suite. It's a real, legitimate application the user
has built. Now he wants to run it before doing make install.
Roumen
--
behdad
http://behdad.org/
Those who would give up Essential Liberty to purchase a little
Temporary Safety, deserve neither Liberty nor Safety.
-- Benjamin Franklin, 1759
Re: add cegcc support
* Vincent Torri wrote on Thu, Apr 17, 2008 at 06:31:34PM CEST: The README file explains it: run make check VERBOSE=yes TESTS=tests/demo-shared.test tests/demo-make.test tests/demo-exec.test Also, please run the other half of the tests (the new testsuite) using make check-local and post tests/testsuite.log, please. You can run both with make -k check. Here are the log of the test suite. After it, the output of the 2 failed tests (there are 2 now, after some modifications of the patch) What about tests/testsuite.log? Asking for the third time now.
Re: Feature request: setting env vars for binary wrappers
Behdad Esfahbod wrote: On Tue, 2006-06-13 at 17:00 -0400, Ralf Wildenhues wrote: [ this is: http://thread.gmane.org/gmane.comp.gnu.libtool.bugs/5319 or http://lists.gnu.org/archive/html/bug-libtool/2006-03/msg00013.html ] Hi Behdad, everyone, Hi again, Sorry for the delay again. So, yeah, replying after two years... I know. Hope you still remember this issue and the patch is not too stale by now... * Behdad Esfahbod wrote on Thu, Mar 09, 2006 at 04:59:38PM CET: On Wed, 1 Mar 2006, Ralf Wildenhues wrote: * Behdad Esfahbod wrote on Wed, Mar 01, 2006 at 05:15:17AM CET: This is a feature request for libtool. Let me describe the problem as I face it, so you may have a workaround for it already: I'm using libtool in the Pango text rendering engine. The Pango library accesses a file in $prefix/etc/pango/pangorc to find its modules at run-time. Now all I want to do is to be able to run the examples in pango/example when Pango is not installed. The easiest way that can be done is to set the PANGO_RC_PATH envvar to a special pangorc file. Another is to pass the --pangorc path-to-pango-rc to the examples if they understand it. What I need from libtool is to let me do one of these things in the wrapper it creates for uninstalled binaries. [ my concerns were: we don't always build a wrapper ATM. ] Sure, a wrapper is created if need be. And this feature can be just another reason to create a wrapper. If we can find an answer to this question to define coherent semantics, I'm all for it. Ok, this is my view of the problem: Running uninstalled programs requires some modifications to the environment. One is to make sure that the uninstalled libraries are linked. Other changes may be needed, on a per application basis. Libtool is free to choose how to implement these. So far it has only supported the library path modification, and implemented that using a wrapper script on some platforms, or using rpath on others (right?) More or less, yes. The same goes with the other modifications. They can be easily implemented using a wrapper. So if there are such modifications requested, a wrapper should be used. Yep. And that's really the easiest solution: as soon as extra arguments or extra environment variables are passed, we always build a wrapper. My current workaround has been making pango-view accept a --pangorc path-to-pangorc parameter and defaulting to ./pangorc. But that opens up a known security problem... So I really want to fix it the right way. I don't quite understand how this fixes any security problems... but here you go with a suggestion (against current CVS HEAD). The attached patch implements two new link flags, -wrapper-arg and -wrapper-env, to prepend arguments to programs, and modify their environment. They will force creating a (shell) wrapper. Here's the hairy part about it: somebody may eventually want to extend the C wrapper that is currently used on w32 to encompass all the functionality that the shell wrapper currently has. And while I don't have current plans about this, I still don't want to add any unnecessary obstackles to it. Fair enough. So whatever we add to the shell wrapper should be doable easily in a C wrapper as well. This led me to add these restrictions: the -wrapper-env works like putenv: you pass an argument `var=value', the variable will be exported, the value will _not_ be interpreted by the shell any more. For now you cannot unset variables (this is to cater for hosts with a shell that does not support `unset'), and, e.g., -wrapper-env 'foo=$bar' will cause the environment variable `foo' to contain the four characters $, b, a, and r, not the contents of the variable $bar. But make variables are expanded, right? Similarly, -wrapper-arg will add exactly one literal argument to the exec. I've put suitable quoting in place for this to work with most kinds of input, and of course a test to this end. What do you think? OK for HEAD right now, or do you think this is too intrusive now? Sounds good to me. Can it finally go in?! I think we should not backport this to 1.5.x, it is a new feature. Cheers, Ralf Cheers, I think that all above is out of libtool scope. It's is exceptional project specific (lets skip cross-compilation environment) and is subject of project regression test suite. The project is responsible to set appropriate test environment before to run regression test. Please let me know when I don't understand request properly. Roumen
Re: Feature request: setting env vars for binary wrappers
Behdad Esfahbod wrote:
On Thu, 2008-04-17 at 23:38 +0300, Roumen Petrov wrote:
I think that all above is out of libtool scope.
It's is exceptional project specific (lets skip cross-compilation
environment) and is subject of project regression test suite.
The project is responsible to set appropriate test environment before
to run regression test.
Please let me know when I don't understand request properly.
It's not about regression testing at all. It's about making binaries in
uninstalled builds work. For example, many GNOME applications need to
load their UI from XML files. If you build and not install them and try
to run from the build dir, not surprisingly, the XML file is not found
at destination, and the program will fail to start. With my proposed
additions, programs can for example check for an env var for an
alternative prefix, and the Makefile.am can pass that information to
libtool to put into the wrapper. Then running from uninstalled build
will work.
If running uninstalled build is not a goal, why bother about
LD_LIBRARY_PATH'ing the uninstalled library path at all?
Roumen
Cheers,
Exactly, libtool do home work and set LD_LIBRARY_PATH (DYLD_LIBRARY_PATH
, LIBRARY_PATH, PATH and etc. host/platform specific environment library
search paths) but for application specific environment is author/project
responsibility. I see that you understand case when a library isn't
installed at specified(system) location. This library will load
dependent libraries from default(system) library search path. So the
wrapper script help correct library to be loaded so the libtool home
work is done.
But if user run directly an application installed in non-default
location the user is responsible to set environment.
If its a regression/unit test the correct application environment has to
be set in Makefile{|.in|.am} and the program/library will inherit it.
Roumen
Re: Feature request: setting env vars for binary wrappers
Behdad Esfahbod wrote:
[SNIP]
But if user run directly an application installed in non-default
location the user is responsible to set environment.
I'm not talking about application installed in non-default location.
I'm talking about uninstalled application.
There is no significant difference !
If its a regression/unit test the correct application environment has
to
be set in Makefile{|.in|.am} and the program/library will inherit it.
No, it's not a test suite. It's a real, legitimate application the user
has built. Now he wants to run it before doing make install.
And if application don't read environment, next request is libtool
wrapper script to pass arguments to application command line.
The whole idea is libtool overkill.
Roumen
Re: Feature request: setting env vars for binary wrappers
On Fri, 2008-04-18 at 02:32 +0300, Roumen Petrov wrote:
Behdad Esfahbod wrote:
[SNIP]
But if user run directly an application installed in non-default
location the user is responsible to set environment.
I'm not talking about application installed in non-default location.
I'm talking about uninstalled application.
There is no significant difference !
I thought there is. The former is not supported, while I'm under the
impression that the latter is.
If its a regression/unit test the correct application environment has
to
be set in Makefile{|.in|.am} and the program/library will inherit it.
No, it's not a test suite. It's a real, legitimate application the user
has built. Now he wants to run it before doing make install.
And if application don't read environment, next request is libtool
wrapper script to pass arguments to application command line.
The argument passing is part of the patch too. But one or the other is
enough, because the application developer can use whatever is available
to them. Currently, there is no way to fix this problem with autotools.
With the proposed changes, there will be. That's all.
The whole idea is libtool overkill.
Fair enough. Just suggest an alternative please, instead of acting as
if the problem does not exist.
Roumen
--
behdad
http://behdad.org/
Those who would give up Essential Liberty to purchase a little
Temporary Safety, deserve neither Liberty nor Safety.
-- Benjamin Franklin, 1759
Re: add cegcc support
Hello Vincent, * Vincent Torri wrote on Fri, Apr 18, 2008 at 06:51:45AM CEST: What about tests/testsuite.log? Thank you very much for providing this. # -*- compilation -*- 37. am-subdir.at:33: testing ... [...] checking build system type... x86_64-unknown-linux-gnu checking host system type... ./am-subdir.at:78: exit code was 1, expected All failing tests fail due to config.sub files which are not up to date. Can you please go ahead and the config.sub file that comes with your installed Automake package to contain your proposed changes? It should be located at $automake_prefix/share/automake-1.10/config.sub. And then please rerun 'make check-local' and repost testsuite.log, there should be few failures left. Thank you, Ralf
Re: add cegcc support
* Vincent Torri wrote on Fri, Apr 18, 2008 at 07:25:01AM CEST: All failing tests fail due to config.sub files which are not up to date. Can you please go ahead and the config.sub file that comes with your installed Automake package to contain your proposed changes? It should be located at $automake_prefix/share/automake-1.10/config.sub. And then please rerun 'make check-local' and repost testsuite.log, there should be few failures left. indeed, 3 remaining failure: Two of them are expected failures. For the third, try the patch below. If it works, then no reason to post the testsuite.log again. I will then apply your patch after we have sorted out the git mess. Thanks! Ralf diff --git a/tests/lt_dladvise.at b/tests/lt_dladvise.at index 617539f..4ea41e5 100644 --- a/tests/lt_dladvise.at +++ b/tests/lt_dladvise.at @@ -286,7 +286,7 @@ dlopenable='resident local global' # - # case $host_os in -cygwin* | mingw*) +cygwin* | mingw* | cegcc*) # These hosts do not support linking without -no-undefined CPPFLAGS=$CPPFLAGS -DHAVE_UNDEFINED_SYMBOLS=0 ;;
