Re: "Run tests with low max_cmd_len" on MSYS/MSVC
* Bob Friesenhahn wrote on Thu, Jan 22, 2009 at 02:04:07AM CET: > On Wed, 21 Jan 2009, Ralf Wildenhues wrote: >> >> We may need to think about speeding up func_to_host_path, e.g., by not >> forking for arguments that don't need conversion, or by converting >> several paths with a constant amount of forks. But that can be done >> separately. > > There is also the possibility of building a translation cache which is > preserved across libtool invokations. The only concern is if the mounts > get changed after the cache is generated. Indeed, that seems like a viable possibility, too. I still think just excluding obviously safe cases will help in the vast majority of practical cases: e.g.: - file names without any directory separators, - relative file names (in those cases where no conversion between directory separators is needed). Cheers, Ralf
Re: "Run tests with low max_cmd_len" on MSYS/MSVC
Den 2009-01-21 22:54, skrev Ralf Wildenhues: Hi Peter, * Peter Rosin wrote on Wed, Jan 21, 2009 at 02:26:36AM CET: This patch together with [1] and [2] will make "Run tests with low max_cmd_len" on MSYS/MSVC behave the same as the individual tests. The patch fixes a couple more of the /abs/path issues already fixed in [1] and [2]. However, I fear that these hunks may be more controversial, as they touch code that affects other platforms as well. Well, for non-w32 systems the patch is pretty obviously ok, as func_to_host_path doesn't do much there, and not slowly. :-) For Cygwin and MinGW, it will likely make things much slower, since this will cause lots of forks. Indeed. From a correctness perspective, the patch looks good to go, though. For MinGW yes, for Wine this is probably not needed (Wine allows opening both posix and win32 files in its CreateFile implementation), and for Cygwin it is probably at least theoretically wrong (but it might work, I don't know how liberal open is in Cygwin...) As soon as you try to use Cygwin to launch native Win32 programs you are on thin ice. So, libtool on Cygwin should IMHO assume that the MinGW toolchain is a Cygwin hosted MinGW targeting toolchain. If the MinGW toolchain is like that - and I believe that both the soon historic "gcc -mno-cygwin" as well as the new "i686-pc-mingw32-gcc" toolchains are - then libtool has no business tampering with the posix paths using "cygpath -w" in these cases. Maybe there should be a second (optional) arg to func_to_host_path that indicates why you want to convert the path? Or perhaps a new func_to_toolchain_path that only converts the path if the toolchain isn't really for $host? I.e. is instead running in some kind of emulator or in some other subsystem or otherwise has a different notion of paths (like MSVC on MSYS). But when to use func_to_toolchain_path would be system specific as it is only needed on MSYS when MSYS for one reason or another fails to convert the path (an /absolute/path is stored in a command file or appears after some prefix on the command line that MSYS doesn't recognize, e.g. -out:/absolute/path). On Cygwin it would be needed for all /absolute/paths passed to the toolchain. On Wine it wouldn't be wrong to do as on Cygwin, but it would perhaps be overkill to always convert as Wine handles posix paths in CreateFile. Cheers, Peter
Re: "Run tests with low max_cmd_len" on MSYS/MSVC
On Wed, 21 Jan 2009, Ralf Wildenhues wrote: We may need to think about speeding up func_to_host_path, e.g., by not forking for arguments that don't need conversion, or by converting several paths with a constant amount of forks. But that can be done separately. There is also the possibility of building a translation cache which is preserved across libtool invokations. The only concern is if the mounts get changed after the cache is generated. Bob == Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: "Run tests with low max_cmd_len" on MSYS/MSVC
* Charles Wilson wrote on Wed, Jan 21, 2009 at 11:11:22PM CET:
> Ralf Wildenhues wrote:
>
> > We may need to think about speeding up func_to_host_path, e.g., by not
> > forking for arguments that don't need conversion, or by converting
> > several paths with a constant amount of forks. But that can be done
> > separately.
>
> FWIW, as part of re-writing this:
> [PATCH] [cygwin]: Add cross-compile support to cwrapper
> http://lists.gnu.org/archive/html/libtool-patches/2009-01/msg6.html
> I plan to split up the various case $host/case $build clauses in the
> func_to_host_path functions into separate functions. Then, using a
> configure test determine which function should be called via a libtool
> variable $build_to_host_path_cmd and $build_to_host_pathlist_cmd.
>
> The default value will be a function that simply assigns
> build_to_host_path_result=$1
>
> Then, we can target specific functions for speedups as needed.
>
> Ok plan?
Sounds like an idea.
Note that in sh one can also just define functions conditionally:
if $cond; then
func_foo ()
{
...
}
else
func_foo ...
also note that we have a section in libtool.m4 where we already define
some per-system functions which are then added literally to libtool.
That way, libtool doesn't ever contain the ones not intended for this
setup.
Cheers,
Ralf
Re: "Run tests with low max_cmd_len" on MSYS/MSVC
Ralf Wildenhues wrote: > We may need to think about speeding up func_to_host_path, e.g., by not > forking for arguments that don't need conversion, or by converting > several paths with a constant amount of forks. But that can be done > separately. FWIW, as part of re-writing this: [PATCH] [cygwin]: Add cross-compile support to cwrapper http://lists.gnu.org/archive/html/libtool-patches/2009-01/msg6.html I plan to split up the various case $host/case $build clauses in the func_to_host_path functions into separate functions. Then, using a configure test determine which function should be called via a libtool variable $build_to_host_path_cmd and $build_to_host_pathlist_cmd. The default value will be a function that simply assigns build_to_host_path_result=$1 Then, we can target specific functions for speedups as needed. Ok plan? -- Chuck
Re: "Run tests with low max_cmd_len" on MSYS/MSVC
Hi Peter, * Peter Rosin wrote on Wed, Jan 21, 2009 at 02:26:36AM CET: > This patch together with [1] and [2] will make "Run tests with > low max_cmd_len" on MSYS/MSVC behave the same as the individual > tests. > > The patch fixes a couple more of the /abs/path issues already > fixed in [1] and [2]. However, I fear that these hunks may be > more controversial, as they touch code that affects other > platforms as well. Well, for non-w32 systems the patch is pretty obviously ok, as func_to_host_path doesn't do much there, and not slowly. :-) For Cygwin and MinGW, it will likely make things much slower, since this will cause lots of forks. From a correctness perspective, the patch looks good to go, though. We may need to think about speeding up func_to_host_path, e.g., by not forking for arguments that don't need conversion, or by converting several paths with a constant amount of forks. But that can be done separately. Thanks, Ralf > [1] http://lists.gnu.org/archive/html/libtool-patches/2009-01/msg00090.html > [2] http://lists.gnu.org/archive/html/libtool-patches/2009-01/msg00092.html > > 2009-01-21 Peter Rosin > > Convert paths to host format > * libltdl/config/ltmain.m4sh (func_mode_link): When exporting > symbols, linking and creating archives using command files (or > response files), make sure that both the name of the command > file and the content are made up of file names in a format > appropriate for the host. Fixes stresstest.at on MSYS/MSVC when > run with low command line length.
"Run tests with low max_cmd_len" on MSYS/MSVC
Hi!
This patch together with [1] and [2] will make "Run tests with
low max_cmd_len" on MSYS/MSVC behave the same as the individual
tests.
The patch fixes a couple more of the /abs/path issues already
fixed in [1] and [2]. However, I fear that these hunks may be
more controversial, as they touch code that affects other
platforms as well.
Cheers,
Peter
[1] http://lists.gnu.org/archive/html/libtool-patches/2009-01/msg00090.html
[2] http://lists.gnu.org/archive/html/libtool-patches/2009-01/msg00092.html
2009-01-21 Peter Rosin
Convert paths to host format
* libltdl/config/ltmain.m4sh (func_mode_link): When exporting
symbols, linking and creating archives using command files (or
response files), make sure that both the name of the command
file and the content are made up of file names in a format
appropriate for the host. Fixes stresstest.at on MSYS/MSVC when
run with low command line length.
diff --git a/libltdl/config/ltmain.m4sh b/libltdl/config/ltmain.m4sh
index 36dd3fb..7047339 100644
--- a/libltdl/config/ltmain.m4sh
+++ b/libltdl/config/ltmain.m4sh
@@ -6833,11 +6833,13 @@ EOF
save_libobjs=$libobjs
save_output=$output
output=${output_objdir}/${output_la}.nm
- libobjs=$nm_file_list_spec$output
+ func_to_host_path "$output"
+ libobjs=$nm_file_list_spec$func_to_host_path_result
delfiles="$delfiles $output"
func_verbose "creating $NM input file list: $output"
for obj in $save_libobjs; do
- $ECHO "$obj"
+ func_to_host_path "$obj"
+ $ECHO "$func_to_host_path_result"
done > "$output"
eval cmd=\"$cmd1\"
func_show_eval "$cmd" 'exit $?'
@@ -6999,10 +7001,12 @@ EOF
fi
for obj
do
- $ECHO "$obj" >> $output
+ func_to_host_path "$obj"
+ $ECHO "$func_to_host_path_result" >> $output
done
delfiles="$delfiles $output"
- output=$firstobj\"$file_list_spec$output\"
+ func_to_host_path "$output"
+ output=$firstobj\"$file_list_spec$func_to_host_path_result\"
else
if test -n "$save_libobjs"; then
func_verbose "creating reloadable object files..."
@@ -7839,10 +7843,12 @@ EOF
elif test -n "$archiver_list_spec"; then
func_verbose "using command file archive linking..."
for obj in $oldobjs; do
- $ECHO \""$obj"\"
+ func_to_host_path "$obj"
+ $ECHO "\"$func_to_host_path_result\""
done > $output_objdir/$libname.libcmd
save_oldobjs="$oldobjs"
- oldobjs=" $archiver_list_spec$output_objdir/$libname.libcmd"
+ func_to_host_path "$output_objdir/$libname.libcmd"
+ oldobjs=" $archiver_list_spec$func_to_host_path_result"
eval cmds=\"\$old_archive_cmds\"
oldobjs="$save_oldobjs"
else
