-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Please keep replies on the list.
According to Kyle Sallee on 10/21/2006 1:43 AM: > I was looking at > /libltdl/config/ltmain.m4sh > one of the first uses of tem_deplibs has a nearby call to > func_stripname '-R' '' "$libdir" > That uses a separate invocation of sed > everytime -R is encountered. > Sort of expensive if there is more than one occurance. > > Why not fix them all at once? > > For example the xrpath string could still be built > without calling func_stripname. > Finally, > > if test -n "$xrpath"; > then xrpath="` echo "$xrpath" | sort -u | sed "s/^-R//"`" > fi > > Well, my point is that since the func_stripname > requires an invocation of sed then all the -R > should be stripped with a single invocation of sed > rather than on every occurance where $libdir begins with -R > > BTW, I like how func_stripname reutnrs $func_stripname_results > instead of echoing it and then having to gather it using an > assignment to the output of a subshell. > That definitely helps speed it up. > > If dependency_libs is a long list it might still be fast > to process it in two separate pipes of commands. > The first can gather the -R > the second can gather the rest. > > Instead of doing temp_deplibs"$temp_deplibs $libdir" ;; > on each itteration... > dependency_libs="` echo $temp_deplibs | sed "s/-R[^ ]*::g" `" > > Counting echo as a process... > The above requires 3 process. > Likewise, dependency_libs could be determined 3 processes. > 3 processes to determine duplicate dependency libs. > > Even if that seems a bit expensive for > calls of libtool not involving a lot of librarie > at least the potential invocations of sed > to remove the -R could be reduced to only 1. > > There are places for matching duplicates such as: > > *" $deplib "*) specialdeplibs="$specialdeplibs $deplib" ;; > *" $temp_xrpath "*) ;; > > That is incredibly expensive to do when the string is long. > > ltmain.m4sh > > is rather complex. > > Perhaps more functions would be helpful? > > What about if the different major tasks of the > libtool script were put in separate scripts? > The invocation of libtool could determine which one > of those other scripts should be called depending upon > the desired task and the platform the task will run on? > > The complexity currently is daunting. > I see where it scoops out the -R > and then a page down the -L and so forth... > > Encoding the methods in BASH using the for > loops and case statements and so forth are rather > lengthy and complex compared to a 3 process method > involving a subshell assignment. > However, it could be further reduced if > instead of assigning those to bash variables if > the output was piped to a temporary file. > > echo $temp_deplibs | sed "s/-R[^ ]*::g" > /tmp/libtool_something.$tmp > > one invocation of echo one invocation of sed. > The sed statement could be more complex to eliminate both the -R and -L > or whatever else may be undesirable in creating that list. > > While I prefer one item per line instead of all items on a line > separated by space > the all items on a line separated by a space has one advantage. > I know this works in bash, but I do not know about /bin/sh > > read < file > > The above will put into $REPLY the entire first line of file > without using a subshell. > It is faster than... > REPLY="$(< file )" > It is faster than > REPLY="$( cat file )" > > But it only retrieves one line. > > But using multi-lined streams means > that sort and uniq could become useful. > Those case statements to identify duplicates is signifigantly slow. > > It seems as if libtool could be modified to both invoke less processes > when running and also eliminate some of the slow shell syntax. > > I understand that you are strongly against using forks > where the equivilent can be coded without it. > But in some of those instances I have examined there > are multiple invocations of programs such as sed > which means that there are several fork potentials. > If no greater change is made then at least those > instances where an item is modified before it is > put into a a string of variables it can be made so > that the string of variables is made and then the > entire string is modified with a single invocation of sed. > > In that 26M trace I have. > I counted > 2432 instances of /bin/sed > That is a lot of forking. > > What would be helpful is to look for instances of code > that is within a loop where a fork is performed > for each instance of a certain case statement. > Then, build the string without the fork > and after the string is built if it even exists > then run sed once to make it the way it should be, yes? > > I would like to know more about how costly forks are > on the platforms where it is seriously slow, please. > 2432 invocations of /bin/sed during execution of libtool > has got to slow it like a sucking chest wound. > > Here is my conjecture... > Substitution of the code that with potential to cause excessive forking > along with substitution of snowball code, > along with substitution of those very expensive case > statements to detect duplicates could gain 2 magnitudes > of speed increase on really challenging tasks like koffice. > At the same time it on platforms that are fork expensive > could cause an insignificant increase in execution time > on very light libtool requests that would ordinarlly execute > quickly anyway. > > If a change in the code makes the best cases as much as > 10x slower and the worst cases 100x faster then the > average effect of the change will be faster compilations involving libtool. > > I am not saying that replacement of slow sections of bash code > along with the potentials for multiple forks to call sed for example > with a single 3 fork invocation will be faster in every case and every > platform. > But it should crush the libtool time spent > on koffice objects from seconds to miliseconds. > > Do you suppose anyone but me would agree that optimizing libtool > for best performance on large tasks is better than having libtool > optimized for best performance on simple tasks and horrifically > slow performance on large tasks? Afterall, who cares if it runs > immeasurably slower on the fastest and easiests tasks if > knowticible seconds are shaved off the large tasks like koffice? > I do not care if execution of libtool takes 0.1 seconds or 0.2 seconds. > However, I do care if execution of libtool takes 0.5 seconds and 25 > seconds. > Does that make sense? > > libtool is currently optimized for very small and easy tasks. > But why optimize for the case which is already destined to be fast? > Instead it aught to be optimized for the large tasks so that compilation > of a single object file does not require some 2000 invocations of sed, > right? > > Lemme know if I have a snowball's chance > in hell of eliminating libtool's snowballing? > I expect some will adamantly oppose the use > of grep, sed, sort, uniq in pipes to accomplish > what can be done in bash with for loops and case statements. > But if the large libtool tasks are to be speed up by magnitudes > then the paradigm should be shifted. > 5 definite forks is better than the possibility > of no forks with the possibility of hundreds of forks. > > For the simpliest libtool tasks more forks may be executed than before. > But for the more complex tasks hundreds of forks could be avoided. > The way it stands now there is a huge disparity between > libtool execution speed for small and large tasks. > However, if optimized for large tasks then > libtool would still execute fast for small tasks, > and the large tasks would execute only a fraction > slower than the small tasks since nearly the same > amount of forks would be executed in both cases. > > The way it is now large tasks not only snowball, > not only lag across those case statements to locate > duplicates, but they also incur a huge amount of forks. > 2000+ invocations of /bin/sed is unreasonable, yes? > - -- Life is short - so eat dessert first! Eric Blake [EMAIL PROTECTED] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.1 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFOhSn84KuGfSFAYARAlJsAJ9OUUSqmxLocb8EDpU6CMLO/A+5aQCePFAL DYTSXCbUqnLj/Irvj7gROw4= =x5n3 -----END PGP SIGNATURE----- _______________________________________________ http://lists.gnu.org/mailman/listinfo/libtool
