Re: Tools to modify shared libraries
On Tue, Jun 17, 2003 at 06:38:29AM -0700, Joe Kelsey wrote: Basically, what I want to do is remove several entries from the *front* of the dynamic section. Actually, I would settle for just removing all of a certain tag (such as DT_NEEDED) from the dynamic section. I'm very interested, having a working Flash 6 would be great! Isn't there a way to change these into bogus dependencies, or dependencies on a FreeBSD shared object or something? --Stijn -- In the force if Yoda's so strong, construct a sentence with words in the proper order then why can't he? pgp0.pgp Description: PGP signature
Re: Tools to modify shared libraries
Stijn Hoop wrote: On Tue, Jun 17, 2003 at 06:38:29AM -0700, Joe Kelsey wrote: Basically, what I want to do is remove several entries from the *front* of the dynamic section. Actually, I would settle for just removing all of a certain tag (such as DT_NEEDED) from the dynamic section. I'm very interested, having a working Flash 6 would be great! Isn't there a way to change these into bogus dependencies, or dependencies on a FreeBSD shared object or something? No. I removed the dependencies by manually editing the .so with emacs... Now, here is a list of the undefined symbols beginning with __: 598: 79 FUNCGLOBAL DEFAULT UND [EMAIL PROTECTED] (4) 939: 231 FUNCGLOBAL DEFAULT UND [EMAIL PROTECTED] (2) 1129: 109 FUNCGLOBAL DEFAULT UND [EMAIL PROTECTED] (4) 1656: 172 FUNCWEAK DEFAULT UND __deregister_frame_info 1703: 815 FUNCGLOBAL DEFAULT UND [EMAIL PROTECTED] (2) 1952: 815 FUNCGLOBAL DEFAULT UND [EMAIL PROTECTED] (2) 2133: 4 OBJECT GLOBAL DEFAULT UND [EMAIL PROTECTED] (2) 2763: 157 FUNCWEAK DEFAULT UND [EMAIL PROTECTED] (9) 3161: 4 OBJECT GLOBAL DEFAULT UND [EMAIL PROTECTED] (2) 3319: 1642 FUNCGLOBAL DEFAULT UND [EMAIL PROTECTED] (2) 3376: 26 FUNCGLOBAL DEFAULT UND __terminate 3624: 129 FUNCWEAK DEFAULT UND __register_frame_info 3716: 0 NOTYPE WEAK DEFAULT UND __gmon_start__ The old flash library had these symbols: 34: 0 NOTYPE GLOBAL DEFAULT UND __builtin_vec_new 36: 0 NOTYPE GLOBAL DEFAULT UND __builtin_vec_delete 1163: 0 NOTYPE GLOBAL DEFAULT UND __errno_location flashpluginwrapper provides the two __builtin references. I do not think we need to provide the WEAK symbols. We do need to provide the others, specifically __write, __ctype_toupper, __ctype_b, __assert_failure, __xstate, __fxstate, and __strtoul_internal. Anyone want to start modifying flashpluginwrapper? /Joe ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
Apparently, On Mon, Jun 16, 2003 at 03:39:19PM -0700, Joe Kelsey said words to the effect of; Has anyone ever come across general-purpose tools for modifying shared libraries? What I want to do is to edit the list of needed shared libraries to correct the common mistakes that developers make in creating shared objects with large lists of shared libraries. Specifically, I want to modify linux-flashplugin6/libflashplayer.so to remove all of the idiotic references to shared libraries that the Flash 6 developers added. Since this is a plugin for Mozilla, it does not need to specify any extra shared libraries especially since Mozilla has already loaded all of them! I want to do this to make Flash 6 work with flashpluginwrapper. If I can remove the list of needed shared libraries from the DYNAMIC section of the shared library, everyone will be that much closer to a real flash 6 plugin for BSD. I've needed to do similar things to remove the interp section from static binaries with full dynamic symbol tables. I just wrote a C program that mmapped the elf file, parsed the headers and changed the p_type of the PT_INTERP program header entries to PT_NULL. You may be able to do something similar by changing the d_tag of the DT_NEEDED entries to some value larger than DT_COUNT so they will be ignored by the dynamic linker. Jake ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
Brandon S. Allbery KF8NH wrote: On Mon, 2003-06-16 at 18:39, Joe Kelsey wrote: Has anyone ever come across general-purpose tools for modifying shared libraries? What I want to do is to edit the list of needed shared libraries to correct the common mistakes that developers make in creating shared objects with large lists of shared libraries. GNU objcopy --- but it's a bit *too* general; you'd need to extract the .dynamic section, edit it using some binary editing tool, and re-add it. I have tried and failed... I even looked at the code. objcopy is horrible as far as it goes. Maybe if I could learn more about bfd in general, there might be something I could do with objcopy, but it does not look promising. I doubt there are any tools of the kind you're looking for because the details are too different between different systypes; even if someone had developed one, it's just as likely to be for Linux or Solaris as for *BSD, and as a result wouldn't be particularly useful. All 32-bit Elf libraries look the same as far as the DT_NEEDED entries in the DYNAMIC section. In fact, *all* ELF libraries look the same relative to the size of the string table offset used for entries. The DYNAMIC section is the simplest of all sections, generally consisting of a tag and a value, both in the native word size (e.g., 32 or 64). I am surprised that no one has done this yet. I tried elfsh, but the version in ports is too old. I got the most recent version from the website, but it has a lot of linuxisms in it. Basically, what I want to do is remove several entries from the *front* of the dynamic section. Actually, I would settle for just removing all of a certain tag (such as DT_NEEDED) from the dynamic section. /Joe ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
On Tue, Jun 17, 2003 at 06:38:29AM -0700, Joe Kelsey wrote: Basically, what I want to do is remove several entries from the *front* of the dynamic section. Actually, I would settle for just removing all of a certain tag (such as DT_NEEDED) from the dynamic section. It's more constructive to fix the linker than it is to patch the ELF files created by it. The linker knows which libraries are really needed and should be able to create the minimal list of (true) dependencies. -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
Marcel Moolenaar wrote: On Tue, Jun 17, 2003 at 06:38:29AM -0700, Joe Kelsey wrote: Basically, what I want to do is remove several entries from the *front* of the dynamic section. Actually, I would settle for just removing all of a certain tag (such as DT_NEEDED) from the dynamic section. It's more constructive to fix the linker than it is to patch the ELF files created by it. The linker knows which libraries are really needed and should be able to create the minimal list of (true) dependencies. This cannot be accomplished by fixing the linker. The issue is one of attempting to use a *linux* shared library in a native application. Have you ever lookad at the flashpluginwarpper port? It provides a library to perload which intercepts the linux syscalls and translates them to bsd syscalls to allow linux shared libraries (specifically, the linux flash library) in native binaries. This works fine for the old flash plugin since that shared library did not include any DT_NEEDED entires in its .dynamic section. However, the new Flash 6 linux shared library has a number of explicit references to linux-only shared libraries as DT_NEEDED references in its .dynamic section. This prevents us from using the flashpluginwrapper trick to allow use of Flash 6 in native Mozilla. Do you understand now? /Joe ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
On Tue, Jun 17, 2003 at 08:49:23AM -0700, Joe Kelsey wrote: Marcel Moolenaar wrote: On Tue, Jun 17, 2003 at 06:38:29AM -0700, Joe Kelsey wrote: Basically, what I want to do is remove several entries from the *front* of the dynamic section. Actually, I would settle for just removing all of a certain tag (such as DT_NEEDED) from the dynamic section. It's more constructive to fix the linker than it is to patch the ELF files created by it. The linker knows which libraries are really needed and should be able to create the minimal list of (true) dependencies. This cannot be accomplished by fixing the linker. The issue is one of attempting to use a *linux* shared library in a native application. Linux uses the same linker (GNU ld). Fixing the linker will have the same effect on Linux as it will have on FreeBSD and hence will prevent unnecessary dependencies in Linux libraries to Linux libraries and thus remove the need to patch ELF files in the long run. -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
Marcel Moolenaar wrote: On Tue, Jun 17, 2003 at 08:49:23AM -0700, Joe Kelsey wrote: Marcel Moolenaar wrote: On Tue, Jun 17, 2003 at 06:38:29AM -0700, Joe Kelsey wrote: Basically, what I want to do is remove several entries from the *front* of the dynamic section. Actually, I would settle for just removing all of a certain tag (such as DT_NEEDED) from the dynamic section. It's more constructive to fix the linker than it is to patch the ELF files created by it. The linker knows which libraries are really needed and should be able to create the minimal list of (true) dependencies. This cannot be accomplished by fixing the linker. The issue is one of attempting to use a *linux* shared library in a native application. Linux uses the same linker (GNU ld). Fixing the linker will have the same effect on Linux as it will have on FreeBSD and hence will prevent unnecessary dependencies in Linux libraries to Linux libraries and thus remove the need to patch ELF files in the long run. The problem cannot be resolved by fixing ld. The problem arises from people who specify unnecessary libraries on their ld command lines. ld cannot tell the difference between a required library and an unnecessary library at link time. Only the runtime loader can do this, and the FreeBSD runtime loader has numerous problems in this area. Sometimes a shared library has to include a required library reference since the shared library author knows in advance that the programs using the library do not have the same requirements. Most often, clueless programmers reference every single library ever known to them on their linker command lines in the off-chance that it *might* make a difference at load time. However, this leads to shared libraries containing references to explicitlyly versioned libraries, thus leading to the proliferation of unnecessarily versioned shared libraries, etc., etc. I can think of many reasons for post-linker tools to modify shared libraries. Most of them involve fixing the egregious mistakes perpetrated by users who have not thought out what requirements a shared library outgt to carry with it. Others involve patching and other kinds of code modification. /Joe ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
On Tue, Jun 17, 2003 at 09:13:04AM -0700, Joe Kelsey wrote: It's more constructive to fix the linker than it is to patch the ELF files created by it. The linker knows which libraries are really needed and should be able to create the minimal list of (true) dependencies. This cannot be accomplished by fixing the linker. The issue is one of attempting to use a *linux* shared library in a native application. Linux uses the same linker (GNU ld). Fixing the linker will have the same effect on Linux as it will have on FreeBSD and hence will prevent unnecessary dependencies in Linux libraries to Linux libraries and thus remove the need to patch ELF files in the long run. The problem cannot be resolved by fixing ld. The problem arises from people who specify unnecessary libraries on their ld command lines. ld cannot tell the difference between a required library and an unnecessary library at link time. Yes it can. Symbol resolution is a fundamental part in linking. Hence, the linker has all the information it needs to filter the gratuitously long list of libraries programmers tend to give it and keep the libraries that actually contributed to the link. -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
Marcel Moolenaar wrote: Yes it can. Symbol resolution is a fundamental part in linking. Hence, the linker has all the information it needs to filter the gratuitously long list of libraries programmers tend to give it and keep the libraries that actually contributed to the link. I know of no way to do this in the case of shared libraries. When linking shared libraries, the linker *cannot* resolve any references to other shared libraries other than list them in the .dynamic section with some sort of tag such as DT_NEEDED. Please explain to me how the linker can prune the shared library list at link time. Sorry that this has veered off into a dead-end. I promise to cut the mailing list from any further discussions on this dead-end thread. /Joe ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
On Tue, Jun 17, 2003 at 09:31:54AM -0700, Joe Kelsey wrote: Marcel Moolenaar wrote: Yes it can. Symbol resolution is a fundamental part in linking. Hence, the linker has all the information it needs to filter the gratuitously long list of libraries programmers tend to give it and keep the libraries that actually contributed to the link. I know of no way to do this in the case of shared libraries. When linking shared libraries, the linker *cannot* resolve any references to other shared libraries other than list them in the .dynamic section with some sort of tag such as DT_NEEDED. Please explain to me how the linker can prune the shared library list at link time. If a symbol is unresolved, it must be present in one of the libraries on the link line. If the symbol is in an archive library, you pull in the code. Otherwise, if it's in a shared library, you record the dependency on that library. In the end you have no unresolved symbols, all code from archives has been linked in and you have a complete list of dependencies that is a subset of the libraries given on the command line. For partial linking (incremental linking) the list of unresolved symbols does not have to be empty. -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
On Tue, 17 Jun 2003 09:01:41 -0700 Marcel Moolenaar [EMAIL PROTECTED] wrote: On Tue, Jun 17, 2003 at 08:49:23AM -0700, Joe Kelsey wrote: Linux uses the same linker (GNU ld). Fixing the linker will have the same effect on Linux as it will have on FreeBSD and hence will prevent unnecessary dependencies in Linux libraries to Linux libraries and thus remove the need to patch ELF files in the long run. LD putting a library in DT_NEEDED regardless of whether or not library exports any required symbols as long as it appears on command line is a feature, not a bug AFAIK. -- Alexander Kabaev ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
In the last episode (Jun 17), Marcel Moolenaar said: On Tue, Jun 17, 2003 at 01:02:36PM -0400, Alexander Kabaev wrote: On Tue, 17 Jun 2003 09:01:41 -0700 Marcel Moolenaar [EMAIL PROTECTED] wrote: On Tue, Jun 17, 2003 at 08:49:23AM -0700, Joe Kelsey wrote: Linux uses the same linker (GNU ld). Fixing the linker will have the same effect on Linux as it will have on FreeBSD and hence will prevent unnecessary dependencies in Linux libraries to Linux libraries and thus remove the need to patch ELF files in the long run. LD putting a library in DT_NEEDED regardless of whether or not library exports any required symbols as long as it appears on command line is a feature, not a bug AFAIK. It's a bug because DT_NEEDED serves the purpose of recording library dependencies. Any library that does not contribute to symbol resolution is by definition not a dependency. Hence, its presence in DT_NEEDED only makes the dependency information wrong. Dependency information that's wrong is untrustworthy and unreliable and thus unusable. Hence, a bug. Immediate consequences of broken dependency information is the increased startup time of shared binaries, the restriction in use of libraries in cases where they can be used and the obstruction in replacing libraries with different implementations by possibly causing artificial conflicts due to unnecessary loading of libraries. Only explicit user directives should allow adding libraries to DT_NEEDED regardless of whether there's actually a dependency. Dependencies can change after a program is linked, though. How about the contrived case of a program needing the openpty function, so -lutil is linked in. Then 6 months later openpty is moved to libc, making the dependency on libutil unneeded. The end result is the same as if a new program is unnecessarily linked with -lutil, but cannot be fixed with ld because at the time it was linked, the first program actually did need libutil. -- Dan Nelson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
On Tue, Jun 17, 2003 at 01:02:36PM -0400, Alexander Kabaev wrote: On Tue, 17 Jun 2003 09:01:41 -0700 Marcel Moolenaar [EMAIL PROTECTED] wrote: On Tue, Jun 17, 2003 at 08:49:23AM -0700, Joe Kelsey wrote: Linux uses the same linker (GNU ld). Fixing the linker will have the same effect on Linux as it will have on FreeBSD and hence will prevent unnecessary dependencies in Linux libraries to Linux libraries and thus remove the need to patch ELF files in the long run. LD putting a library in DT_NEEDED regardless of whether or not library exports any required symbols as long as it appears on command line is a feature, not a bug AFAIK. It's a bug because DT_NEEDED serves the purpose of recording library dependencies. Any library that does not contribute to symbol resolution is by definition not a dependency. Hence, its presence in DT_NEEDED only makes the dependency information wrong. Dependency information that's wrong is untrustworthy and unreliable and thus unusable. Hence, a bug. Immediate consequences of broken dependency information is the increased startup time of shared binaries, the restriction in use of libraries in cases where they can be used and the obstruction in replacing libraries with different implementations by possibly causing artificial conflicts due to unnecessary loading of libraries. Only explicit user directives should allow adding libraries to DT_NEEDED regardless of whether there's actually a dependency. -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tools to modify shared libraries
On Mon, 2003-06-16 at 18:39, Joe Kelsey wrote: Has anyone ever come across general-purpose tools for modifying shared libraries? What I want to do is to edit the list of needed shared libraries to correct the common mistakes that developers make in creating shared objects with large lists of shared libraries. GNU objcopy --- but it's a bit *too* general; you'd need to extract the .dynamic section, edit it using some binary editing tool, and re-add it. I doubt there are any tools of the kind you're looking for because the details are too different between different systypes; even if someone had developed one, it's just as likely to be for Linux or Solaris as for *BSD, and as a result wouldn't be particularly useful. -- brandon s allbery [openafs/solaris/japh/freebsd] [EMAIL PROTECTED] system administrator [linux/heimdal/too many hats] [EMAIL PROTECTED] electrical and computer engineering KF8NH carnegie mellon university [better check the oblivious first -ke6sls] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
