Re: binutils symbol hiding and versioning (was Re: [PATCH] note the __sF change in src/UPDATING)
Loren James Rittle wrote: FYI, the libstdc++-v3 maintainers on the FSF side are only guaranteeing forward ABI compatibility of any sort if libstdc++.so is built with symbol versioning and symbol hiding. FWIW: symbol versioning is incredibly broken. It attempts to do in UNIX what interface versioning does in Windows, through the use of class factories accessed via IUnknown. The point of the exercise is to allow multiple simultaneous versions of an interface to be exported by a single library. The main reason this is bas is the same reason that Novell must, in their SDK's, support interface versions all the way back to NetWare 1.x: in order to hve the largest possible user base, a software vendor would have to be stupid to write to anything other than the lowest common denominator of interfaces: it's really stupid to limit your customers to running NetWare 4.2 or above, when there is still such a large installed base of 3.x, 4.0, and 4.1 versions of NetWare. The only thing you do when you do that is to disenfranchise postential customers for your product. Windows uses this approach because they do not have the concept of shared object versioning; VCRTL32.DLL is VCRTL32.DLL, no matter what, so a version change that permits old applications to continue working is a the same DLL, plus extensions (since there is no version in the file name,multiple versions can only exist simultaneously if they exist in the same file). It would be a very big mistake to actually utilize symbol name versioning on a UNIX system, and buy into this model, even if the idea was supported by the tools. That Linux has bought into the idea of using this is, frankly, Linux's problem, and they are going to regret it in the future as much, or more, than they regretted implementing shared library support in the SVR3.2 way, of linking libraries to fixed locations, and carving up chunks of the user virtual address space to implement them, back when they first supported shared libraries. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: binutils symbol hiding and versioning (was Re: [PATCH] note the __sF change in src/UPDATING)
Loren James Rittle wrote: FWIW: symbol versioning is incredibly broken. It attempts to do in UNIX what interface versioning does in Windows, through the use of class factories accessed via IUnknown. You might be absolutely correct in general. However, please read http://gcc.gnu.org/onlinedocs/libstdc++/abi.txt . It is clear that symbol versioning is not being used at all like you supposed it might have been (mis-)used. To provide multiple API's in a single shared library image, to avoid a number of images being necessary? That's exactly how it's being used; they call it age gracefully, but what that means is that multiple API versions remain supported for a very, very long time, without having seperate libraries involved. Technically, the symbol decoration used in C++ is also in error; it's there simply to avoid having to change the object file format to accomodate interface attributes seperately from the symbol name (just as adding an @@version name suffix does). If this were not ther, the linker could automatically create glue code for doing argument coercion, and it would save a lot of code that currently has to be supplied by programmers. FWIW: There is no concept of IUnknown or implementation factories (and, yes, I do understand those concepts) in how libstdc++.so (v3) is using symbol versioning. I invite you to take a close look at how that library is actually using symbol hiding and versioning before you attempt to cast your judgment. If you have informed comments, then please direction them to [EMAIL PROTECTED] not me personally (as a libstdc++-v3 maintainer, I will read them over there like all others). I'm well aware of how it's used; the IUnknown reference was an analogy; the document you refer to specifically states that it's to avoid a proliferation of shared library files. That's exactly the purpose of IUnknown version information for class factories, as well. Part of the problem here is that GCC dropped the minor version number from shared libraries in binutils, and FreeBSD and Linux followed suit. Now this turns out to be a mistake, and rather than admitting the mistake, instead now we have more decoration occurring in the symbol name to fake up another orthogonal namespace. Traditional UNIX systems have minor versions on shared libraries to address this, and bump major versions only if existing interfaces change, not when interfaces are added (thus program foo linked against lib.so.M.N works just fine against lib.so.M.N+K). If you don't like the Microsoft DLL version analogy, a different analogy is the Aztec C support for directories, by naming files with their complete path, and treating the character / as a path component seperator in order to achieve a namespace escape, when the Mac FS didn't support directory hierarchies. In all cases, what's happening is a namespace incursion in order to permit coexistance of otherwise conflicting implementations. Short summation: We only mark the first version of the library that a new symbol is added. E.g. there will never be [EMAIL PROTECTED] and also [EMAIL PROTECTED] The first release after an ELF library version number bump resets all tags to be the same. As clearly documented, libstdc++.so (v3) will bump the major version number just like FreeBSD does on installed shared libraries to express other sorts of C++ compiler or library ABI change. This still fails when the OS version does not bump each time the compiler version bumps. I guess this is OK, if you are a compiler vendor, but less OK, when you are an OS vendor. 8-). If the system compiler of FreeBSD still wanted to install multiple versions of libstdc++.so (v3) with major number bumps for other reasons (because it is considered safer for compatibility by the system designers), that would be quite fine. But completely ignoring the symbol hiding features will make the FreeBSD C++ system compiler and environment worse than the Linux version and worse than a g++ installed from equivalent FSF sources IMHO (since we will leak all sorts of internal implementation symbols that are not suppose to influence user application link behavior). Anyways, Alex was already going to look into trying this for the FreeBSD 5.0 system compiler so hopefully this will not be the case. No, it will make it incompatible, which is rather annoying, but it's an introduced incompatability that came from the compiler, and we shouldn't pretend it isn't. In any case, the issue was in attempting to prevent the exposure of data interfaces, and symbols not part of the defined API; this is still goinf to cause problems for the reasons this discussion came up in the first place: other language compilers that need to link against system libraries, and share implementation instances so that they can be linked against foreign object files that use the same underlying implementation. For the purposes of this discussion, that's the stdio implementation, as exposed
binutils symbol hiding and versioning (was Re: [PATCH] note the __sF change in src/UPDATING)
Doug Rabson wrote: In the windows world, all this is handled by having a strict list of explicit symbol exports, either in the source code using syntax extensions or with a file supplied to the linker. I'm not sure whether binutils supports this kind of thing but it would allow us to cut down the set of symbols exported from libc.so. It does. Proof by example: libstdc++.so and libgcc_s.so built from sources included with FSF gcc 3.1 and all forward releases support this feature on FreeBSD. It works fine on FreeBSD 4.7 and 5.0 (with system copy of binutils in both cases). I don't know the exact version of binutils that added this feature but it was before 2.12[.2]. The ld features being used are both symbol versioning and symbol hiding. (Speaking of which, I am reminded to insert this advisement of a related minor issue with the current system compiler:) libstdc++.so as built along with the system compiler does not make use of these features yet on FreeBSD 5.0 since the system-static Makefiles to build the compiler-related libraries have not been updated to account for the feature and many extra symbols are exported as a result. libstdc++.so uses this linker features to present a more stable ABI, no matter the concrete implementation under the hood of the library. I am hoping that the gcc maintainer on the FreeBSD system-side will reconsider this issue before 5.0 is released. FYI, the libstdc++-v3 maintainers on the FSF side are only guaranteeing forward ABI compatibility of any sort if libstdc++.so is built with symbol versioning and symbol hiding. Also, I believe it may be an error to even ship a libstdc++.so unless it is linked against libgcc_s.so (and it currently is not so linked since we don't build libgcc_s.so). As I understand it, if a user of the system compiler attempts to build a stand-alone C++ shared library (i.e. no link against libstdc++.so) but then later uses it in an application that also uses libstdc++.so, EH will fail unless libgcc_s.so was properly built and used. I understand that FreeBSD circa 1996(?) attempted to use a shared libgcc.so; libgcc_s.so is similar but different. For one libgcc_s.so is only part of what was libgcc.a; i.e. all code still links against libgcc.a when libgcc_s.so is built and used properly. Regards, Loren To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Sheldon Hearn [EMAIL PROTECTED] writes: : If the final word on this whole issue is You can't run binaries : compiled for 4.x-RELEASE on 5.x-RELEASE then we should start puckering : up. : : Developers tend to remember these things and you don't have to screw : them too many times before they remove FreeBSD downloads from their web : sites and replace them with a brief message explaining that We don't : have time to cope with FreeBSD's disregard for backward compatibility. : : At that point, whining about how they shouldn't have been fiddling with : these symbols in the first place is bound to be a fruitless exercise. This is why I put __sF back, and made sure that 4.x had the right 'forward compatible' libraries. It is also why I backported the __std{in,out,err}p symbols. I hope that minor incompatibility within a minor release is offset by __sF not being referencede by any more libraries. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On (2002/11/08 18:13), Daniel Eischen wrote: The problem is that you cannot have 4.x packages and 5.x packages co-mingled on the same system. that's what I'm trying to fix. You'd have to rebuild the 4.x packages before they are fixed. I don't think this is a show-stopper. Just recompile all your ports or use the pre-built 5.0 packages. If we keep thinking only in terms of software for which source is available, we'll soon find ourselves completely estranged from commercial ventures and we can kiss any hope of serious vendor support for FreeBSD good-bye. If the final word on this whole issue is You can't run binaries compiled for 4.x-RELEASE on 5.x-RELEASE then we should start puckering up. Developers tend to remember these things and you don't have to screw them too many times before they remove FreeBSD downloads from their web sites and replace them with a brief message explaining that We don't have time to cope with FreeBSD's disregard for backward compatibility. At that point, whining about how they shouldn't have been fiddling with these symbols in the first place is bound to be a fruitless exercise. Ciao, Sheldon. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Friday 08 November 2002 11:13 pm, Daniel Eischen wrote: On Fri, 8 Nov 2002, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Daniel Eischen [EMAIL PROTECTED] writes: : All the ports are going to be rebuilt for the release anyways, : so this doesn't affect fresh installs, correct? It is only a : problem when mixing older 4.x and 5.0 libraries/binaries with : __sF-free libc (if I understand things correctly). The problem is that you cannot have 4.x packages and 5.x packages co-mingled on the same system. that's what I'm trying to fix. You'd have to rebuild the 4.x packages before they are fixed. I don't think this is a show-stopper. Just recompile all your ports or use the pre-built 5.0 packages. : This is 5.0; it is a major release and there will be some flies : in the ointment. I say bite the bullet now -- don't wait. : If we want to provide an optional compatability hack to libc : so that folks can compile it with __sF support, then I think : that is better than leaving __sF in the release, perhaps : with a mktemp(3)-like warning if possible (?). You'd need a run-time warning for this to be effective. I'm not sure that ld.so can do this right now. Could you put __sF in it's own file, and put the error in a .init section? We don't care about static binaries, right? They shouldn't have a problem. This is not a fly in the pointment, but rather a major incompatibility that makes it impossible to have a reasonable mix. If it's really a hassle for folks, then just provide the optional compatability hack and make them rebuild libc. Or provide a pre-built version that doesn't get installed by default. So what you are saying, basically, is that we should ship a release, for the first time ever, which can't run old binaries. Sorry, that isn't acceptable. The correct and robust way of doing things is to stop creating binaries (in both 4.x and 5.x) that reference __sF, then wait a full release cycle for the change to propagate. We can then remove the symbol in 6.x. In general, its a very poor idea to simply remove a feature that was supported in the last release of a package. The normal route is to deprecate (but still support) the feature in one release and remove it in the next. -- Doug Rabson Mail: [EMAIL PROTECTED] Phone: +44 20 8348 6160 To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Sat, 9 Nov 2002, Doug Rabson wrote: On Friday 08 November 2002 11:13 pm, Daniel Eischen wrote: On Fri, 8 Nov 2002, M. Warner Losh wrote: This is not a fly in the pointment, but rather a major incompatibility that makes it impossible to have a reasonable mix. If it's really a hassle for folks, then just provide the optional compatability hack and make them rebuild libc. Or provide a pre-built version that doesn't get installed by default. So what you are saying, basically, is that we should ship a release, for the first time ever, which can't run old binaries. Sorry, that isn't acceptable. The correct and robust way of doing things is to stop creating binaries (in both 4.x and 5.x) that reference __sF, then wait a full release cycle for the change to propagate. We can then remove the symbol in 6.x. If you put an optional compatibility hack in libc, then folks can still use it, plus they will be informed that it is going away at some point. I don't think removing it from stdio.h is good enough. Unless there's a way of putting out an error message at run-time, we need some other way of being a little bit of a nuisance. In general, its a very poor idea to simply remove a feature that was supported in the last release of a package. The normal route is to deprecate (but still support) the feature in one release and remove it in the next. Do kld's from 4.x work on -current? Just curious -- I don't really know. -- Dan Eischen To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Saturday 09 November 2002 4:28 pm, Daniel Eischen wrote: On Sat, 9 Nov 2002, Doug Rabson wrote: On Friday 08 November 2002 11:13 pm, Daniel Eischen wrote: On Fri, 8 Nov 2002, M. Warner Losh wrote: This is not a fly in the pointment, but rather a major incompatibility that makes it impossible to have a reasonable mix. If it's really a hassle for folks, then just provide the optional compatability hack and make them rebuild libc. Or provide a pre-built version that doesn't get installed by default. So what you are saying, basically, is that we should ship a release, for the first time ever, which can't run old binaries. Sorry, that isn't acceptable. The correct and robust way of doing things is to stop creating binaries (in both 4.x and 5.x) that reference __sF, then wait a full release cycle for the change to propagate. We can then remove the symbol in 6.x. If you put an optional compatibility hack in libc, then folks can still use it, plus they will be informed that it is going away at some point. I don't think removing it from stdio.h is good enough. Unless there's a way of putting out an error message at run-time, we need some other way of being a little bit of a nuisance. I don't see the need for this. We aren't supporting the original broken API (i.e. __sF in stdio.h). We do need to support the broken ABI for another release cycle otherwise we break everyone who tries to upgrade from 4.x to 5.x, which is a bad thing. In general, its a very poor idea to simply remove a feature that was supported in the last release of a package. The normal route is to deprecate (but still support) the feature in one release and remove it in the next. Do kld's from 4.x work on -current? Just curious -- I don't really know. The kernel ABI is hopeless. It changes almost daily :-(. At one time, I thought I could change this but these days, I don't think anyone except me cares about having a stable ABI in the kernel. -- Doug Rabson Mail: [EMAIL PROTECTED] Phone: +44 20 8348 6160 To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
Doug Rabson wrote: The kernel ABI is hopeless. It changes almost daily :-(. At one time, I thought I could change this but these days, I don't think anyone except me cares about having a stable ABI in the kernel. I care. It's almost the most important thing to be able to build anything of value. But a stable API is required for a stable ABI. It does absolutely no good to build a cathedral, if, by the time you are done, the roads have moved so that there is no road which leads to it. The next most important thing is to document it, so that people can use it. All coordinates are expressed in logitude and lattitute. North is that way. No cows are permitted in the city limits. All cats and dogs which are uncollared and tagged will be captured and spayed and neutered. If you can't agree on a coordinate system (OLDCARD? NEWCARD? REDCARD? BLUECARD?), then at least agree to get rid of data interfaces; today, people are adding to them with sysctl exposured of structs, as if it's not going to be a problem. It's a problem. It's one thing to say The cathedral is a about a mile yonder of of Johnson's barn, just down the road a piece from Smithy, which is about a league yonder, and then follow the creek for a spell, and something else entirely to say The cathedral is at 117 East 32st street; this is 12421 West 1st street. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Terry Lambert [EMAIL PROTECTED] writes: : If you can't agree on a coordinate system (OLDCARD? NEWCARD? : REDCARD? BLUECARD?), then at least agree to get rid of data : interfaces; Ironically, NEWCARD and OLDCARD are driver compatible because it doesn't use complex data types... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Thursday 07 November 2002 9:42 pm, Tim Kientzle wrote: Terry Lambert asked: Any chance we could get rid of all externally visable symbols that are not defined as being there by some standard, and not just __sF, since we are breaking the FORTRAN compiler and other third party code already? This cannot be entirely done if you still want to manage library bloat. In short, library routines have shared interfaces between them---common variables or common functions---that are internal to the library and should not be used by applications. To avoid this, you would have to bundle library functions together, which causes bloat. Worse, you would have to avoid or drastically limit your use of macros. (Any macro that uses one of these internal symbols generates a dependency in the compiled application.) It _would_ be a good idea to document any internal library symbols used by macros. Removing such symbols is a good way to break existing compiled applications. Library design involves a lot of tradeoffs. In the windows world, all this is handled by having a strict list of explicit symbol exports, either in the source code using syntax extensions or with a file supplied to the linker. I'm not sure whether binutils supports this kind of thing but it would allow us to cut down the set of symbols exported from libc.so. -- Doug Rabson Mail: [EMAIL PROTECTED] Phone: +44 20 8348 6160 To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
From [EMAIL PROTECTED] Fri Nov 8 02:45:04 2002 Date: Fri, 08 Nov 2002 00:39:35 -0700 (MST) To: [EMAIL PROTECTED] Subject: Re: [PATCH] note the __sF change in src/UPDATING From: M. Warner Losh [EMAIL PROTECTED] In message: [EMAIL PROTECTED] Ray Kohler [EMAIL PROTECTED] writes: : Hear hear, I agree. There's no need to expose what ought to be : private data to the world, especially when we can get the additional : benefit here of letting us play with the implementation. -current already does this. The problem is that we're trying to shoot the bad access in the head, and that is what is screwing people. So the problem isn't that we're trying to export private data to the world. Quite the contrary, we're trying to eliminate it and having growing pains. Exactly. That's why I'm arguing against putting __sF back (or adopting equally crapulent measures). Growing pains are a necessary evil. (I also agree that we probably ought to staticize any other things of this nature while we're at it and get the pain over with.) - @ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
Doug Rabson wrote: It _would_ be a good idea to document any internal library symbols used by macros. Removing such symbols is a good way to break existing compiled applications. Library design involves a lot of tradeoffs. In the windows world, all this is handled by having a strict list of explicit symbol exports, either in the source code using syntax extensions or with a file supplied to the linker. I'm not sure whether binutils supports this kind of thing but it would allow us to cut down the set of symbols exported from libc.so. Linux does it, so binutils supports it. Linux does this for kernel modules, and for libraries. On Linux, glibc2 (or whatever it's called this morning) does not export any symbols that it uses internally. In the general case, it's better to eat some small amount of overhead, than to export a data interface. Note that data interfaces get in the way of the LGPL relink clause for shared libraries. For example, if you had a GPL'ed program that was written in Modula 3 or FORTRAN 95, that had a reference to the __sF, then distribution of that program would demand distribution of the intermediate library source code, where normally it would not be required, due to the relink clause not being satisfied, since an attempt to link against the 5.0 libc would fail. Data interfaces are a pain in the ass legally, as well as making your code not work. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Ray Kohler [EMAIL PROTECTED] writes: : From [EMAIL PROTECTED] Fri Nov 8 02:45:04 2002 : Date: Fri, 08 Nov 2002 00:39:35 -0700 (MST) : To: [EMAIL PROTECTED] : Subject: Re: [PATCH] note the __sF change in src/UPDATING : From: M. Warner Losh [EMAIL PROTECTED] : : In message: [EMAIL PROTECTED] : Ray Kohler [EMAIL PROTECTED] writes: : : Hear hear, I agree. There's no need to expose what ought to be : : private data to the world, especially when we can get the additional : : benefit here of letting us play with the implementation. : : -current already does this. The problem is that we're trying to shoot : the bad access in the head, and that is what is screwing people. So : the problem isn't that we're trying to export private data to the : world. Quite the contrary, we're trying to eliminate it and having : growing pains. : : Exactly. That's why I'm arguing against putting __sF back (or : adopting equally crapulent measures). Growing pains are a necessary evil. : (I also agree that we probably ought to staticize any other things of : this nature while we're at it and get the pain over with.) Yes, but this is too painful. If we were going to do this, the time for the pain was 6-9 months ago, not just before the release. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
From [EMAIL PROTECTED] Fri Nov 8 11:30:05 2002 Date: Fri, 08 Nov 2002 09:27:32 -0700 (MST) To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [PATCH] note the __sF change in src/UPDATING From: M. Warner Losh [EMAIL PROTECTED] In message: [EMAIL PROTECTED] Ray Kohler [EMAIL PROTECTED] writes: : From [EMAIL PROTECTED] Fri Nov 8 02:45:04 2002 : Date: Fri, 08 Nov 2002 00:39:35 -0700 (MST) : To: [EMAIL PROTECTED] : Subject: Re: [PATCH] note the __sF change in src/UPDATING : From: M. Warner Losh [EMAIL PROTECTED] : : In message: [EMAIL PROTECTED] : Ray Kohler [EMAIL PROTECTED] writes: : : Hear hear, I agree. There's no need to expose what ought to be : : private data to the world, especially when we can get the additional : : benefit here of letting us play with the implementation. : : -current already does this. The problem is that we're trying to shoot : the bad access in the head, and that is what is screwing people. So : the problem isn't that we're trying to export private data to the : world. Quite the contrary, we're trying to eliminate it and having : growing pains. : : Exactly. That's why I'm arguing against putting __sF back (or : adopting equally crapulent measures). Growing pains are a necessary evil. : (I also agree that we probably ought to staticize any other things of : this nature while we're at it and get the pain over with.) Yes, but this is too painful. If we were going to do this, the time for the pain was 6-9 months ago, not just before the release. Yeah, that's true. I think unfortunately we ought to wait and do it next cycle, but when we do, do it completely and as quickly as possible. - @ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Fri, 8 Nov 2002, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Ray Kohler [EMAIL PROTECTED] writes: : From [EMAIL PROTECTED] Fri Nov 8 02:45:04 2002 : Date: Fri, 08 Nov 2002 00:39:35 -0700 (MST) : To: [EMAIL PROTECTED] : Subject: Re: [PATCH] note the __sF change in src/UPDATING : From: M. Warner Losh [EMAIL PROTECTED] : : In message: [EMAIL PROTECTED] : Ray Kohler [EMAIL PROTECTED] writes: : : Hear hear, I agree. There's no need to expose what ought to be : : private data to the world, especially when we can get the additional : : benefit here of letting us play with the implementation. : : -current already does this. The problem is that we're trying to shoot : the bad access in the head, and that is what is screwing people. So : the problem isn't that we're trying to export private data to the : world. Quite the contrary, we're trying to eliminate it and having : growing pains. : : Exactly. That's why I'm arguing against putting __sF back (or : adopting equally crapulent measures). Growing pains are a necessary evil. : (I also agree that we probably ought to staticize any other things of : this nature while we're at it and get the pain over with.) Yes, but this is too painful. If we were going to do this, the time for the pain was 6-9 months ago, not just before the release. All the ports are going to be rebuilt for the release anyways, so this doesn't affect fresh installs, correct? It is only a problem when mixing older 4.x and 5.0 libraries/binaries with __sF-free libc (if I understand things correctly). This is 5.0; it is a major release and there will be some flies in the ointment. I say bite the bullet now -- don't wait. If we want to provide an optional compatability hack to libc so that folks can compile it with __sF support, then I think that is better than leaving __sF in the release, perhaps with a mktemp(3)-like warning if possible (?). -- Dan Eischen To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Fri, Nov 08, 2002 at 12:17:00PM -0500, Daniel Eischen wrote: On Fri, 8 Nov 2002, M. Warner Losh wrote: Yes, but this is too painful. If we were going to do this, the time for the pain was 6-9 months ago, not just before the release. All the ports are going to be rebuilt for the release anyways, so this doesn't affect fresh installs, correct? It is only a problem when mixing older 4.x and 5.0 libraries/binaries with __sF-free libc (if I understand things correctly). This is 5.0; it is a major release and there will be some flies in the ointment. I say bite the bullet now -- don't wait. I agree with Dan. Let's do it now. My understanding is that 5.0 will be an early adopter release and production systems should run 4.7{8,9,..} until 5.1 is released. To accomplish the change, I think we need to do: 1. Install a complete set of 4.7 shared libs in COMPAT4X. This should porivde the necessary runtime compatibility with 4.x. 2. Bump all shared library on 5.0. This will get rid of any interdependencies among the libraries and it deals with the version number problems I detailed in the thread Ghost of __sF ... a couple a days ago. 3. Put a big fat WARNING in src/UPDATING about the problem 4. Put the same WARNING in /etc/motd, so people currently run -current will know to update their ports. 5. Broadcast the WARNING to appropriate mailing lists and newsgroups. -- Steve To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
+---[ Steve Kargl ]-- | | I agree with Dan. Let's do it now. My understanding is | that 5.0 will be an early adopter release and production | systems should run 4.7{8,9,..} until 5.1 is released. | | To accomplish the change, I think we need to do: | 1. Install a complete set of 4.7 shared libs in COMPAT4X. | This should porivde the necessary runtime compatibility | with 4.x. | 2. Bump all shared library on 5.0. This will get rid of | any interdependencies among the libraries and it deals | with the version number problems I detailed in the thread | Ghost of __sF ... a couple a days ago. | 3. Put a big fat WARNING in src/UPDATING about the problem | 4. Put the same WARNING in /etc/motd, so people currently | run -current will know to update their ports. | 5. Broadcast the WARNING to appropriate mailing lists and | newsgroups. 6. Assume Crash Position. -- Totally Holistic Enterprises Internet| | Andrew Milton The Internet (Aust) Pty Ltd | M:+61 416 022 411 | ACN: 082 081 472 ABN: 83 082 081 472 |[EMAIL PROTECTED]| Carpe Daemon To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Sat, Nov 09, 2002 at 04:16:11AM +1000, Andrew Kenneth Milton wrote: 6. Assume Crash Position. Thanks for your important contribution to a discussion which is addressing a rather serious problem. Here's the important part of the Ghost... thread. The following 4.7 libs make reference to __sF. libalias.so.4libbz2.so.1 libc.so.4libc_r.so.4 libcam.so.2 libcom_err.so.2 libcrypto.so.2 libdes.so.3 libdevstat.so.2 libdialog.so.4libedit.so.3 libfetch.so.3 libftpio.so.5libg2c.so.1 libgmp.so.3 libhistory.so.4 libipsec.so.1libisc.so.1 libkdb.so.3 libkrb.so.3 libkrb5.so.5 libkvm.so.2 libm.so.2libmp.so.3 libncp.so.1 libncurses.so.5 libopie.so.2 libpam.so.1 libpcap.so.2 libperl.so.3 libreadline.so.4 libroken.so.5 libskey.so.2 libsmb.so.1 libssh.so.2 libstdc++.so.3 libutil.so.3 Here's the fun part. The following 5.0 libraries have the same version number as their 4.x counterparts. Try running a 4.x app linked against one of these libaries on a 5.0 machine. You should also note that this list may not be complete list of libraries suffering this problem. libalias.so.4libbz2.so.1 libcam.so.2 libcom_err.so.2 libcrypto.so.2 libdes.so.3 libdialog.so.4 libfetch.so.3 libftpio.so.5libg2c.so.1 libhistory.so.4 libipsec.so.1 libisc.so.1 libkvm.so.2 libm.so.2libncp.so.1 libncurses.so.5 libopie.so.2 libpcap.so.2 libreadline.so.4 libsmb.so.1 libssh.so.2 libutil.so.3 -- Steve To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Daniel Eischen [EMAIL PROTECTED] writes: : All the ports are going to be rebuilt for the release anyways, : so this doesn't affect fresh installs, correct? It is only a : problem when mixing older 4.x and 5.0 libraries/binaries with : __sF-free libc (if I understand things correctly). The problem is that you cannot have 4.x packages and 5.x packages co-mingled on the same system. that's what I'm trying to fix. You'd have to rebuild the 4.x packages before they are fixed. : This is 5.0; it is a major release and there will be some flies : in the ointment. I say bite the bullet now -- don't wait. : If we want to provide an optional compatability hack to libc : so that folks can compile it with __sF support, then I think : that is better than leaving __sF in the release, perhaps : with a mktemp(3)-like warning if possible (?). You'd need a run-time warning for this to be effective. I'm not sure that ld.so can do this right now. This is not a fly in the pointment, but rather a major incompatibility that makes it impossible to have a reasonable mix. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Steve Kargl [EMAIL PROTECTED] writes: : On Fri, Nov 08, 2002 at 12:17:00PM -0500, Daniel Eischen wrote: : On Fri, 8 Nov 2002, M. Warner Losh wrote: : : : Yes, but this is too painful. If we were going to do this, the time : for the pain was 6-9 months ago, not just before the release. : : All the ports are going to be rebuilt for the release anyways, : so this doesn't affect fresh installs, correct? It is only a : problem when mixing older 4.x and 5.0 libraries/binaries with : __sF-free libc (if I understand things correctly). : : This is 5.0; it is a major release and there will be some flies : in the ointment. I say bite the bullet now -- don't wait. : : I agree with Dan. Let's do it now. My understanding is : that 5.0 will be an early adopter release and production : systems should run 4.7{8,9,..} until 5.1 is released. That's not a viable option. Even if 5.0 is an early adapter release, then that would argue for keeping __sF in libc to aid in the transition. : To accomplish the change, I think we need to do: : 1. Install a complete set of 4.7 shared libs in COMPAT4X. : This should porivde the necessary runtime compatibility : with 4.x. That's not true. You can't mix and match 4.x and 5.x packages right now. : 2. Bump all shared library on 5.0. This will get rid of : any interdependencies among the libraries and it deals : with the version number problems I detailed in the thread : Ghost of __sF ... a couple a days ago. Not a viable option. Unless you have patches that do this properly for ports, where we are getting screwed now. : 3. Put a big fat WARNING in src/UPDATING about the problem Been there, done that, didn't help. : 4. Put the same WARNING in /etc/motd, so people currently : run -current will know to update their ports. : 5. Broadcast the WARNING to appropriate mailing lists and : newsgroups. Been there, done that, didn't help. So in short, these plans won't help anything :-(. My plan is as follows: 1) Restore __sF to libc for 5.0. 2) Fix 4.x binaries so that __sF isn't referened in new binaries. This should have been done in Aug 2001, but wasn't. Depending on how things go, __sF will be removed in 5.1 or 5.2 after we have the transition period we had originally planned for 4.x, but implemented a year ago. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Steve Kargl [EMAIL PROTECTED] writes: : Here's the fun part. The following 5.0 libraries have the same : version number as their 4.x counterparts. Try running a 4.x : app linked against one of these libaries on a 5.0 machine. You : should also note that this list may not be complete list of : libraries suffering this problem. : : libalias.so.4libbz2.so.1 libcam.so.2 libcom_err.so.2 : libcrypto.so.2 libdes.so.3 libdialog.so.4 libfetch.so.3 : libftpio.so.5libg2c.so.1 libhistory.so.4 libipsec.so.1 : libisc.so.1 libkvm.so.2 libm.so.2libncp.so.1 : libncurses.so.5 libopie.so.2 libpcap.so.2 libreadline.so.4 : libsmb.so.1 libssh.so.2 libutil.so.3 You've also not listed all the package binaries that have similar problems. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
Andrew Kenneth Milton wrote: +---[ Steve Kargl ]-- | | I agree with Dan. Let's do it now. My understanding is | that 5.0 will be an early adopter release and production | systems should run 4.7{8,9,..} until 5.1 is released. | | To accomplish the change, I think we need to do: | 1. Install a complete set of 4.7 shared libs in COMPAT4X. | This should porivde the necessary runtime compatibility | with 4.x. | 2. Bump all shared library on 5.0. This will get rid of | any interdependencies among the libraries and it deals | with the version number problems I detailed in the thread | Ghost of __sF ... a couple a days ago. | 3. Put a big fat WARNING in src/UPDATING about the problem | 4. Put the same WARNING in /etc/motd, so people currently | run -current will know to update their ports. | 5. Broadcast the WARNING to appropriate mailing lists and | newsgroups. 6. Assume Crash Position. 7. Profit! -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
M. Warner Losh wrote: My plan is as follows: 1) Restore __sF to libc for 5.0. 2) Fix 4.x binaries so that __sF isn't referened in new binaries. This should have been done in Aug 2001, but wasn't. Depending on how things go, __sF will be removed in 5.1 or 5.2 after we have the transition period we had originally planned for 4.x, but implemented a year ago. Before anyone's head explodes, this will *NOT* generate *new* references to __sF. Warner is *NOT* talking about changing the definitions in stdio back to their old values. *NEW* code will get the *NEW* definitions, with certain specific exceptions. The exceptions are for new code linked against old libraries, or compiled with old tools, and linked agoinst those old tools' libraries (e.g. Modula 3 and FORTRAN 95). The issue this raises is that if/when the size of struct file changes, the old programs will quit working, unless *ALL* of the libraries which reference the obsoleted symbols varsions are bumped between 4.x and 5.x, and *ALL* of them are made available as part of the binary compatability package. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Fri, 8 Nov 2002, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Daniel Eischen [EMAIL PROTECTED] writes: : All the ports are going to be rebuilt for the release anyways, : so this doesn't affect fresh installs, correct? It is only a : problem when mixing older 4.x and 5.0 libraries/binaries with : __sF-free libc (if I understand things correctly). The problem is that you cannot have 4.x packages and 5.x packages co-mingled on the same system. that's what I'm trying to fix. You'd have to rebuild the 4.x packages before they are fixed. I don't think this is a show-stopper. Just recompile all your ports or use the pre-built 5.0 packages. : This is 5.0; it is a major release and there will be some flies : in the ointment. I say bite the bullet now -- don't wait. : If we want to provide an optional compatability hack to libc : so that folks can compile it with __sF support, then I think : that is better than leaving __sF in the release, perhaps : with a mktemp(3)-like warning if possible (?). You'd need a run-time warning for this to be effective. I'm not sure that ld.so can do this right now. Could you put __sF in it's own file, and put the error in a .init section? We don't care about static binaries, right? They shouldn't have a problem. This is not a fly in the pointment, but rather a major incompatibility that makes it impossible to have a reasonable mix. If it's really a hassle for folks, then just provide the optional compatability hack and make them rebuild libc. Or provide a pre-built version that doesn't get installed by default. -- Dan Eischen To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Daniel Eischen [EMAIL PROTECTED] writes: : On Fri, 8 Nov 2002, M. Warner Losh wrote: : : In message: [EMAIL PROTECTED] : Daniel Eischen [EMAIL PROTECTED] writes: : : All the ports are going to be rebuilt for the release anyways, : : so this doesn't affect fresh installs, correct? It is only a : : problem when mixing older 4.x and 5.0 libraries/binaries with : : __sF-free libc (if I understand things correctly). : : The problem is that you cannot have 4.x packages and 5.x packages : co-mingled on the same system. that's what I'm trying to fix. You'd : have to rebuild the 4.x packages before they are fixed. : : I don't think this is a show-stopper. Just recompile all your : ports or use the pre-built 5.0 packages. I disagree. : : This is 5.0; it is a major release and there will be some flies : : in the ointment. I say bite the bullet now -- don't wait. : : If we want to provide an optional compatability hack to libc : : so that folks can compile it with __sF support, then I think : : that is better than leaving __sF in the release, perhaps : : with a mktemp(3)-like warning if possible (?). : : You'd need a run-time warning for this to be effective. I'm not sure : that ld.so can do this right now. : : Could you put __sF in it's own file, and put the error in : a .init section? We don't care about static binaries, right? : They shouldn't have a problem. More details please. I'd love for there to be a way to know which binaries use __sF. : This is not a fly in the pointment, but rather a major incompatibility : that makes it impossible to have a reasonable mix. : : If it's really a hassle for folks, then just provide the : optional compatability hack and make them rebuild libc. : Or provide a pre-built version that doesn't get installed : by default. I'm not sure that I agree with that solution. __sF was supposed to die in -stable around 4.3, but the ball got dropped on the floor. If it had, then I wouldn't care so much now. I don't see it as a big deal to have it for 5.0, but not 5.1 or 5.2 depending... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Fri, 8 Nov 2002, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Daniel Eischen [EMAIL PROTECTED] writes: : On Fri, 8 Nov 2002, M. Warner Losh wrote: : : In message: [EMAIL PROTECTED] : Daniel Eischen [EMAIL PROTECTED] writes: : : All the ports are going to be rebuilt for the release anyways, : : so this doesn't affect fresh installs, correct? It is only a : : problem when mixing older 4.x and 5.0 libraries/binaries with : : __sF-free libc (if I understand things correctly). : : The problem is that you cannot have 4.x packages and 5.x packages : co-mingled on the same system. that's what I'm trying to fix. You'd : have to rebuild the 4.x packages before they are fixed. : : I don't think this is a show-stopper. Just recompile all your : ports or use the pre-built 5.0 packages. I disagree. No problem :-) : : This is 5.0; it is a major release and there will be some flies : : in the ointment. I say bite the bullet now -- don't wait. : : If we want to provide an optional compatability hack to libc : : so that folks can compile it with __sF support, then I think : : that is better than leaving __sF in the release, perhaps : : with a mktemp(3)-like warning if possible (?). : : You'd need a run-time warning for this to be effective. I'm not sure : that ld.so can do this right now. : : Could you put __sF in it's own file, and put the error in : a .init section? We don't care about static binaries, right? : They shouldn't have a problem. More details please. I'd love for there to be a way to know which binaries use __sF. I'm not even close to a shared library/linker expert, but I thought there was a way to have something run when an object file got loaded. From rtld(1): After all shared libraries have been successfully loaded, ld-elf.so.1 proceeds to resolve external references from both the main program and all objects loaded. A mechanism is provided for initialization routines to be called on a per-object basis, giving a shared object an opportunity to perform any extra set-up before execution of the program proper begins. This is useful for C++ libraries that contain static construc- tors. If you put __sF in it's own file with whatever is needed for the above, won't that work? -- Dan Eischen To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Fri, Nov 08, 2002 at 04:16:06PM -0700, M. Warner Losh wrote: I'd love for there to be a way to know which binaries use __sF. The following script run on your bin, sbin, lib, and libexec directories does a pretty decent job of finding files that contain refrences to __sF and listing the ports that use them (depend on portupgrade). -- Brooks #!/bin/sh sym=__sF for file in $*; do if [ -n `nm ${file} 21 | egrep ${sym}$` ]; then echo ${file} `pkg_which $file` fi done msg46400/pgp0.pgp Description: PGP signature
Re: [PATCH] note the __sF change in src/UPDATING
Quoting Brooks Davis [EMAIL PROTECTED]: | On Fri, Nov 08, 2002 at 04:16:06PM -0700, M. Warner Losh wrote: | I'd love for there to be a way to know which binaries use __sF. | | The following script run on your bin, sbin, lib, and libexec directories | does a pretty decent job of finding files that contain refrences to __sF | and listing the ports that use them (depend on portupgrade). | | -- Brooks | | | #!/bin/sh | | sym=__sF | | for file in $*; do | if [ -n `nm ${file} 21 | egrep ${sym}$` ]; then | echo ${file} `pkg_which $file` | fi | done | FWIW, while playing with the script I found libc_p.a from today's build. ed BTW, I did remove the $ from $*. I thought it was running too fast :-) -- - To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Fri, Nov 08, 2002 at 05:22:13PM -0800, [EMAIL PROTECTED] wrote: BTW, I did remove the $ from $*. I thought it was running too fast :-) You're supposed to pass it a list of files not run it in a directory. -- Brooks -- Any statement of the form X is the one, true Y is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 msg46402/pgp0.pgp Description: PGP signature
Re: [PATCH] note the __sF change in src/UPDATING
On Fri, Nov 08, 2002 at 05:05:23PM -0800, Brooks Davis wrote: On Fri, Nov 08, 2002 at 04:16:06PM -0700, M. Warner Losh wrote: I'd love for there to be a way to know which binaries use __sF. The following script run on your bin, sbin, lib, and libexec directories does a pretty decent job of finding files that contain refrences to __sF and listing the ports that use them (depend on portupgrade). #!/bin/sh sym=__sF for file in $*; do if [ -n `nm ${file} 21 | egrep ${sym}$` ]; then echo ${file} `pkg_which $file` fi done nm doesn't work on shared libraries. You can use strings to find __sF in shared libs. troutmask:kargl[201] cd /mnt/usr/lib/ troutmask:kargl[202] nm libm.a | grep sF U __sF troutmask:kargl[203] nm libm.so.2 | grep sF nm: libm.so.2: no symbols troutmask:kargl[204] strings libm.so.2 | grep sF __sF -- Steve To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
At 6:13 PM -0500 11/8/02, Daniel Eischen wrote: On Fri, 8 Nov 2002, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Daniel Eischen [EMAIL PROTECTED] writes: : All the ports are going to be rebuilt for the release anyways, : so this doesn't affect fresh installs, correct? It is only a : problem when mixing older 4.x and 5.0 libraries/binaries with : __sF-free libc (if I understand things correctly). The problem is that you cannot have 4.x packages and 5.x packages co-mingled on the same system. that's what I'm trying to fix. You'd have to rebuild the 4.x packages before they are fixed. I don't think this is a show-stopper. Just recompile all your ports or use the pre-built 5.0 packages. This assumes that you have the source code to everything you are running, which may not be true. This also assumes that all you need to do is type another 'make' in some directory, and your source code will pleasantly compile without any new problems from any of the long list of changes to include-files, ports, or the new version of gcc. Yes, every user can eventually plow their way through all that work, and by the time they do we'll probably already have 5.1-release out. Philosophically I would prefer that __sF was completely gone for 5.0-release, but at the moment I think Warner is right in saying that we need to keep it in libc until at least 5.1. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Brooks Davis [EMAIL PROTECTED] writes: : On Fri, Nov 08, 2002 at 04:16:06PM -0700, M. Warner Losh wrote: : I'd love for there to be a way to know which binaries use __sF. : : The following script run on your bin, sbin, lib, and libexec directories : does a pretty decent job of finding files that contain refrences to __sF : and listing the ports that use them (depend on portupgrade). I'm talking a warning at runtime for libraries that have the timebomb in them. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Steven G. Kargl [EMAIL PROTECTED] writes: : M. Warner Losh said: : In message: [EMAIL PROTECTED] : Steven G. Kargl [EMAIL PROTECTED] writes: : : M. Warner Losh said: : : In message: [EMAIL PROTECTED] : : Steven G. Kargl [EMAIL PROTECTED] writes: : : : Could someone add the following patch to UPDATING? : : : Change the words to whatever suits your fancy. : : : : I'm trying to devise a good way to deal with this breakage and hope it : : is transient. I'm not hopeful :-( I'll consider adding this to UPDATING. : : : : : : Any user who installed DP1 and a bunch of ports will : : have a unpleasant experience when they update to : : post 20021031 sources. All their ports will stop : : working and it isn't a simple matter to update : : those ports. : : I thought DP1 didn't generate binaries or libraries that contained : __sF. : : : http://www.freebsd.org/releases/5.0R/schedule.html : : DP1 was released on 5 Apr 2002. This may have occurred during : Peter's last attempt at killing __sF. DP2 is in progress : according to the web page and __sF should not be present. : There will also be that group of -current users who lag : a few weeks behind in their buildworlds. But std{in,out,err} #defines were changes from __sF to __std{in,out,err}p almost a year ago. That's what causes __sF to appear... Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In article [EMAIL PROTECTED], M. Warner Losh [EMAIL PROTECTED] wrote: In message: [EMAIL PROTECTED] Steven G. Kargl [EMAIL PROTECTED] writes: : Could someone add the following patch to UPDATING? : Change the words to whatever suits your fancy. I'm trying to devise a good way to deal with this breakage and hope it is transient. I'm not hopeful :-( I'll consider adding this to UPDATING. FWIW, the only OS fix that will make stock ezm3/pm3/CVSup buildable on -current is to make __sF global again and arrange for: stdin == __sF[0] stdout == __sF[1] stderr == __sF[2] John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Thu, Nov 07, 2002 at 08:40:32AM -0800, John Polstra wrote: In article [EMAIL PROTECTED], M. Warner Losh [EMAIL PROTECTED] wrote: In message: [EMAIL PROTECTED] Steven G. Kargl [EMAIL PROTECTED] writes: : Could someone add the following patch to UPDATING? : Change the words to whatever suits your fancy. I'm trying to devise a good way to deal with this breakage and hope it is transient. I'm not hopeful :-( I'll consider adding this to UPDATING. FWIW, the only OS fix that will make stock ezm3/pm3/CVSup buildable on -current is to make __sF global again and arrange for: stdin == __sF[0] stdout == __sF[1] stderr == __sF[2] John, As the author of cvsup, I'm sure you know what is required. But, I rebuilt cvsup from net/cvsup yesterday on a new world, and it appears to be working fine. What problems should I expect? -- Steve To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In article [EMAIL PROTECTED], Steve Kargl [EMAIL PROTECTED] wrote: I rebuilt cvsup from net/cvsup yesterday on a new world, and it appears to be working fine. What problems should I expect? It's possible that if you already have a working installation of pm3 or ezm3, you'll be able to build CVSup on -current. But if you try to build pm3 or ezm3 from scratch, you'll find that the build fails. John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Thu, Nov 07, 2002 at 09:21:53AM -0800, John Polstra wrote: In article [EMAIL PROTECTED], Steve Kargl [EMAIL PROTECTED] wrote: I rebuilt cvsup from net/cvsup yesterday on a new world, and it appears to be working fine. What problems should I expect? It's possible that if you already have a working installation of pm3 or ezm3, you'll be able to build CVSup on -current. But if you try to build pm3 or ezm3 from scratch, you'll find that the build fails. I removed all ports because of the __sF symbol problem. I simply did cd /usr/ports/net/cvsup ; make install and this automatically installed ezm3. pkg_info doesn't show pm3 installed on system. Perhaps, only pm3 is the port that will have the problem. -- Steve To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In article [EMAIL PROTECTED], Steve Kargl [EMAIL PROTECTED] wrote: On Thu, Nov 07, 2002 at 09:21:53AM -0800, John Polstra wrote: It's possible that if you already have a working installation of pm3 or ezm3, you'll be able to build CVSup on -current. But if you try to build pm3 or ezm3 from scratch, you'll find that the build fails. I removed all ports because of the __sF symbol problem. I simply did cd /usr/ports/net/cvsup ; make install and this automatically installed ezm3. pkg_info doesn't show pm3 installed on system. Perhaps, only pm3 is the port that will have the problem. That would surprise me, but I haven't tried it myself. Inspection of the ezm3 bootstrap shows that it has references to __sF. John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] John Polstra [EMAIL PROTECTED] writes: : In article [EMAIL PROTECTED], : M. Warner Losh [EMAIL PROTECTED] wrote: : In message: [EMAIL PROTECTED] : Steven G. Kargl [EMAIL PROTECTED] writes: : : Could someone add the following patch to UPDATING? : : Change the words to whatever suits your fancy. : : I'm trying to devise a good way to deal with this breakage and hope it : is transient. I'm not hopeful :-( I'll consider adding this to UPDATING. : : FWIW, the only OS fix that will make stock ezm3/pm3/CVSup buildable on : -current is to make __sF global again and arrange for: : : stdin == __sF[0] : stdout == __sF[1] : stderr == __sF[2] Why does cvsup need this to be the case? Now you have me curious. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Thu, Nov 07, 2002 at 09:35:19AM -0800, John Polstra wrote: In article [EMAIL PROTECTED], Steve Kargl [EMAIL PROTECTED] wrote: On Thu, Nov 07, 2002 at 09:21:53AM -0800, John Polstra wrote: It's possible that if you already have a working installation of pm3 or ezm3, you'll be able to build CVSup on -current. But if you try to build pm3 or ezm3 from scratch, you'll find that the build fails. I removed all ports because of the __sF symbol problem. I simply did cd /usr/ports/net/cvsup ; make install and this automatically installed ezm3. pkg_info doesn't show pm3 installed on system. Perhaps, only pm3 is the port that will have the problem. That would surprise me, but I haven't tried it myself. Inspection of the ezm3 bootstrap shows that it has references to __sF. Well, I just pkg_deinstall's both ezm3 and cvsup. I re-installed both without problems. I then used cvsup to pull down some source updates. However, here's the strange or maybe fortunate part troutmask:kargl[246] cd /usr/local/lib/m3 troutmask:kargl[247] find . -name \*.a | xargs nm -A | grep __sF ./pkg/m3core/FreeBSD4/libm3core.a:Cstdio.mo: U __sF troutmask:kargl[248] strings /usr/local/sbin/cvsupd | grep __sF troutmask:kargl[249] strings /usr/local/bin/cvsup | grep __sF -- Steve To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In article [EMAIL PROTECTED], Steve Kargl [EMAIL PROTECTED] wrote: On Thu, Nov 07, 2002 at 09:35:19AM -0800, John Polstra wrote: That would surprise me, but I haven't tried it myself. Inspection of the ezm3 bootstrap shows that it has references to __sF. Well, I just pkg_deinstall's both ezm3 and cvsup. I re-installed both without problems. I then used cvsup to pull down some source updates. However, here's the strange or maybe fortunate part troutmask:kargl[246] cd /usr/local/lib/m3 troutmask:kargl[247] find . -name \*.a | xargs nm -A | grep __sF ./pkg/m3core/FreeBSD4/libm3core.a:Cstdio.mo: U __sF troutmask:kargl[248] strings /usr/local/sbin/cvsupd | grep __sF troutmask:kargl[249] strings /usr/local/bin/cvsup | grep __sF Oh, I think I understand it now. PM3 uses shared libraries, so the undefined reference from libm3core.so matters. But ezm3 uses only static libraries, and Cstdio.mo probably isn't even included in the link (because nothing actually uses it). That explains why ezm3 works in spite of the fact that part of it references __sF. John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In article [EMAIL PROTECTED], M. Warner Losh [EMAIL PROTECTED] wrote: In message: [EMAIL PROTECTED] John Polstra [EMAIL PROTECTED] writes: : FWIW, the only OS fix that will make stock ezm3/pm3/CVSup buildable on : -current is to make __sF global again and arrange for: : : stdin == __sF[0] : stdout == __sF[1] : stderr == __sF[2] Why does cvsup need this to be the case? Now you have me curious. It's not CVSup, it's Modula-3. It thinks it knows that stdin, stdout, and stderr are defined as above, but they're not any more. Because Modula-3 isn't C and doesn't use C header files, it cannot automatically track such changes like C programs do. John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
Terry Lambert asked: Any chance we could get rid of all externally visable symbols that are not defined as being there by some standard, and not just __sF, since we are breaking the FORTRAN compiler and other third party code already? This cannot be entirely done if you still want to manage library bloat. In short, library routines have shared interfaces between them---common variables or common functions---that are internal to the library and should not be used by applications. To avoid this, you would have to bundle library functions together, which causes bloat. Worse, you would have to avoid or drastically limit your use of macros. (Any macro that uses one of these internal symbols generates a dependency in the compiled application.) It _would_ be a good idea to document any internal library symbols used by macros. Removing such symbols is a good way to break existing compiled applications. Library design involves a lot of tradeoffs. Tim Kientzle To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] John Polstra [EMAIL PROTECTED] writes: : In article [EMAIL PROTECTED], : M. Warner Losh [EMAIL PROTECTED] wrote: : In message: [EMAIL PROTECTED] : John Polstra [EMAIL PROTECTED] writes: : : : FWIW, the only OS fix that will make stock ezm3/pm3/CVSup buildable on : : -current is to make __sF global again and arrange for: : : : : stdin == __sF[0] : : stdout == __sF[1] : : stderr == __sF[2] : : Why does cvsup need this to be the case? Now you have me curious. : : It's not CVSup, it's Modula-3. It thinks it knows that stdin, : stdout, and stderr are defined as above, but they're not any more. : Because Modula-3 isn't C and doesn't use C header files, it cannot : automatically track such changes like C programs do. Gotcha. I'm thinking very seriously about keeping __sF support (but creating no new binaries with it in it) and the freeze on sizeof(FILE) through the 5.x series of releases because we botched the compatibility stuff so badly to give people a chance to catch their breaths before that reorg can happen. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In article [EMAIL PROTECTED], M. Warner Losh [EMAIL PROTECTED] wrote: In message: [EMAIL PROTECTED] John Polstra [EMAIL PROTECTED] writes: : : It's not CVSup, it's Modula-3. It thinks it knows that stdin, : stdout, and stderr are defined as above, but they're not any more. : Because Modula-3 isn't C and doesn't use C header files, it cannot : automatically track such changes like C programs do. Gotcha. I'm thinking very seriously about keeping __sF support (but creating no new binaries with it in it) and the freeze on sizeof(FILE) through the 5.x series of releases because we botched the compatibility stuff so badly to give people a chance to catch their breaths before that reorg can happen. I'm kind of on the fence about it. The point of hiding __sF is to remove all dependencies on the size of the FILE structure from applications, and that's a very worthwhile thing to do. Modula-3 is a special case (and a pathological one), and it shouldn't influence the decision too much. I don't think there's a way to fix it entirely in the OS without re-establishing the dependency on the size of FILE. We are lucky that ezm3 just happens to work. The PM3 port can be fixed with a 5.0-specific patch or two, but ... not today. John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
Tim Kientzle wrote: Terry Lambert asked: Any chance we could get rid of all externally visable symbols that are not defined as being there by some standard, and not just __sF, since we are breaking the FORTRAN compiler and other third party code already? This cannot be entirely done if you still want to manage library bloat. In short, library routines have shared interfaces between them---common variables or common functions---that are internal to the library and should not be used by applications. To avoid this, you would have to bundle library functions together, which causes bloat. Worse, you would have to avoid or drastically limit your use of macros. (Any macro that uses one of these internal symbols generates a dependency in the compiled application.) It _would_ be a good idea to document any internal library symbols used by macros. Removing such symbols is a good way to break existing compiled applications. Library design involves a lot of tradeoffs. I don't buy this. Specifically, I do not buy the idea that there is a necessity for the existance of data interfaces, as opposed to procedural interfaces. And procedural interfaces are fixable with weak symbols. The thing that's screwing us with __sF is that it is a data block that's directly referenced without the use of an accessor function, and thefore there are global data references, rather than accessor or mutator function references. As a result, when the underlying data changes, the code that has already been compiled to reference it, must also change. This problem has already been fixed with errno. It has already been fixed with strerror() (though unfortunately, the sys_errlist[] and sys_nerr references remain unwrapped). It has been fixed in many other places (e.g. the curses library used to export row and column count integers, and doesn't any more). It's very obvious to me that defining a *non-inline* function to return the correct __sF entry by entry ID will erase this problem for a future version of FreeBSD, while leaving the __sF invisible for a revision cycle, so that new applications will get the function reference, and old applications will get the __sF reference they expect, and continue to compile and link correctly. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
M. Warner Losh wrote: Gotcha. I'm thinking very seriously about keeping __sF support (but creating no new binaries with it in it) and the freeze on sizeof(FILE) through the 5.x series of releases because we botched the compatibility stuff so badly to give people a chance to catch their breaths before that reorg can happen. Redefining stdio as: #define stdin (__stdio(0)) #define stdout (__stdio(1)) #define stderr (__stdio(2)) And then defining an __stdio() function that returns a pointer to a struct FILE, would let you change the underlying implementation however you wanted, later, without disrupting newly compiled programs. This may not be entirely happy for static declarations: FILE *fp = stdin; /* default */ ...but that's a compiler problem (I think). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
From [EMAIL PROTECTED] Thu Nov 7 18:30:04 2002 Date: Thu, 07 Nov 2002 15:26:57 -0800 From: Terry Lambert [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [PATCH] note the __sF change in src/UPDATING Specifically, I do not buy the idea that there is a necessity for the existance of data interfaces, as opposed to procedural interfaces. And procedural interfaces are fixable with weak symbols. The thing that's screwing us with __sF is that it is a data block that's directly referenced without the use of an accessor function, and thefore there are global data references, rather than accessor or mutator function references. As a result, when the underlying data changes, the code that has already been compiled to reference it, must also change. This problem has already been fixed with errno. It has already been fixed with strerror() (though unfortunately, the sys_errlist[] and sys_nerr references remain unwrapped). It has been fixed in many other places (e.g. the curses library used to export row and column count integers, and doesn't any more). It's very obvious to me that defining a *non-inline* function to return the correct __sF entry by entry ID will erase this problem for a future version of FreeBSD, while leaving the __sF invisible for a revision cycle, so that new applications will get the function reference, and old applications will get the __sF reference they expect, and continue to compile and link correctly. Hear hear, I agree. There's no need to expose what ought to be private data to the world, especially when we can get the additional benefit here of letting us play with the implementation. - @ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Terry Lambert [EMAIL PROTECTED] writes: : M. Warner Losh wrote: : Gotcha. I'm thinking very seriously about keeping __sF support (but : creating no new binaries with it in it) and the freeze on sizeof(FILE) : through the 5.x series of releases because we botched the : compatibility stuff so badly to give people a chance to catch their : breaths before that reorg can happen. : : Redefining stdio as: : : #define stdin (__stdio(0)) : #define stdout(__stdio(1)) : #define stderr(__stdio(2)) : : And then defining an __stdio() function that returns a pointer : to a struct FILE, would let you change the underlying implementation : however you wanted, later, without disrupting newly compiled programs. : : This may not be entirely happy for static declarations: : : FILE *fp = stdin; /* default */ : : ...but that's a compiler problem (I think). That's what __std{in,out,err}p do, and it saves function calls all over the place. This patch just makes -stable more compatible with current. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Ray Kohler [EMAIL PROTECTED] writes: : Hear hear, I agree. There's no need to expose what ought to be : private data to the world, especially when we can get the additional : benefit here of letting us play with the implementation. -current already does this. The problem is that we're trying to shoot the bad access in the head, and that is what is screwing people. So the problem isn't that we're trying to export private data to the world. Quite the contrary, we're trying to eliminate it and having growing pains. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
M. Warner Losh wrote: -current already does this. The problem is that we're trying to shoot the bad access in the head, and that is what is screwing people. So the problem isn't that we're trying to export private data to the world. Quite the contrary, we're trying to eliminate it and having growing pains. No, it's actually an attempt to change which private data is exported; the new code exports three private values, instead of an array of private values. Data interfaces are evil (proc size mismatch). Probably FILE * should be void *. 8-(. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
Steven G. Kargl wrote: Could someone add the following patch to UPDATING? Change the words to whatever suits your fancy. +20021031 + Revision 1.24 of lib/libc/stdio/findfp.c has made __sF static. + This changes the visibility of __sF to a symbol internal to + libc. All applications linked against libc or a library that + depends on libc that were compiled prior to 31 Oct 2002 will + need to be updated after a make world. An error message that + includes undefined reference to `__sF' is an indication that + that application needs to be recompiled. + Any chance we could get rid of all externally visable symbols that are not defined as being there by some standard, and not just __sF, since we are breaking the FORTRAN compiler and other third party code already? It would be nice if this only ever happened *once*. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
On Wed, Nov 06, 2002 at 04:38:55PM -0800, Terry Lambert wrote: Steven G. Kargl wrote: Could someone add the following patch to UPDATING? Change the words to whatever suits your fancy. +20021031 + Revision 1.24 of lib/libc/stdio/findfp.c has made __sF static. + This changes the visibility of __sF to a symbol internal to + libc. All applications linked against libc or a library that + depends on libc that were compiled prior to 31 Oct 2002 will + need to be updated after a make world. An error message that + includes undefined reference to `__sF' is an indication that + that application needs to be recompiled. + Any chance we could get rid of all externally visable symbols that are not defined as being there by some standard, and not just __sF, since we are breaking the FORTRAN compiler and other third party code already? This isn't restricted to my Fortran 95 problem, which I've found an acceptable work-around. I just spent the better part of a day re-installing 122 ports because these ports were linked against an earlier libc. All of this is 3rd party software. But, if you have a pre-20021031 world and you build only a post-20021021 libc, then a whole bunch of libraries in /usr/lib will becomed unusable. -- Steve To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
Steve Kargl wrote: Any chance we could get rid of all externally visable symbols that are not defined as being there by some standard, and not just __sF, since we are breaking the FORTRAN compiler and other third party code already? This isn't restricted to my Fortran 95 problem, which I've found an acceptable work-around. I just spent the better part of a day re-installing 122 ports because these ports were linked against an earlier libc. All of this is 3rd party software. But, if you have a pre-20021031 world and you build only a post-20021021 libc, then a whole bunch of libraries in /usr/lib will becomed unusable. I'm more concerned about this happening again, the next time someone decides another symbol shouldn't be exposed. If we could get the pain over with all at once, that'd be a good thing. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
In message: [EMAIL PROTECTED] Steven G. Kargl [EMAIL PROTECTED] writes: : Could someone add the following patch to UPDATING? : Change the words to whatever suits your fancy. I'm trying to devise a good way to deal with this breakage and hope it is transient. I'm not hopeful :-( I'll consider adding this to UPDATING. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
M. Warner Losh said: In message: [EMAIL PROTECTED] Steven G. Kargl [EMAIL PROTECTED] writes: : Could someone add the following patch to UPDATING? : Change the words to whatever suits your fancy. I'm trying to devise a good way to deal with this breakage and hope it is transient. I'm not hopeful :-( I'll consider adding this to UPDATING. Any user who installed DP1 and a bunch of ports will have a unpleasant experience when they update to post 20021031 sources. All their ports will stop working and it isn't a simple matter to update those ports. -- Steve http://troutmask.apl.washington.edu/~kargl/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: [PATCH] note the __sF change in src/UPDATING
M. Warner Losh said: In message: [EMAIL PROTECTED] Steven G. Kargl [EMAIL PROTECTED] writes: : M. Warner Losh said: : In message: [EMAIL PROTECTED] : Steven G. Kargl [EMAIL PROTECTED] writes: : : Could someone add the following patch to UPDATING? : : Change the words to whatever suits your fancy. : : I'm trying to devise a good way to deal with this breakage and hope it : is transient. I'm not hopeful :-( I'll consider adding this to UPDATING. : : : Any user who installed DP1 and a bunch of ports will : have a unpleasant experience when they update to : post 20021031 sources. All their ports will stop : working and it isn't a simple matter to update : those ports. I thought DP1 didn't generate binaries or libraries that contained __sF. http://www.freebsd.org/releases/5.0R/schedule.html DP1 was released on 5 Apr 2002. This may have occurred during Peter's last attempt at killing __sF. DP2 is in progress according to the web page and __sF should not be present. There will also be that group of -current users who lag a few weeks behind in their buildworlds. -- Steve http://troutmask.apl.washington.edu/~kargl/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message