Re: [dev] Replace ranlib(1) calls?
On Wed, Aug 03, 2022 at 05:14:20PM +0200, Laslo Hunhold wrote: > I have one question here: Doesn't ranlib(1) need to understand the > ar-format to be able to add the symbol-table-file? > Not if it uses "ar r" to add the symbol table to the archive. Although that is contingent on the symbol table having a valid file name, which I don't know. Ciao, Markus
Re: [dev] Replace ranlib(1) calls?
On Wed, 3 Aug 2022 16:38:14 +0200 Markus Wichmann wrote: Dear Markus, thanks for sharing your thoughts! > That design would afford some flexibility to the whole business: ar > doesn't need to know the object file format and ranlib doesn't need to > know the ar file format. I have one question here: Doesn't ranlib(1) need to understand the ar-format to be able to add the symbol-table-file? With best regards Laslo
Re: [dev] Replace ranlib(1) calls?
On Wed, Aug 03, 2022 at 03:21:30PM +0200, Laslo Hunhold wrote: > In a way, ar(1) is merely an archiver, but POSIX added a function that > only applies to exclusively object-file-containing-archives, but stays > away from specifics[0]: > [...] Yeah, POSIX be like that sometimes. You can have a perfectly simple model of how a program is supposed to work, then POSIX comes along and specifies one thing that fits in about as elegantly as a blue whale into a stationary cupboard. Another example would be the interactive mode for cp/mv/rm, applications which have absolutely no business reading from terminal being forced to do it. Or alternatively, you just sod this one aspect of it. In this case, a simple archiver is supposed to do special things based on the content of the files it is archiving. Or, alternatively, you say no to that requirement, implement ar as a simple archiver, and ranlib as a program that reads the archive, makes a symbol table, and adds it to the archive. Possibly even by calling the ar program. That design would afford some flexibility to the whole business: ar doesn't need to know the object file format and ranlib doesn't need to know the ar file format. Indeed ld could read the symbol table and object files from the archive by calling ar, then nothing in the system would depend on the precise format of the ar files. That would be modular. I daresay, it would suck less than what POSIX wants you to do. But those are just the 2 cents of a grumpy old millenial. Ciao, Markus
Re: [dev] Replace ranlib(1) calls?
On Mon, 1 Aug 2022 11:32:20 +0200 "Roberto E. Vargas Caballero" wrote: Dear Roberto, > There is a big important reason why scc ar does not generates a link > table, because then ar can handle any type of files, because ar is > just an archiver. Making ar(1) to generate symbol tables means that > ar has to do detection of type files, meaning that it has to support > all the possible binary formats. > > At this moment scc ar is able to generate libraries for any system, > without a symbol table yes, but thay are going to work. In this point > we have several standard to consider. POSIX does not enter in binary > formats, it is out of the scope of the standard, and htis is why > as(1) and ld(1) are out of the specification, and it does not mean > that you should not use them, it is just that they cannot be > standarized. ELF specification implies that ar should generate a > symbol table, but not a.out ar. If you want to be sure that you have > a symbl table then you *need* ranlib, otherwise it depends of the > system where you execute. > > Again, ranlib(1) is out of POSIX because it cannot standarize this > kind of topics, not because it is legacy. In fact, MacOS requires use > of ranlib (or at least it needed the last time that I tested scc in > MacOS). You cannot know what system require it or not, so the only > safe option is to use ranlib. now I understand, thanks for shedding light on this. This is a strong point and I understand the notion of the standards consortium. In a way, ar(1) is merely an archiver, but POSIX added a function that only applies to exclusively object-file-containing-archives, but stays away from specifics[0]: "When an archive consists entirely of valid object files, the implementation shall format the archive so that it is usable as a library for link editing (see c99 and fort77). When some of the archived files are not valid object files, the suitability of the archive for library use is undefined." And explicitly as an XSI-extension "When ar creates an archive, it creates administrative information indicating whether a symbol table is present in the archive. When there is at least one object file that ar recognizes as such in the archive, an archive symbol table shall be created in the archive and maintained by ar; it is used by the link editor to search the archive. Whenever the ar utility is used to create or update the contents of such an archive, the symbol table shall be rebuilt. The -s option shall force the symbol table to be rebuilt." What makes me wonder here is that you do in fact implement ranlib(1)[1] in scc. Could you please elaborate what difference it makes here and why you don't simply add the s-flag to ar(1)? Let me know if I'm wrong, but I don't see the difference between running ar(1) and then ranlib(1) to create a _foreign_ object-file-archive and doing so with the equivalent functionality implemented in ar(1). In both cases, you have to do file-detection, right? POSIX also gives you a lot of leeway, I think, with "When some of the archived files are not valid object files, the suitability of the archive for library use is undefined." you can pretty much skip creating the symbol table for foreign binaries. With best regards Laslo [0]:https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ar.html [1]:http://git.simple-cc.org/scc/file/src/cmd/ranlib%2Ec%2Ehtml
Re: [dev] Replace ranlib(1) calls?
On Mon, 1 Aug 2022 11:21:19 +0200 "Roberto E. Vargas Caballero" wrote: Dear Roberto, > -std=c99 is not part of any standard and our Makefiles are full of > them. as(1), ld(1), and cc(1) are not part of POSIX and we keep using > them. Again, Please stop doing this kind of patches and center in > changes that actually improve code. If we find a system where using > ranlib is a problem then we can revisit this. you bring up good points, however, the difference I see here is that POSIX explicitly specifies an alternative. If POSIX specified a standard calling-convention for the C-compiler or linker, I'd use it over the non-standardized forms. And I even agree with you: Consider pax(1) for instance, and the fact that POSIX doesn't even specify tar(1) anymore and pax(1) is rarely found on systems. I'll think about this more and make a decision before the next release. With best regards Laslo
Re: [dev] Replace ranlib(1) calls?
Hi, On Sun, Jul 31, 2022 at 01:00:23AM +0200, Laslo Hunhold wrote: > I must admit that I only copied the ar-ranlib-invocation likewise from > what I found on the internet for my libraries, so even though I don't > think it makes any difference (i.e. increases portability apart from > standards-legalese), it helps to make ranlib more and more superfluous > and promote the best practice, which is why I pushed your described > change for libgrapheme[0]. There is a big important reason why scc ar does not generates a link table, because then ar can handle any type of files, because ar is just an archiver. Making ar(1) to generate symbol tables means that ar has to do detection of type files, meaning that it has to support all the possible binary formats. At this moment scc ar is able to generate libraries for any system, without a symbol table yes, but thay are going to work. In this point we have several standard to consider. POSIX does not enter in binary formats, it is out of the scope of the standard, and htis is why as(1) and ld(1) are out of the specification, and it does not mean that you should not use them, it is just that they cannot be standarized. ELF specification implies that ar should generate a symbol table, but not a.out ar. If you want to be sure that you have a symbl table then you *need* ranlib, otherwise it depends of the system where you execute. Again, ranlib(1) is out of POSIX because it cannot standarize this kind of topics, not because it is legacy. In fact, MacOS requires use of ranlib (or at least it needed the last time that I tested scc in MacOS). You cannot know what system require it or not, so the only safe option is to use ranlib. Regards,
Re: [dev] Replace ranlib(1) calls?
Hi, On Sun, Jul 31, 2022 at 12:59:23AM +0200, Laslo Hunhold wrote: > Granted, it's one thing what a standard defines and another what is > actually used in everyday life, but calling a standard-defined option > as _less_ portable than an undefined historical artifact is a stretch. -std=c99 is not part of any standard and our Makefiles are full of them. as(1), ld(1), and cc(1) are not part of POSIX and we keep using them. Again, Please stop doing this kind of patches and center in changes that actually improve code. If we find a system where using ranli is a problem then we can revisit this. Roberto,
Re: [dev] Replace ranlib(1) calls?
On Wed, 20 Jul 2022 12:23:01 + Tom Schwindl wrote: Dear Tom, > I've noticed that we use the non-standard ranlib(1) program to create > symbol tables for archives created by ar(1). This affects, as far as > I can tell, every creation of static libraries we have. > > ranlib(1) is, in fact, legacy. It was used to create symbol tables > for archives before ar(1) was able to do that by itself. > POSIX now specifies that ar(1) should create a symbol table by > default, when there is at least one recognized object file[1]. > Additionally, the `s` option exists which does exactly the same as > ranlib(1)[2]. > > We could replace the ranlib(1) calls with: > > ar rcs files... > > which not only makes the creation of static libraries more portable, > but also simpler. this is a very good observation, thank you! I must admit that I only copied the ar-ranlib-invocation likewise from what I found on the internet for my libraries, so even though I don't think it makes any difference (i.e. increases portability apart from standards-legalese), it helps to make ranlib more and more superfluous and promote the best practice, which is why I pushed your described change for libgrapheme[0]. Thanks for making suckless software suck less. :) With best regards Laslo [0]:https://git.suckless.org/libgrapheme/commit/5fc87aedad86a8410a360ba718096e613ddf11f6.html
Re: [dev] Replace ranlib(1) calls?
On Sat, 23 Jul 2022 23:06:49 + Tom Schwindl wrote: Dear Tom, > If a system says it's POSIX compliant, we can assume that the `-s' > option exists, but there is no standard which tells us whether > ranlib(1) is available or not. while I agree with your point in general, keep in mind that the s-flag is, after all, a POSIX XSI-extension (X/Open System Interfaces). This is still better than ranlib(1), though, which has a very murky "foothold" in regard to standardization. Granted, it's one thing what a standard defines and another what is actually used in everyday life, but calling a standard-defined option as _less_ portable than an undefined historical artifact is a stretch. With best regards Laslo
Re: [dev] Replace ranlib(1) calls?
On Wed, 20 Jul 2022 12:23:01 + Tom Schwindl wrote: Dear Tom, > I've noticed that we use the non-standard ranlib(1) program to create > symbol tables for archives created by ar(1). This affects, as far as > I can tell, every creation of static libraries we have. > > ranlib(1) is, in fact, legacy. It was used to create symbol tables > for archives before ar(1) was able to do that by itself. > POSIX now specifies that ar(1) should create a symbol table by > default, when there is at least one recognized object file[1]. > Additionally, the `s` option exists which does exactly the same as > ranlib(1)[2]. > > We could replace the ranlib(1) calls with: > > ar rcs files... > > which not only makes the creation of static libraries more portable, > but also simpler. this is a very good observation, thank you! I must admit that I only copied the ar-ranlib-invocation likewise from what I found on the internet for my libraries, so even though I don't think it makes any difference (i.e. increases portability apart from standards-legalese), it helps to make ranlib more and more superfluous and promote the best practice, which is why I pushed your described change for libgrapheme[0]. Thanks for making suckless software suck less. :) With best regards Laslo [0]:https://git.suckless.org/libgrapheme/commit/5fc87aedad86a8410a360ba718096e613ddf11f6.html
Re: [dev] Replace ranlib(1) calls?
On Sat, 23 Jul 2022 23:06:49 + Tom Schwindl wrote: Dear Tom, > If a system says it's POSIX compliant, we can assume that the `-s' > option exists, but there is no standard which tells us whether > ranlib(1) is available or not. while I agree with your point in general, keep in mind that the s-flag is, after all, a POSIX XSI-extension (X/Open System Interfaces). This is still better than ranlib(1), though, which has a very murky "foothold" in regard to standardization. Granted, it's one thing what a standard defines and another what is actually used in everyday life, but calling a standard-defined option as _less_ portable than an undefined historical artifact is a stretch. With best regards Laslo
Re: [dev] Replace ranlib(1) calls?
On Sun, Jul 24, 2022 at 08:04:05PM +0200, Страхиња Радић wrote: > On 22/07/23 11:06, Tom Schwindl wrote: > > Again, you'd have to rely on a tool which isn't defined. > > If a system says it's POSIX compliant, we can assume that the `-s' option > > exists, but there is no standard which tells us whether ranlib(1) is > > available > > or not. > > Standards are not the Holy Grail. > > https://harmful.cat-v.org/standards/ Plus there is what Georg Jellinek called the "normative force of the factual": ranlib is de-facto standard on all UNIX variants I know of (and I know weird ones line Cygwin and Solaris). It's a bit like vfork(), gone but not forgotten, and every once in a blue moon something comes along that still uses it. On many systems, ranlib is even just implemented as a call to ar. But we don't need to care about that. The important fact is that in practice, ranlib is on all systems anyone has cared to use our software on so far, and future systems are more likely to hack in a version of ranlib that to send patches to get rid of it. And if they decide to have ar generate a symbol table irrespective of the "-s" option, and link ranlib to true, that is also fine. Ciao, Markus
Re: [dev] Replace ranlib(1) calls?
On 22/07/23 11:06, Tom Schwindl wrote: > Again, you'd have to rely on a tool which isn't defined. > If a system says it's POSIX compliant, we can assume that the `-s' option > exists, but there is no standard which tells us whether ranlib(1) is available > or not. Standards are not the Holy Grail. https://harmful.cat-v.org/standards/ signature.asc Description: PGP signature
Re: [dev] Replace ranlib(1) calls?
Tom, > The main "problem" here is that ranlib(1) isn't a standardized tool, so there > is no guarantee on what its behaviour will be or whether it exists at all. > We assume it's there because it always has been or GNU and LLVM include it. an option might be to wait until the bug report comes in "Hey, ranlib isn't available on my system!!! How do i build ...?". cheers, Greg
Re: [dev] Replace ranlib(1) calls?
Hi, On Fri, Jul 22, 2022 at 05:26:52PM +0200, Roberto E. Vargas Caballero wrote: > Hi, > > On Wed, Jul 20, 2022 at 12:23:01PM +, Tom Schwindl wrote: > > ranlib(1) is, in fact, legacy. It was used to create symbol tables for > > archives > > before ar(1) was able to do that by itself. > > POSIX now specifies that ar(1) should create a symbol table by default, when > > there is at least one recognized object file[1]. Additionally, the `s` > > option exists > > which does exactly the same as ranlib(1)[2]. > > Well, I disagree about ranlib to be obsolete. Ar is guaranteed to add an > index in COFF and ELF systems, but not in a.out. I also know that 99.99% > of the people will ignore such systems, but I personally care about them. > Also, using ranlib doesn't harm so I personally don't see the advantage > of removing it. The main "problem" here is that ranlib(1) isn't a standardized tool, so there is no guarantee on what its behaviour will be or whether it exists at all. We assume it's there because it always has been or GNU and LLVM include it. Relying on a utility which isn't defined by the specification we're using doesn't make much sense to me. Especially if there *is* a *defined* flag which does exactly the same thing as ranlib(1). Bothering with systems which do not comply with "our" standards (by which I mean POSIX) isn't worth it, unless there are concrete plans and good reasons to support them (at least in my opinion). > > > ar rcs files... > > > > which not only makes the creation of static libraries more portable, > > but also simpler. > > More portable? It is just the opposite, if you use ranlib you can support > systems where ar doesn't create an index in the archive (for example if > you use scc ar :P). > Again, you'd have to rely on a tool which isn't defined. If a system says it's POSIX compliant, we can assume that the `-s' option exists, but there is no standard which tells us whether ranlib(1) is available or not. We'd have to introduce exceptions and thus increase the complexity to support edge cases because *they* do not comply to standards. -- Best Regards, Tom Schwindl
Re: [dev] Replace ranlib(1) calls?
Hi, On Wed, Jul 20, 2022 at 12:23:01PM +, Tom Schwindl wrote: > ranlib(1) is, in fact, legacy. It was used to create symbol tables for > archives > before ar(1) was able to do that by itself. > POSIX now specifies that ar(1) should create a symbol table by default, when > there is at least one recognized object file[1]. Additionally, the `s` option > exists > which does exactly the same as ranlib(1)[2]. Well, I disagree about ranlib to be obsolete. Ar is guaranteed to add an index in COFF and ELF systems, but not in a.out. I also know that 99.99% of the people will ignore such systems, but I personally care about them. Also, using ranlib doesn't harm so I personally don't see the advantage of removing it. > ar rcs files... > > which not only makes the creation of static libraries more portable, > but also simpler. More portable? It is just the opposite, if you use ranlib you can support systems where ar doesn't create an index in the archive (for example if you use scc ar :P). Regards,
[dev] Replace ranlib(1) calls?
Hi all, I've noticed that we use the non-standard ranlib(1) program to create symbol tables for archives created by ar(1). This affects, as far as I can tell, every creation of static libraries we have. ranlib(1) is, in fact, legacy. It was used to create symbol tables for archives before ar(1) was able to do that by itself. POSIX now specifies that ar(1) should create a symbol table by default, when there is at least one recognized object file[1]. Additionally, the `s` option exists which does exactly the same as ranlib(1)[2]. We could replace the ranlib(1) calls with: ar rcs files... which not only makes the creation of static libraries more portable, but also simpler. [1] https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/utilities/ar.html [2] https://sourceware.org/binutils/docs/binutils/ranlib.html -- Best Regards, Tom Schwindl
