Re: Trivial program size inflation

2023-07-05 Thread RVP
On Wed, 5 Jul 2023, Emmanuel Dreyfus wrote: It is documented somewhere? Last time I tried to use it, I had to make a static build. If you mean the .spec file syntax, then yes: https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html As for the patch itself, I'm travelling at present, so I don't

Re: Trivial program size inflation

2023-07-05 Thread Emmanuel Dreyfus
On Wed, Jul 05, 2023 at 06:35:45AM +, RVP wrote: > (I had to fiddle with them to get `-pg' working with shared > libs...) It is documented somewhere? Last time I tried to use it, I had to make a static build. -- Emmanuel Dreyfus m...@netbsd.org

Re: Trivial program size inflation

2023-07-05 Thread RVP
On Tue, 4 Jul 2023, Mouse wrote: I've failed to find where the second -lgcc -lgcc_eh is specified, though (I thought there may be explanatory comments); I searched /usr/src for -lgcc_eh and found only three hits, none of which appear to specify the duplication. I may take a closer look

Re: Trivial program size inflation

2023-07-04 Thread Taylor R Campbell
> Date: Tue, 4 Jul 2023 18:30:58 + > From: Taylor R Campbell > > Other low-hanging fruit I see would be to split getenv out of the > other environment variable operations, to get rid of locking and > rbtree stuff. But it's diminishing returns at this point -- not clear > the attached

Re: Trivial program size inflation

2023-07-04 Thread Taylor R Campbell
> Date: Fri, 30 Jun 2023 13:37:10 -0400 (EDT) > From: Mouse > > sparc, my mutant 1.4T: > > textdatabss dec hex filename > 12616 124 288 13028 32e4main > > amd64, my mutant 5.2: > >text data bss dec hex filename > 152613

Re: Trivial program size inflation

2023-07-04 Thread Mouse
> No, ld reads the archive in its sequence in the arg list [...] then > when it reaches the end of the .a file, it is done, and nothing will > ever return to it again (the .a can be included on the link command > line more than once - about 1 in 10,000,000 cases [statistic with 0 > supporting

Re: Trivial program size inflation

2023-07-04 Thread Joerg Sonnenberger
On Tue, Jul 04, 2023 at 12:31:33PM +, RVP wrote: > On Tue, 4 Jul 2023, RVP wrote: > > > I think lld(1) does things differently. I'll check this later... > > > > And I was right! On Linux (what I have at hand now): > > ``` > $ clang -fuse-ld=lld -Wl,-Map=map.txt -static -s -o foo foo.c -L.

Re: Trivial program size inflation

2023-07-04 Thread Robert Elz
Date:Tue, 4 Jul 2023 12:03:20 + (UTC) From:RVP Message-ID: | It should've noticed _all_ the functions in | the supplied archive, right? No, ld reads the archive in its sequence in the arg list, looks at each included .o file in the order they appear in the

Re: Trivial program size inflation

2023-07-04 Thread RVP
On Tue, 4 Jul 2023, RVP wrote: I think lld(1) does things differently. I'll check this later... And I was right! On Linux (what I have at hand now): ``` $ clang -fuse-ld=lld -Wl,-Map=map.txt -static -s -o foo foo.c -L. -lmm $ fgrep -A3 ./libmm.a map.txt [...] -- 224c30

Re: Trivial program size inflation

2023-07-04 Thread Mouse
> Can't say I understand why ld(1) behaves this way, though. (I'm > pretty sure I ran ranlib too.) It should've noticed _all_ the > functions in the supplied archive, right? Not if there were no unsatisfied references to them at the time it gets to that .a in the list. With .a archives, the

Re: Trivial program size inflation

2023-07-04 Thread RVP
On Tue, 4 Jul 2023, Joerg Sonnenberger wrote: On Tue, Jul 04, 2023 at 08:38:22AM +, RVP wrote: b) the alloc functions in libc.a not being marked as being "weak". This doesn't really matter at all, in fact, it is likely going to make the situation *worse*. You're right. And, kre@

Re: Trivial program size inflation

2023-07-04 Thread Joerg Sonnenberger
On Tue, Jul 04, 2023 at 08:38:22AM +, RVP wrote: > b) the alloc functions in libc.a not being marked as being "weak". This doesn't really matter at all, in fact, it is likely going to make the situation *worse*. The linker errors happen because you are trying to mix and match malloc

Re: Trivial program size inflation

2023-07-04 Thread RVP
On Tue, 4 Jul 2023, RVP wrote: So, we should: a) collapse the newly added functions back into a single file. Ah, looks like this patch will need a free_aligned_sized() to free the aligned memory. Regular free(), realloc() won't work here (as-is). -RVP

Re: Trivial program size inflation

2023-07-04 Thread RVP
On Mon, 3 Jul 2023, Taylor R Campbell wrote: However, if I try to link something that calls malloc or realloc with -static -lbsdmalloc, ld objects: $ cat null.c #include int main(void) { return malloc(1) != NULL; } $ make null LDLIBS=-static\ -lbsdmalloc\ -Wl,-Map=null.map cc -O2-o null

Re: Trivial program size inflation

2023-07-03 Thread Steffen Nurpmeso
Robert Elz wrote in <2939.1688393...@jacaranda.noi.kre.to>: |Date:Sun, 2 Jul 2023 15:51:06 -0400 (EDT) |From:Mouse |Message-ID: <202307021951.paa07...@stone.rodents-montreal.org> | || For example, a program that calls printf but never uses any || floating-point

Re: Trivial program size inflation

2023-07-03 Thread Robert Elz
Date:Mon, 3 Jul 2023 15:21:24 + From:Taylor R Campbell Message-ID: <20230703152125.8798060...@jupiter.mumble.net> | Not sure why calling malloc or realloc still causes jemalloc.o to be | pulled in, in the presence of -lbsdmalloc. libbsdmalloc doesn't appear

Re: Trivial program size inflation

2023-07-03 Thread Taylor R Campbell
> Date: Mon, 3 Jul 2023 07:45:27 + (UTC) > From: RVP > > On Mon, 3 Jul 2023, RVP wrote: > > > Somebody should maybe add calloc() to bsdmalloc. > > And posix_memalign() (any others?) too, else you end up with 2 > different arenas and free()/realloc() operating on the wrong one. The

Re: Trivial program size inflation

2023-07-03 Thread Robert Elz
Date:Sun, 2 Jul 2023 15:51:06 -0400 (EDT) From:Mouse Message-ID: <202307021951.paa07...@stone.rodents-montreal.org> | For example, a program that calls printf but never uses any | floating-point values at all will not, in theory, need floating point | support.

Re: Trivial program size inflation

2023-07-03 Thread Martin Husemann
On Sun, Jul 02, 2023 at 10:53:31PM +, Emmanuel Dreyfus wrote: > On Sun, Jul 02, 2023 at 11:16:12PM +0200, Joerg Sonnenberger wrote: > > > At least in 9.3, dlopen() in a static binary does not work. Try using > > > a NSS module from a statically lunked binary to check that. > > > > It does

Re: Trivial program size inflation

2023-07-03 Thread RVP
On Mon, 3 Jul 2023, RVP wrote: Somebody should maybe add calloc() to bsdmalloc. And posix_memalign() (any others?) too, else you end up with 2 different arenas and free()/realloc() operating on the wrong one. -RVP

Re: Trivial program size inflation

2023-07-03 Thread RVP
On Mon, 3 Jul 2023, Mouse wrote: Looking at the resulting null.map, I see, among many other lines, /usr/lib/libc.a(jemalloc.o) /usr/lib/libc.a(tls.o) (calloc) which makes sense: if nothing in null.o, crt0.o, crti.o, or crtbeginT.o refers to anything in the libgnumalloc file containing

Re: Trivial program size inflation

2023-07-02 Thread Mouse
> I'm sorta curious why linking against one of our small malloc > implementations still pulls in jemalloc: Using my shell on ftp.n.o (9.0_STABLE): cc -v says the ld line for cc -o null null.c -lgnumalloc is ld -plugin /usr/libexec/liblto_plugin.so -plugin-opt=/usr/libexec/lto-wrapper

Re: Trivial program size inflation

2023-07-02 Thread Simon Burge
Taylor R Campbell wrote: > A quicker way to address most of it is to just define your own malloc: > > $ cat null.o > #include > void *malloc(size_t n) { return NULL; } > void *realloc(void *p, size_t n) { return NULL; } > void *calloc(size_t n, size_t sz) { return NULL; } > void free(void *p) {}

Re: Trivial program size inflation

2023-07-02 Thread Taylor R Campbell
> Date: Sat, 1 Jul 2023 15:11:56 - (UTC) > From: mlel...@serpens.de (Michael van Elst) > > crt0 pulls in > - atexit > - environment > - static TLS > - stack guard > > which all more or less pull in jemalloc, stdio and string functions. > > You need to replace these with dummies (compile

Re: Trivial program size inflation

2023-07-02 Thread Joerg Sonnenberger
On Sun, Jul 02, 2023 at 06:52:55PM -0400, Mouse wrote: > > The CSU code has pretty much no idea on what the rest of the world is > > going to do. It does: [...] > > > There is no way with ELF to decide at link time which of those > > features are used by the program and therefore no way to

Re: Trivial program size inflation

2023-07-02 Thread Emmanuel Dreyfus
On Sun, Jul 02, 2023 at 11:16:12PM +0200, Joerg Sonnenberger wrote: > > At least in 9.3, dlopen() in a static binary does not work. Try using > > a NSS module from a statically lunked binary to check that. > > It does work in the sense that it always fails. It fails silently, there not even an

Re: Trivial program size inflation

2023-07-02 Thread Mouse
> The CSU code has pretty much no idea on what the rest of the world is > going to do. It does: [...] > There is no way with ELF to decide at link time which of those > features are used by the program and therefore no way to remove any > of them. I don't think that's true. If the program

Re: Trivial program size inflation

2023-07-02 Thread Joerg Sonnenberger
On Sun, Jul 02, 2023 at 05:04:10PM +0200, Martin Husemann wrote: > The other things that we *might* look into (if someone volunteers) is to > better modularize the CSU code, but it is not immediately clear how > that could/should be done. The CSU code has pretty much no idea on what the rest of

Re: Trivial program size inflation

2023-07-02 Thread Joerg Sonnenberger
On Sun, Jul 02, 2023 at 01:20:51AM +, Emmanuel Dreyfus wrote: > On Sat, Jul 01, 2023 at 02:25:03PM +, RVP wrote: > > Not to forget the code for C++ support. And, of course even static > > binaries may call dlopen() and friends. So that dl*() and the ELF bits > > right there. > > At least

Re: Trivial program size inflation

2023-07-02 Thread Michael van Elst
mo...@rodents-montreal.org (Mouse) writes: >> Way more interesting than useless tech demo sizes would size >> inflation of a real world minimal program, when linked statically. >Why? If I'm looking at overhead size, I am most interested in just the >overhead size, which is exactly what a no-op

Re: Trivial program size inflation

2023-07-02 Thread Mouse
> With most real world programs (hopefully) nearly 100% of what you see > as overhead now is actually needed - and it still may be bigger than > what we hope for due to suboptimal modularization. True. But this is not always as fixable as that wording implies. For example, a program that calls

Re: Trivial program size inflation

2023-07-02 Thread Martin Husemann
On Sun, Jul 02, 2023 at 01:49:57PM -0400, Mouse wrote: > Why? If I'm looking at overhead size, I am most interested in just the > overhead size, which is exactly what a no-op program gives. If I want > to look at the overhead of printf, or malloc, or something, I'd use a > program that just

Re: Trivial program size inflation

2023-07-02 Thread Mouse
> Note that a "do nothing" binary is a useless tech demo I chose it because, to a good approximation, the resulting size is nothing but overhead. It is useless as far as its own functionality goes; it is not useless in that it very clearly measures pure overhead. I actually started out on

Re: Trivial program size inflation

2023-07-02 Thread Martin Husemann
On Sun, Jul 02, 2023 at 12:34:25PM +0200, tlaro...@polynum.com wrote: > It is curious that you react this way in a thread where, you as others, > have had your jaw drop seeing the size of a literally do_nothing > executable. This was unseen precisely because few use static linking. I'm not sure

Re: Trivial program size inflation

2023-07-02 Thread Jason Thorpe
> On Jul 1, 2023, at 6:20 PM, Emmanuel Dreyfus wrote: > > On Sat, Jul 01, 2023 at 02:25:03PM +, RVP wrote: >> Not to forget the code for C++ support. And, of course even static >> binaries may call dlopen() and friends. So that dl*() and the ELF bits >> right there. > > At least in 9.3,

Re: Trivial program size inflation

2023-07-02 Thread tlaronde
Le Sat, Jul 01, 2023 at 12:04:25PM -0700, Jason Thorpe a écrit : > > > On Jul 1, 2023, at 8:20 AM, tlaro...@polynum.com wrote: > > > > This is also what I meant by "static seems to be considered deprecated". > > Honestly, I find the obsession with static linking hilariously quaint. > NetBSD

Re: Trivial program size inflation

2023-07-02 Thread RVP
On Sun, 2 Jul 2023, Emmanuel Dreyfus wrote: On Sat, Jul 01, 2023 at 02:25:03PM +, RVP wrote: Not to forget the code for C++ support. And, of course even static binaries may call dlopen() and friends. So that dl*() and the ELF bits right there. At least in 9.3, dlopen() in a static binary

Re: Trivial program size inflation

2023-07-01 Thread Emmanuel Dreyfus
On Sat, Jul 01, 2023 at 02:25:03PM +, RVP wrote: > Not to forget the code for C++ support. And, of course even static > binaries may call dlopen() and friends. So that dl*() and the ELF bits > right there. At least in 9.3, dlopen() in a static binary does not work. Try using a NSS module

Re: Trivial program size inflation

2023-07-01 Thread Jason Thorpe
> On Jul 1, 2023, at 1:41 PM, Michael van Elst wrote: > > thor...@me.com (Jason Thorpe) writes: > >> Obviously this is not feasible to do with static binaries > > In the world of go, you skip libc and static binaries are the norm. Go sounds poorly designed, then. -- thorpej

Re: Trivial program size inflation

2023-07-01 Thread RVP
On Sat, 1 Jul 2023, Mouse wrote: But that support shouldn't be brought in if the program doesn't use i18n, NSS, or whatever. As in, sure, let's say i18n_something.o refers to dlopen. But if i18n_something.o itself is not brought in, its reference to dlopen won't be brought in either. Even

Re: Trivial program size inflation

2023-07-01 Thread Michael van Elst
thor...@me.com (Jason Thorpe) writes: >Obviously this is not feasible to do with static binaries In the world of go, you skip libc and static binaries are the norm.

Re: Trivial program size inflation

2023-07-01 Thread Jason Thorpe
> On Jul 1, 2023, at 8:20 AM, tlaro...@polynum.com wrote: > > This is also what I meant by "static seems to be considered deprecated". Honestly, I find the obsession with static linking hilariously quaint. NetBSD already bends backwards to an extreme degree by ensuring that old version of

Re: Trivial program size inflation

2023-07-01 Thread Jason Thorpe
> On Jul 1, 2023, at 8:51 AM, Mouse wrote: > > Why TLS?? Is there some networking going on under the hood? Thread Local Storage -- thorpej

Re: Trivial program size inflation

2023-07-01 Thread Mouse
>> Oh, I didn't mean the program needing to call dlopen() directly. >> libc itself may load shared objects to support things like i18n and >> NSS on an as-needed basis. But that support shouldn't be brought in if the program doesn't use i18n, NSS, or whatever. As in, sure, let's say

Re: Trivial program size inflation

2023-07-01 Thread Michael van Elst
mo...@rodents-montreal.org (Mouse) writes: > amd64, 9.0_STABLE (ftp.n.o): >textdata bss dec hex filename > 562318 29064 2176416 2767798 2a3bb6 main >> amd64, 9.0_STABLE: >>data bss dec hex filename >>2873 186 723131 c3b

Re: Trivial program size inflation

2023-07-01 Thread Mouse
amd64, 9.0_STABLE (ftp.n.o): textdata bss dec hex filename 562318 29064 2176416 2767798 2a3bb6 main > amd64, 9.0_STABLE: >data bss dec hex filename >2873 186 723131 c3b a.out Is that linked dynamic, or did you stub out

Re: Trivial program size inflation

2023-07-01 Thread tlaronde
Le Sat, Jul 01, 2023 at 03:00:21PM +, RVP a écrit : > On Sat, 1 Jul 2023, Mouse wrote: > > > dlopen, that doesn't make sense to me. For a statically linked > > program, the linker can tell whether it calls dlopen et al. > > > > Oh, I didn't mean the program needing to call dlopen()

Re: Trivial program size inflation

2023-07-01 Thread Michael van Elst
mo...@rodents-montreal.org (Mouse) writes: >>> amd64, 9.0_STABLE (ftp.n.o): >>>textdata bss dec hex filename >>> 562318 29064 2176416 2767798 2a3bb6 main amd64, 9.0_STABLE: data bss dec hex filename 2873 186 723131 c3b a.out crt0

Re: Trivial program size inflation

2023-07-01 Thread RVP
On Sat, 1 Jul 2023, Mouse wrote: dlopen, that doesn't make sense to me. For a statically linked program, the linker can tell whether it calls dlopen et al. Oh, I didn't mean the program needing to call dlopen() directly. libc itself may load shared objects to support things like i18n and

Re: Trivial program size inflation

2023-07-01 Thread Mouse
>> [...size of a do-nothing program...] > [...] > Not to forget the code for C++ support. And, of course even static > binaries may call dlopen() and friends. So that dl*() and the ELF > bits right there. dlopen, that doesn't make sense to me. For a statically linked program, the linker can

Re: Trivial program size inflation

2023-07-01 Thread RVP
On Fri, 30 Jun 2023, Mouse wrote: Based on something at work, I was looking at executable sizes. I eventually tried a program stripped about as far down as I could: int main(void); int main(void) { return(0); } and built it -static. size on the resulting binary: [...] amd64, 9.0_STABLE

Re: Trivial program size inflation

2023-07-01 Thread Taylor R Campbell
> Date: Sat, 1 Jul 2023 06:57:40 -0700 > From: Jason Thorpe > > Oh look, the entirety of jemalloc seems to be included in the > binary. WTF knows why that's happening, but apparently it is, and > jemalloc pulls in a ton of additional stuff. Comes in from at least two different things: 1.

Re: Trivial program size inflation

2023-07-01 Thread Jason Thorpe
> On Jun 30, 2023, at 11:30 PM, tlaro...@polynum.com wrote: > > Le Fri, Jun 30, 2023 at 01:37:10PM -0400, Mouse a écrit : >> Based on something at work, I was looking at executable sizes. I >> eventually tried a program stripped about as far down as I could: >> >> int main(void); >> int

Re: Trivial program size inflation

2023-07-01 Thread Mouse
>> and built it -static. size on the resulting binary: >> sparc, my mutant 1.4T: >> textdatabss dec hex filename >> 12616 124 288 13028 32e4main >> amd64, my mutant 5.2: >>text data bss dec hex filename >> 152613 4416

Re: Trivial program size inflation

2023-07-01 Thread tlaronde
Le Fri, Jun 30, 2023 at 01:37:10PM -0400, Mouse a écrit : > Based on something at work, I was looking at executable sizes. I > eventually tried a program stripped about as far down as I could: > > int main(void); > int main(void) > { > return(0); > } > > and built it -static. size on the

Trivial program size inflation

2023-06-30 Thread Mouse
Based on something at work, I was looking at executable sizes. I eventually tried a program stripped about as far down as I could: int main(void); int main(void) { return(0); } and built it -static. size on the resulting binary: sparc, my mutant 1.4T: textdatabss dec hex