Re: [HACKERS] Examining the output of: ldd `which postgres`
[Up front: yes, I'm following up to a post that's nearly three months old. I can't find any more recent discussion of this issue.] [EMAIL PROTECTED] (Tom Lane) writes: (Of course, if you can show that there's a significant penalty in backend launch time from having useless shlibs linked in, I'd get more excited about it...) How about failure to start at all, when an otherwise-unnecessary shared library is removed from the system? For example, all of our boxes have readline as a non-shared library...except for one. At some point, a newer, non-shared version was installed on this particular machine, and the shared lib was removed. The next time the machine was rebooted, some months later, Postgres wouldn't start due to the missing dependency. I've been re-linking the backend by hand without readline and ncurses after compiling a new version, and just not worrying about the rest of the tools. Today after finding this thread, I decided to see what could be removed. I wrote a short combo of shell and perl to brute-force relink everything in the pgsql/bin directory, to see what could be removed. Boy, was I surprised: Relinking ./src/bin/scripts/clusterdb Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/scripts/createdb Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/scripts/createlang Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/scripts/createuser Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/scripts/dropdb Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/scripts/droplang Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/scripts/dropuser Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/interfaces/ecpg/preproc/ecpg Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/pg_controldata/pg_controldata Successfully removed: -lpq -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/pg_dump/pg_dump Successfully removed: -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/pg_dump/pg_dumpall Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/pg_encoding/pg_encoding Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm -lpgport Relinking ./src/bin/pg_id/pg_id Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm -lpgport Relinking ./src/bin/pg_resetxlog/pg_resetxlog Successfully removed: -lpq -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/pg_dump/pg_restore Successfully removed: -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/backend/postgres Successfully removed: -lz -lreadline -lncurses -lresolv -lnsl Relinking ./src/bin/psql/psql Successfully removed: -lz -lcrypt -lresolv -lnsl -ldl -lm Relinking ./src/bin/scripts/vacuumdb Successfully removed: -lz -lreadline -lncurses -lcrypt -lresolv -lnsl -ldl -lm (Code on request to anyone who wants it, but warning: it's stupid.) It might just be me, but it seems like psql starts up faster without the extraneous libs. It's always fast, but seems instantaneous now. I don't know...it seems crazy, but maybe something like my tool could be included, if you want to relink your setup down to the minimum necessary libraries? Todd -- Todd R. Eigenschink TEK Interactive Group, Inc. [EMAIL PROTECTED] http://www.tekinteractive.com/ System Administrator(260) 459-2521 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Examining the output of: ldd `which postgres`
Sean Chittenden wrote: Backend only forks(). I think you would be better off using Makefile macros to _remove_ those two libraries. I see this: $(filter crypt.o getaddrinfo.o inet_aton.o snprintf.o strerror.o path.o thread.o, $(LIBOBJS)) Seems you need the reverse. Ah, well, if it fork()'s, then I don't really care. The best remaining argument for this would be to reduce the total size of a machine's VM page table size and possibly the expense of switching contexts between procs, but that's a pretty weak argument for only .5M of shared RAM. For some reason I thought it exec()'ed a child with the args necessary for it to read in a postgresql.conf. Looks like the comment in backend/storage/ipc/ipci.c is out of date then: * AttachSharedMemoryAndSemaphores * Attaches to the existing shared resources when exec()'d off * by the postmaster. I have updated this comment. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Examining the output of: ldd `which postgres`
Sean Chittenden wrote: % ldd `which postgres` /usr/local/bin/postgres: libintl.so.5 = /usr/local/lib/libintl.so.5 (0x282e6000) libz.so.2 = /lib/libz.so.2 (0x282ef000) libreadline.so.4 = /lib/libreadline.so.4 (0x282fd000) libcrypt.so.2 = /lib/libcrypt.so.2 (0x28325000) libm.so.2 = /lib/libm.so.2 (0x2833e000) libutil.so.3 = /lib/libutil.so.3 (0x28357000) libc.so.5 = /lib/libc.so.5 (0x28363000) libiconv.so.3 = /usr/local/lib/libiconv.so.3 (0x2843d000) libncurses.so.5 = /lib/libncurses.so.5 (0x2852c000) Is it really necessary for postgres to be linked with ncurses (288K) and readline (156K)? It's .5M, not the end of the world, but it seems excessive. I know the postmaster has a CLI interface, but does it really require ncurses or readline? -sc We add those to all links, mostly because it is too confusing to do it per link. It doesn't hurt anything because it is dynamically linked, so doesn't take any disk space, and in fact is never called. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Examining the output of: ldd `which postgres`
% ldd `which postgres` /usr/local/bin/postgres: libintl.so.5 = /usr/local/lib/libintl.so.5 (0x282e6000) libz.so.2 = /lib/libz.so.2 (0x282ef000) libreadline.so.4 = /lib/libreadline.so.4 (0x282fd000) libcrypt.so.2 = /lib/libcrypt.so.2 (0x28325000) libm.so.2 = /lib/libm.so.2 (0x2833e000) libutil.so.3 = /lib/libutil.so.3 (0x28357000) libc.so.5 = /lib/libc.so.5 (0x28363000) libiconv.so.3 = /usr/local/lib/libiconv.so.3 (0x2843d000) libncurses.so.5 = /lib/libncurses.so.5 (0x2852c000) Is it really necessary for postgres to be linked with ncurses (288K) and readline (156K)? It's .5M, not the end of the world, but it seems excessive. I know the postmaster has a CLI interface, but does it really require ncurses or readline? -sc We add those to all links, mostly because it is too confusing to do it per link. It doesn't hurt anything because it is dynamically linked, so doesn't take any disk space, and in fact is never called. My concern wasn't for disk space, but for symbol resolution times and unnecessary VM page table space. Does the backend fork() or exec() a copy of itself when a new connection comes in? I thought it was exec() for some reason. Anyway, given how easy it is to change the LDFLAGS, I was thinking about chasing down where postgres is linked and splitting apart LDFLAGS into two sets of LDFLAGS: LDFLAGS_CLI and LDFLAGS (or LDFLAGS_DAEMON, or some such). It's chump, but a few ms here and there, or a little more IO there eventually add up, especially in the arena of on connection times. -sc -- Sean Chittenden ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Examining the output of: ldd `which postgres`
Sean Chittenden wrote: We add those to all links, mostly because it is too confusing to do it per link. It doesn't hurt anything because it is dynamically linked, so doesn't take any disk space, and in fact is never called. My concern wasn't for disk space, but for symbol resolution times and unnecessary VM page table space. Does the backend fork() or exec() a copy of itself when a new connection comes in? I thought it was exec() for some reason. Anyway, given how easy it is to change the LDFLAGS, I was thinking about chasing down where postgres is linked and splitting apart LDFLAGS into two sets of LDFLAGS: LDFLAGS_CLI and LDFLAGS (or LDFLAGS_DAEMON, or some such). It's chump, but a few ms here and there, or a little more IO there eventually add up, especially in the arena of on connection times. Backend only forks(). I think you would be better off using Makefile macros to _remove_ those two libraries. I see this: $(filter crypt.o getaddrinfo.o inet_aton.o snprintf.o strerror.o path.o thread.o, $(LIBOBJS)) Seems you need the reverse. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Examining the output of: ldd `which postgres`
We add those to all links, mostly because it is too confusing to do it per link. It doesn't hurt anything because it is dynamically linked, so doesn't take any disk space, and in fact is never called. My concern wasn't for disk space, but for symbol resolution times and unnecessary VM page table space. Does the backend fork() or exec() a copy of itself when a new connection comes in? I thought it was exec() for some reason. Anyway, given how easy it is to change the LDFLAGS, I was thinking about chasing down where postgres is linked and splitting apart LDFLAGS into two sets of LDFLAGS: LDFLAGS_CLI and LDFLAGS (or LDFLAGS_DAEMON, or some such). It's chump, but a few ms here and there, or a little more IO there eventually add up, especially in the arena of on connection times. Backend only forks(). I think you would be better off using Makefile macros to _remove_ those two libraries. I see this: $(filter crypt.o getaddrinfo.o inet_aton.o snprintf.o strerror.o path.o thread.o, $(LIBOBJS)) Seems you need the reverse. Ah, well, if it fork()'s, then I don't really care. The best remaining argument for this would be to reduce the total size of a machine's VM page table size and possibly the expense of switching contexts between procs, but that's a pretty weak argument for only .5M of shared RAM. For some reason I thought it exec()'ed a child with the args necessary for it to read in a postgresql.conf. Looks like the comment in backend/storage/ipc/ipci.c is out of date then: * AttachSharedMemoryAndSemaphores * Attaches to the existing shared resources when exec()'d off * by the postmaster. -sc -- Sean Chittenden ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
