Re: ld.so speedup for large binaries with many shared libraries
On Sat, 23 Apr 2011, Dale Rahn wrote: Here is a diff that was originally hatched at c2k10 and finally implemented at k2k11. This has been tested lightly so needs to be tested on all systems with big and small programs. On some machines this can shave 15% off of the startup time of large applications with lots of dynamically loaded libraries. Please test and let me know if there are any problems found. Yes I am intentionally cross posting this to ports@ as large ports are the most affected by this diff. Maybe this will finally get ajacoutot@ off my back ;) Heh ;-) Anyway, I've been running with several variations of that diff on some machines (i386 and macppc) for several weeks without seeing any regressions. And I can confirm large beasts do benefit from it. -- Antoine
Maggio in promozione
MAGGIO IN PROMOZIONE Ancora un regalo da Hermitage LAST MINUTE Nello splendido scenario delle Colline Senesi 3 GIORNI 2 NOTTI EURO 79.00 SERVIZI INCLUSI: Camera doppia o matrimoniale con servizi privati, tv color, telefono diretto . 1 Sola notte Euro 49.00 Camera doppia o matrimoniale con servizi privati, tv color, telefono diretto Pranzo o cena incluse bevande b, 19.00 MAGGIO DAL 2 AL 30 ( http://www.hermitagehotel.it/ ) ( http://www.hermitagehotel.it/ ) Escursioni in partenza da Chianciano Terme: Montepulciano, Pienza, Monticchiello, Cortona, Val DbOrcia, San QuiricoDbOrcia,Montalcino, Bagni Vignoni, San Casciano dei Bagni,Abbadia di Monte Oliveto Maggiore, Siena, San Gimignano, Arezzo,Firenze, Roma, CittC della Pieve, Orvieto, Perugia, Assisi, Lago Trasimeno. Trattamenti termali-benessere per: b San Casciano dei Bagni, Bagni Vignoni, San Filippo, Montepulciano e Chianciano Termeb B Direzione Hermitage Hotel Info Booking: Hermitage Hotel Via P. Ingegnoli, 39 - 53042 Chianciano Terme (SI) Tel/Fax : 0578/62606 E-mail [demime 1.01d removed an attachment of type application/octet-stream which had a name of Week-end 2011.pdf]
Re: malloc: speed vs randomization
On Tue, Apr 26, 2011 at 09:13:47PM +0200, Otto Moerbeek wrote: Second version of diff. This is a conservative one, i.e. it does not change randomization in any way. The diff achieves a speedup by: - Move from long units to short, making the test for a complete non-free unit more effective. - In the randomization loop, also consider whole units instead of always doing it bit by bit. Got only one test report so far :-( Tested myself on amd64, i386, loongson, vax, hppa, and alpha. On a particular test all but the last two show a nice speedup. hppa and alpha only show a marginal speedup. I think this is ready to go in. -Otto
Re: [PATCH] Bypass routing table for mcast packets when using IP_MULTICAST_IF
On Thu, Apr 07, 2011 at 10:47:41AM -0300, Christiano F. Haesbaert wrote:
Keeping this up.
On 17 March 2011 00:51, Christiano F. Haesbaert haesba...@haesbaert.org
wrote:
On 12 June 2010 16:26, Christiano F. Haesbaert haesba...@haesbaert.org
wrote:
Hi,
The following will disregard the routing table for multicast packets
when the application chose the interface with IP_MULTICAST_IF, if not,
normal lookup will take place.
Ripd now no longer needs to explicitly add the mcast host route to
bypass the default reject to 224/4, follows the diff to remove it.
Regarding the normal lookup when IP_MULTICAST_IF wan't set, UNP
regards it as a BSD descendant behaviour, I really don't know anything
else.
I've tested it in a small setup, if someone could confirm it in a
scarier environment would be nice.
What about this version? It moves the multicast source selection before
the route selection. IMO this is a nicer way to write this instead of
making an already complex if statement totaly unreadable.
In the end this will help more daemons to run with multicast_host=NO
when doing multicast. It also matches the behaviour of SOCK_RAW sockets
since those bypass this code and so don't suffer from the blackhole route
added by /etc/netstart.
--
:wq Claudio
Index: in_pcb.c
===
RCS file: /cvs/src/sys/netinet/in_pcb.c,v
retrieving revision 1.119
diff -u -p -r1.119 in_pcb.c
--- in_pcb.c28 Apr 2011 09:56:27 - 1.119
+++ in_pcb.c28 Apr 2011 10:35:38 -
@@ -824,6 +824,27 @@ in_selectsrc(struct sockaddr_in *sin, st
ia = (struct in_ifaddr *)0;
/*
+* If the destination address is multicast and an outgoing
+* interface has been set as a multicast option, use the
+* address of that interface as our source address.
+*/
+ if (IN_MULTICAST(sin-sin_addr.s_addr) mopts != NULL) {
+ struct ifnet *ifp;
+
+ if (mopts-imo_multicast_ifp != NULL) {
+ ifp = mopts-imo_multicast_ifp;
+ TAILQ_FOREACH(ia, in_ifaddr, ia_list)
+ if (ia-ia_ifp == ifp
+ rtable_l2(rtableid) == ifp-if_rdomain)
+ break;
+ if (ia == 0) {
+ *errorp = EADDRNOTAVAIL;
+ return NULL;
+ }
+ return satosin(ia-ia_addr);
+ }
+ }
+ /*
* If route is known or can be allocated now,
* our src addr is taken from the i/f, else punt.
*/
@@ -872,27 +893,6 @@ in_selectsrc(struct sockaddr_in *sin, st
if (ia == 0) {
*errorp = EADDRNOTAVAIL;
return NULL;
- }
- }
- /*
-* If the destination address is multicast and an outgoing
-* interface has been set as a multicast option, use the
-* address of that interface as our source address.
-*/
- if (IN_MULTICAST(sin-sin_addr.s_addr) mopts != NULL) {
- struct ip_moptions *imo;
- struct ifnet *ifp;
-
- imo = mopts;
- if (imo-imo_multicast_ifp != NULL) {
- ifp = imo-imo_multicast_ifp;
- TAILQ_FOREACH(ia, in_ifaddr, ia_list)
- if (ia-ia_ifp == ifp)
- break;
- if (ia == 0) {
- *errorp = EADDRNOTAVAIL;
- return NULL;
- }
}
}
return satosin(ia-ia_addr);
systat ifstat - undocumented options
systat(1) has undocumented ifstat options. Shall we document them? Check out if_keyboard_callback() in if.c for the code. oks,comments? -mark PS Perhaps this is another pointless diff ;) Apologies in advance. Index: systat.1 === RCS file: /cvs/src/usr.bin/systat/systat.1,v retrieving revision 1.91 diff -u -p -r1.91 systat.1 --- systat.15 Apr 2011 07:35:32 - 1.91 +++ systat.128 Apr 2011 11:25:58 - @@ -259,9 +259,20 @@ represent whether the interface is conne in the case of .Xr carp 4 interfaces, whether the interface is in master or backup state, respectively. +.Pp The character .Ic B changes the counter view between bytes and bits. +Pressing +.Ic b +displays statistics as calculated from boot time. +.Ic r +changes the counters to show their totals as calculated +between display refreshes. +.Ic t +changes the counters to show the average per second over +the display refresh interval, +this is the default. .\See below for more options. .It Ic iostat Display statistics about disk throughput.
Re: systat ifstat - undocumented options
On Thu, Apr 28, 2011 at 04:37:49PM +0500, Mark Lumsden wrote: systat(1) has undocumented ifstat options. Shall we document them? Check out if_keyboard_callback() in if.c for the code. oks,comments? -mark PS Perhaps this is another pointless diff ;) Apologies in advance. Index: systat.1 === RCS file: /cvs/src/usr.bin/systat/systat.1,v retrieving revision 1.91 diff -u -p -r1.91 systat.1 --- systat.1 5 Apr 2011 07:35:32 - 1.91 +++ systat.1 28 Apr 2011 11:25:58 - @@ -259,9 +259,20 @@ represent whether the interface is conne in the case of .Xr carp 4 interfaces, whether the interface is in master or backup state, respectively. +.Pp The character .Ic B changes the counter view between bytes and bits. +Pressing +.Ic b +displays statistics as calculated from boot time. +.Ic r +changes the counters to show their totals as calculated +between display refreshes. +.Ic t +changes the counters to show the average per second over +the display refresh interval, +this is the default. .\See below for more options. .It Ic iostat Display statistics about disk throughput. i'm ok with this diff, but you want a semicolon, not comma, before this is the default. or start a new sentence. i wonder about the .\See below for more options. line though. it might be worth checking cvs to see whether they were once there, and were removed, or some other reason. if not, you could probably just remove the comment. jmc
Re: [PATCH] Bypass routing table for mcast packets when using IP_MULTICAST_IF
On 28 April 2011 07:42, Claudio Jeker cje...@diehard.n-r-g.com wrote:
On Thu, Apr 07, 2011 at 10:47:41AM -0300, Christiano F. Haesbaert wrote:
Keeping this up.
On 17 March 2011 00:51, Christiano F. Haesbaert haesba...@haesbaert.org
wrote:
On 12 June 2010 16:26, Christiano F. Haesbaert haesba...@haesbaert.org
wrote:
Hi,
The following will disregard the routing table for multicast packets
when the application chose the interface with IP_MULTICAST_IF, if not,
normal lookup will take place.
Ripd now no longer needs to explicitly add the mcast host route to
bypass the default reject to 224/4, follows the diff to remove it.
Regarding the normal lookup when IP_MULTICAST_IF wan't set, UNP
regards it as a BSD descendant behaviour, I really don't know anything
else.
I've tested it in a small setup, if someone could confirm it in a
scarier environment would be nice.
What about this version? It moves the multicast source selection before
the route selection. IMO this is a nicer way to write this instead of
making an already complex if statement totaly unreadable.
In the end this will help more daemons to run with multicast_host=NO
when doing multicast. It also matches the behaviour of SOCK_RAW sockets
since those bypass this code and so don't suffer from the blackhole route
added by /etc/netstart.
--
:wq Claudio
Index: in_pcb.c
===
RCS file: /cvs/src/sys/netinet/in_pcb.c,v
retrieving revision 1.119
diff -u -p -r1.119 in_pcb.c
--- in_pcb.c28 Apr 2011 09:56:27 - 1.119
+++ in_pcb.c28 Apr 2011 10:35:38 -
@@ -824,6 +824,27 @@ in_selectsrc(struct sockaddr_in *sin, st
ia = (struct in_ifaddr *)0;
/*
+* If the destination address is multicast and an outgoing
+* interface has been set as a multicast option, use the
+* address of that interface as our source address.
+*/
+ if (IN_MULTICAST(sin-sin_addr.s_addr) mopts != NULL) {
+ struct ifnet *ifp;
+
+ if (mopts-imo_multicast_ifp != NULL) {
+ ifp = mopts-imo_multicast_ifp;
+ TAILQ_FOREACH(ia, in_ifaddr, ia_list)
+ if (ia-ia_ifp == ifp
+ rtable_l2(rtableid) == ifp-if_rdomain)
+ break;
+ if (ia == 0) {
+ *errorp = EADDRNOTAVAIL;
+ return NULL;
+ }
+ return satosin(ia-ia_addr);
+ }
+ }
+ /*
* If route is known or can be allocated now,
* our src addr is taken from the i/f, else punt.
*/
@@ -872,27 +893,6 @@ in_selectsrc(struct sockaddr_in *sin, st
if (ia == 0) {
*errorp = EADDRNOTAVAIL;
return NULL;
- }
- }
- /*
-* If the destination address is multicast and an outgoing
-* interface has been set as a multicast option, use the
-* address of that interface as our source address.
-*/
- if (IN_MULTICAST(sin-sin_addr.s_addr) mopts != NULL) {
- struct ip_moptions *imo;
- struct ifnet *ifp;
-
- imo = mopts;
- if (imo-imo_multicast_ifp != NULL) {
- ifp = imo-imo_multicast_ifp;
- TAILQ_FOREACH(ia, in_ifaddr, ia_list)
- if (ia-ia_ifp == ifp)
- break;
- if (ia == 0) {
- *errorp = EADDRNOTAVAIL;
- return NULL;
- }
}
}
return satosin(ia-ia_addr);
Oh yeah, much nicer :D, I'll test it and report back.
Re: ld.so speedup for large binaries with many shared libraries
On 2011/04/28 08:45, Antoine Jacoutot wrote: On Sat, 23 Apr 2011, Dale Rahn wrote: Here is a diff that was originally hatched at c2k10 and finally implemented at k2k11. This has been tested lightly so needs to be tested on all systems with big and small programs. On some machines this can shave 15% off of the startup time of large applications with lots of dynamically loaded libraries. Please test and let me know if there are any problems found. Yes I am intentionally cross posting this to ports@ as large ports are the most affected by this diff. Maybe this will finally get ajacoutot@ off my back ;) Heh ;-) Anyway, I've been running with several variations of that diff on some machines (i386 and macppc) for several weeks without seeing any regressions. And I can confirm large beasts do benefit from it. No problems with light testing on arm, it's very slightly faster. e.g. starting /usr/local/bin/vim (console version from the gtk+2 package i.e. it's linked with a zillion X-related libs) going from around 1.91s before the diff to 1.88s after. Been running it since you posted it on amd64, no problems seen in normal use.
facture EDF 04/2011
Vous n'arrivez pas ` voir les images de votre newsletter, consultez-la en ligne Sarenza.com, Dij` 1 million de paires vendues !
Importador Directo - Cámaras y Filmadoras
USD 359Panasonic TZ10 / ZS7Caacute;mara digital compacta, visor electroacute;nico / Sensor CCD de 12,20 MP efectivos / Objetivo (en 35 mm) 28,0 - 504,0 mm / Zoom 18x (oacute;ptico) / Soportes compatibles SD Card, SDHC / Pantalla LCD de 3,00 pulgadas USD 345Fuji S2800 HDCaacute;mara digital compacta / visor electroacute;nico / Sensor CCD de 14,00 MP efectivos / Objetivo (en 35 mm) 28,0 - 504,0 mm Zoom 18x (oacute;ptico) / Soportes compatibles SD Card, SDHC Pantalla TFT de 3,00 pulgadas
Re: ld.so speedup for large binaries with many shared libraries
Makes Eclipse start in 13-14 secs instead of 19. Thanks.
Re: malloc: speed vs randomization
On Thu, Apr 28, 2011 at 11:12:40AM +0200, Otto Moerbeek wrote: On Tue, Apr 26, 2011 at 09:13:47PM +0200, Otto Moerbeek wrote: Second version of diff. This is a conservative one, i.e. it does not change randomization in any way. The diff achieves a speedup by: - Move from long units to short, making the test for a complete non-free unit more effective. - In the randomization loop, also consider whole units instead of always doing it bit by bit. Got only one test report so far :-( Tested myself on amd64, i386, loongson, vax, hppa, and alpha. On a particular test all but the last two show a nice speedup. hppa and alpha only show a marginal speedup. I think this is ready to go in. -Otto Working fine on my i386 box. Nothing else to test with at the moment. Ken
Re: Fix running ldd against multiple shared objects
Diff below fixes ldd /usr/lib/*.so.* so that it outputs more than
just the first shared object's dependencies, like the behavior from
ldd /usr/bin/*.
The issue is that dlopen(f, RTLD_TRACE) calls exit() after it's done.
I looked into changing this to properly cleanup and return, but
figured it was easier to just fork first like we already do for
exec()-based tracing.
I was checking this diff after the libc.so and libm.so change today, to
verify what's on the machine.
ldd -x is unused, so I removed it from manpage and ldd.c, diff attached at
the very end of this email.
when you do ldd /usr/lib/*.so.* you get a funny output
/usr/lib/libasn1.so.18.0:
StartEnd Type Open Ref GrpRef Name
000207297000 000207735000 dlib 10 0
/usr/lib/libasn1.so.18.0
/usr/lib/libc.so.58.0:
StartEnd Type Open Ref GrpRef Name
00020151d000 000201a03000 dlib 10 0
/usr/lib/libc.so.58.2
/usr/lib/libc.so.58.1:
StartEnd Type Open Ref GrpRef Name
000203abe000 000203fa4000 dlib 10 0
/usr/lib/libc.so.58.2
/usr/lib/libc.so.58.2:
StartEnd Type Open Ref GrpRef Name
000208fc4000 0002094aa000 dlib 10 0
/usr/lib/libc.so.58.2
/usr/lib/libc.so.59.1:
/usr/lib/libcom_err.so.18.0:
StartEnd Type Open Ref GrpRef Name
000203b1 000203fae000 dlib 10 0
/usr/lib/libcom_err.so.18.0
/usr/lib/libkvm.so.10.0:
StartEnd Type Open Ref GrpRef Name
000206891000 000206c9a000 dlib 10 0
/usr/lib/libkvm.so.10.0
/usr/lib/libm.so.5.2:
StartEnd Type Open Ref GrpRef Name
00020d6b1000 00020dacf000 dlib 10 0
/usr/lib/libm.so.5.3
/usr/lib/libm.so.5.3:
StartEnd Type Open Ref GrpRef Name
0002042ef000 00020470d000 dlib 10 0
/usr/lib/libm.so.5.3
/usr/lib/libmenu.so.5.0:
Notice that ldd now prints the correct file and the Start + End
sections are different but it prints the wrong filename for the Start +
End sections.
ldd now prints libc.so.58.2 for all Start + End sections and is silent
about current libc.so.59.1
ldd now prints libm.so.5.3 for both libm.so.5.2 libm.so.5.3
I checked freebsd changelog and saw they mention that ldd has problem
handling libc.so because ldd itself is linked with libc.so
http://svnweb.freebsd.org/base/head/usr.bin/ldd/ldd.c?view=log
Anyway, matthew's diff doesn't bail with exit 1 but bails out silently.
The modified diff which removes -x from manpage and code now follows...
I also deleted some unused headers.
thanks,
amit
Index: ldd.1
===
RCS file: /cvs/src/libexec/ld.so/ldd/ldd.1,v
retrieving revision 1.8
diff -u ldd.1
--- ldd.1 2 Mar 2009 09:27:34 - 1.8
+++ ldd.1 28 Apr 2011 22:20:04 -
@@ -32,7 +32,6 @@
.Nd list dynamic object dependencies
.Sh SYNOPSIS
.Nm ldd
-.Op Fl x
.Ar program ...
.Sh DESCRIPTION
.Nm
@@ -49,13 +48,6 @@
and then execs
.Ar program .
.Pp
-If
-.Nm
-is invoked with the
-.Fl x
-flag, the tags from
-.Ar program
-are listed without using current ldconfig configuration.
.Sh DIAGNOSTICS
Exit status 0 if no error.
Exit status 1 if arg error.
Index: ldd.c
===
RCS file: /cvs/src/libexec/ld.so/ldd/ldd.c,v
retrieving revision 1.14
diff -u ldd.c
--- ldd.c 2 Mar 2009 09:27:34 - 1.14
+++ ldd.c 28 Apr 2011 22:20:04 -
@@ -27,14 +27,10 @@
#include stdio.h
#include stdlib.h
#include elf_abi.h
-#include err.h
#include fcntl.h
-#include string.h
#include unistd.h
#include dlfcn.h
-#include sys/stat.h
-#include sys/mman.h
#include sys/wait.h
#include sys/param.h
@@ -44,23 +40,11 @@
int
main(int argc, char **argv)
{
- int c, xflag, ret;
+ int c, ret;
- xflag = 0;
- while ((c = getopt(argc, argv, x)) != -1) {
- switch (c) {
- case 'x':
- xflag = 1;
- break;
- default:
- usage();
- /*NOTREACHED*/
- }
- }
+ while ((c = getopt(argc, argv, )) != -1)
+ ; /* EMPTY */
- if (xflag)
- errx(1, -x not yet implemented);
-
argc -= optind;
argv += optind;
@@ -84,7 +68,7 @@
{
extern char *__progname;
- fprintf(stderr, usage: %s [-x] program ...\n, __progname);
+ fprintf(stderr, usage: %s program ...\n, __progname);
exit(1);
}
@@ -94,9 +78,9 @@
{
Elf_Ehdr ehdr;
Elf_Phdr *phdr;
- int fd, i, size, status, interp=0;
+ int fd, i, size, status, interp =
