Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Sat, 14 Aug 2010 17:32:25 +0300 Kostik Belousov kostik...@gmail.com wrote: On Sat, Aug 07, 2010 at 09:37:22PM +0200, Marius Strobl wrote: On Sat, Aug 07, 2010 at 09:09:04PM +0300, Kostik Belousov wrote: On Sat, Aug 07, 2010 at 03:59:39PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 02:11:31PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 01:08:08PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? I would suggest to build rtld and libc with debugging symbols and get full backtrace from the faults. v100# gdb /sbin/ifconfig ifconfig.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as sparc64-marcel-freebsd... Core was generated by `ifconfig'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libbsdxml.so.4...done. Loaded symbols for /lib/libbsdxml.so.4 Reading symbols from /lib/libjail.so.1...done. Loaded symbols for /lib/libjail.so.1 Reading symbols from /lib/libsbuf.so.5...done. Loaded symbols for /lib/libsbuf.so.5 Reading symbols from /lib/libipx.so.5...done. Loaded symbols for /lib/libipx.so.5 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 75
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Sat, Aug 14, 2010 at 05:32:25PM +0300, Kostik Belousov wrote: On Sat, Aug 07, 2010 at 09:37:22PM +0200, Marius Strobl wrote: On Sat, Aug 07, 2010 at 09:09:04PM +0300, Kostik Belousov wrote: On Sat, Aug 07, 2010 at 03:59:39PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 02:11:31PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 01:08:08PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? I would suggest to build rtld and libc with debugging symbols and get full backtrace from the faults. v100# gdb /sbin/ifconfig ifconfig.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as sparc64-marcel-freebsd... Core was generated by `ifconfig'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libbsdxml.so.4...done. Loaded symbols for /lib/libbsdxml.so.4 Reading symbols from /lib/libjail.so.1...done. Loaded symbols for /lib/libjail.so.1 Reading symbols from /lib/libsbuf.so.5...done. Loaded symbols for /lib/libsbuf.so.5 Reading symbols from /lib/libipx.so.5...done. Loaded symbols for /lib/libipx.so.5 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Sat, Aug 07, 2010 at 09:37:22PM +0200, Marius Strobl wrote: On Sat, Aug 07, 2010 at 09:09:04PM +0300, Kostik Belousov wrote: On Sat, Aug 07, 2010 at 03:59:39PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 02:11:31PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 01:08:08PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? I would suggest to build rtld and libc with debugging symbols and get full backtrace from the faults. v100# gdb /sbin/ifconfig ifconfig.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as sparc64-marcel-freebsd... Core was generated by `ifconfig'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libbsdxml.so.4...done. Loaded symbols for /lib/libbsdxml.so.4 Reading symbols from /lib/libjail.so.1...done. Loaded symbols for /lib/libjail.so.1 Reading symbols from /lib/libsbuf.so.5...done. Loaded symbols for /lib/libsbuf.so.5 Reading symbols from /lib/libipx.so.5...done. Loaded symbols for /lib/libipx.so.5 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 75 while (nops 0 ps[nops - 1] == 0) (gdb) bt #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Fri, Aug 06, 2010 at 02:11:31PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 01:08:08PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? I would suggest to build rtld and libc with debugging symbols and get full backtrace from the faults. v100# gdb /sbin/ifconfig ifconfig.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as sparc64-marcel-freebsd... Core was generated by `ifconfig'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libbsdxml.so.4...done. Loaded symbols for /lib/libbsdxml.so.4 Reading symbols from /lib/libjail.so.1...done. Loaded symbols for /lib/libjail.so.1 Reading symbols from /lib/libsbuf.so.5...done. Loaded symbols for /lib/libsbuf.so.5 Reading symbols from /lib/libipx.so.5...done. Loaded symbols for /lib/libipx.so.5 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 75 while (nops 0 ps[nops - 1] == 0) (gdb) bt #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 #1 0x407f4314 in malloc_init () at /usr/home/marius/co/head/src/lib/libc/stdlib/malloc.c:5418 #2 0x407f67d8 in malloc (size=32) at /usr/home/marius/co/head/src/lib/libc/stdlib/malloc.c:5932 #3 0x001069ac in clone_setdefcallback (ifprefix=0x11b8a8 wlan, p=0x10a1a0 wlan_create) at /usr/home/marius/co/head/src/sbin/ifconfig/ifclone.c:106 #4 0x00119864 in __do_global_ctors_aux () #5 0x0010243c in _init () #6 0x00102508 in _start () #7 0x4022719c in .rtld_start ()
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Sat, Aug 07, 2010 at 03:59:39PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 02:11:31PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 01:08:08PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? I would suggest to build rtld and libc with debugging symbols and get full backtrace from the faults. v100# gdb /sbin/ifconfig ifconfig.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as sparc64-marcel-freebsd... Core was generated by `ifconfig'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libbsdxml.so.4...done. Loaded symbols for /lib/libbsdxml.so.4 Reading symbols from /lib/libjail.so.1...done. Loaded symbols for /lib/libjail.so.1 Reading symbols from /lib/libsbuf.so.5...done. Loaded symbols for /lib/libsbuf.so.5 Reading symbols from /lib/libipx.so.5...done. Loaded symbols for /lib/libipx.so.5 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 75 while (nops 0 ps[nops - 1] == 0) (gdb) bt #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 #1 0x407f4314 in malloc_init () at /usr/home/marius/co/head/src/lib/libc/stdlib/malloc.c:5418 #2 0x407f67d8 in malloc (size=32) at /usr/home/marius/co/head/src/lib/libc/stdlib/malloc.c:5932 #3 0x001069ac in clone_setdefcallback (ifprefix=0x11b8a8 wlan, p=0x10a1a0 wlan_create) at
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Sat, Aug 07, 2010 at 09:09:04PM +0300, Kostik Belousov wrote: On Sat, Aug 07, 2010 at 03:59:39PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 02:11:31PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 01:08:08PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? I would suggest to build rtld and libc with debugging symbols and get full backtrace from the faults. v100# gdb /sbin/ifconfig ifconfig.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as sparc64-marcel-freebsd... Core was generated by `ifconfig'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libbsdxml.so.4...done. Loaded symbols for /lib/libbsdxml.so.4 Reading symbols from /lib/libjail.so.1...done. Loaded symbols for /lib/libjail.so.1 Reading symbols from /lib/libsbuf.so.5...done. Loaded symbols for /lib/libsbuf.so.5 Reading symbols from /lib/libipx.so.5...done. Loaded symbols for /lib/libipx.so.5 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 75 while (nops 0 ps[nops - 1] == 0) (gdb) bt #0 0x4089ebdc in getpagesizes (pagesize=0x7fde2f8, nelem=1) at /usr/home/marius/co/head/src/lib/libc/gen/getpagesizes.c:75 #1 0x407f4314 in malloc_init () at /usr/home/marius/co/head/src/lib/libc/stdlib/malloc.c:5418 #2 0x407f67d8 in malloc (size=32)
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. pgpF0UGP8pgpd.pgp Description: PGP signature
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Fri, Aug 06, 2010 at 01:08:08PM +0200, Marius Strobl wrote: On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? I would suggest to build rtld and libc with debugging symbols and get full backtrace from the faults. pgpsfXFxdD7Ze.pgp Description: PGP signature
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Fri, Aug 06, 2010 at 12:04:04PM +0300, Kostik Belousov wrote: On Fri, Aug 06, 2010 at 07:06:33AM +0200, Jeremie Le Hen wrote: Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. I cannot move forward without the help from somebody having access to sparc64 system where the problem is reproducable. Do you have a debug version of the patch which outputs the necessary information? Marius ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
Hi Kib, In-Reply-To: 20100629083901.gg13...@deviant.kiev.zoral.com.ua On Tue, Jun 29, 2010 at 11:39:01AM +0300, Kostik Belousov wrote: On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. Did yu have time to look at this problem? It would be nice to have this in the tree. Thanks, -- Jeremie Le Hen Humans are born free and equal. But some are more equal than others. Coluche ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Tue, Jun 29, 2010 at 10:26:39AM +0200, Marius Strobl wrote: On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Thank you. pgpn7RdK9PZkN.pgp Description: PGP signature
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Mon, Jun 28, 2010 at 05:48:59PM +0300, Kostik Belousov wrote: On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote: Hi Kostik, This patch seems to have faded out from memory. Is it possible to go forward and commit it? I refreshed the patch. Hopefully, nobody will object, and I commit it shortly. Thanks, Regards. On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. Patch completely eliminates sysctl syscalls from the rtld and libc startup. Without the patch, a single run of /bin/ls did 6 sysctls, with the patch, no sysctls is queried at all. Comparing with the originally posted patch, I added support for all architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3) that added two more startup sysctls. Would be nice to get a testing for at least some !x86 architectures before the commit, I added some people who helped me in past, to the Cc:. Doesn't look good on sparc64: ... NFS ROOT: 192.168.1.40:/usr/data/nfsroot/sparc64 dc1: link state changed to UP pid 24 (ifconfig), uid 0: exited on signal 11 Segmentation fault Interface IP-Address Broadcast pid 29 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 30 (grep), uid 0: exited on signal 11 Segmentation fault pid 31 (rcorder), uid 0: exited on signal 11 Segmentation fault pid 32 (date), uid 0: exited on signal 11 Segmentation fault Jun 29 12:20:50 getty[36]: open /dev/ttyv3: No such file or directory ... Unfortunately, I currently lack the time to debug this. Marius ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
On Wed, Jun 23, 2010 at 11:09:59PM +0200, Jeremie Le Hen wrote:
Hi Kostik,
This patch seems to have faded out from memory. Is it possible to go
forward and commit it?
I refreshed the patch. Hopefully, nobody will object, and I commit it
shortly.
Thanks,
Regards.
On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote:
Below is the prototype that seems to work for me both with patched and
old rtld on i386. Patch also contains bits for amd64 that I did not
tested yet. All other arches are not buildable for now.
Patch completely eliminates sysctl syscalls from the rtld and libc
startup. Without the patch, a single run of /bin/ls did 6 sysctls,
with the patch, no sysctls is queried at all.
Comparing with the originally posted patch, I added support for all
architectures, tested amd64 and ia32 on amd64, and converted getpagesizes(3)
that added two more startup sysctls.
Would be nice to get a testing for at least some !x86 architectures
before the commit, I added some people who helped me in past, to the Cc:.
diff --git a/lib/libc/gen/__getosreldate.c b/lib/libc/gen/__getosreldate.c
index 7e26845..1bed6d0 100644
--- a/lib/libc/gen/__getosreldate.c
+++ b/lib/libc/gen/__getosreldate.c
@@ -29,6 +29,8 @@ __FBSDID($FreeBSD$);
#include sys/param.h
#include sys/sysctl.h
+#include errno.h
+#include link.h
int __getosreldate(void);
@@ -51,7 +53,15 @@ __getosreldate(void)
if (osreldate != 0)
return (osreldate);
-
+
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_OSRELDATE, osreldate,
+ sizeof(osreldate));
+ else
+ error = ENOSYS;
+ if (error == 0 osreldate != 0)
+ return (osreldate);
+
oid[0] = CTL_KERN;
oid[1] = KERN_OSRELDATE;
osrel = 0;
diff --git a/lib/libc/gen/getpagesize.c b/lib/libc/gen/getpagesize.c
index d796b9d..b8f0ec1 100644
--- a/lib/libc/gen/getpagesize.c
+++ b/lib/libc/gen/getpagesize.c
@@ -36,6 +36,8 @@ __FBSDID($FreeBSD$);
#include sys/param.h
#include sys/sysctl.h
+#include errno.h
+#include link.h
#include unistd.h
/*
@@ -52,13 +54,23 @@ getpagesize()
int mib[2];
static int value;
size_t size;
+ int error;
+
+ if (value != 0)
+ return (value);
+
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_PAGESZ, value, sizeof(value));
+ else
+ error = ENOSYS;
+ if (error == 0 value != 0)
+ return (value);
+
+ mib[0] = CTL_HW;
+ mib[1] = HW_PAGESIZE;
+ size = sizeof value;
+ if (sysctl(mib, 2, value, size, NULL, 0) == -1)
+ return (-1);
- if (!value) {
- mib[0] = CTL_HW;
- mib[1] = HW_PAGESIZE;
- size = sizeof value;
- if (sysctl(mib, 2, value, size, NULL, 0) == -1)
- return (-1);
- }
return (value);
}
diff --git a/lib/libc/gen/getpagesizes.c b/lib/libc/gen/getpagesizes.c
index b0de939..bfeeef8 100644
--- a/lib/libc/gen/getpagesizes.c
+++ b/lib/libc/gen/getpagesizes.c
@@ -32,6 +32,7 @@ __FBSDID($FreeBSD$);
#include sys/sysctl.h
#include errno.h
+#include link.h
/*
* Retrieves page size information from the system. Specifically, returns the
@@ -51,7 +52,7 @@ getpagesizes(size_t pagesize[], int nelem)
static u_long ps[MAXPAGESIZES];
static int nops;
size_t size;
- int i;
+ int error, i;
if (nelem 0 || (nelem 0 pagesize == NULL)) {
errno = EINVAL;
@@ -59,9 +60,16 @@ getpagesizes(size_t pagesize[], int nelem)
}
/* Cache the result of the sysctl(2). */
if (nops == 0) {
- size = sizeof(ps);
- if (sysctlbyname(hw.pagesizes, ps, size, NULL, 0) == -1)
- return (-1);
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_PAGESIZES, ps, sizeof(ps));
+ else
+ error = ENOSYS;
+ if (error != 0 || ps[0] == 0) {
+ size = sizeof(ps);
+ if (sysctlbyname(hw.pagesizes, ps, size, NULL, 0)
+ == -1)
+ return (-1);
+ }
/* Count the number of page sizes that are supported. */
nops = size / sizeof(ps[0]);
while (nops 0 ps[nops - 1] == 0)
diff --git a/lib/libc/stdlib/malloc.c b/lib/libc/stdlib/malloc.c
index 295a168..41be6d8 100644
--- a/lib/libc/stdlib/malloc.c
+++ b/lib/libc/stdlib/malloc.c
@@ -180,6 +180,7 @@ __FBSDID($FreeBSD$);
#include errno.h
#include limits.h
+#include link.h
#include pthread.h
#include sched.h
#include stdarg.h
@@ -5348,14 +5349,23 @@ small_size2bin_init_hard(void)
static unsigned
malloc_ncpus(void)
{
+ int mib[2];
unsigned ret;
-
Re: Avoiding sysctl at program startup using ELF aux vector (was: concurrent sysctl implementation)
Hi Kostik,
This patch seems to have faded out from memory. Is it possible to go
forward and commit it?
Thanks,
Regards.
On Sat, Jul 25, 2009 at 12:29:16AM +0300, Kostik Belousov wrote:
Below is the prototype that seems to work for me both with patched and
old rtld on i386. Patch also contains bits for amd64 that I did not
tested yet. All other arches are not buildable for now.
Patch completely eliminates sysctl syscalls from the rtld and libc
startup. Without the patch, a single run of /bin/ls did 6 sysctls,
with the patch, no sysctls is queried at all.
diff --git a/lib/libc/gen/__getosreldate.c b/lib/libc/gen/__getosreldate.c
index 69aac07..6a4e5ea 100644
--- a/lib/libc/gen/__getosreldate.c
+++ b/lib/libc/gen/__getosreldate.c
@@ -29,6 +29,8 @@ __FBSDID($FreeBSD$);
#include sys/param.h
#include sys/sysctl.h
+#include errno.h
+#include link.h
/*
* This is private to libc. It is intended for wrapping syscall stubs in
order
@@ -49,7 +51,15 @@ __getosreldate(void)
if (osreldate != 0)
return (osreldate);
-
+
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_OSRELDATE, osreldate,
+ sizeof(osreldate));
+ else
+ error = ENOSYS;
+ if (error == 0 osreldate != 0)
+ return (osreldate);
+
oid[0] = CTL_KERN;
oid[1] = KERN_OSRELDATE;
osrel = 0;
diff --git a/lib/libc/gen/getpagesize.c b/lib/libc/gen/getpagesize.c
index d796b9d..b8f0ec1 100644
--- a/lib/libc/gen/getpagesize.c
+++ b/lib/libc/gen/getpagesize.c
@@ -36,6 +36,8 @@ __FBSDID($FreeBSD$);
#include sys/param.h
#include sys/sysctl.h
+#include errno.h
+#include link.h
#include unistd.h
/*
@@ -52,13 +54,23 @@ getpagesize()
int mib[2];
static int value;
size_t size;
+ int error;
+
+ if (value != 0)
+ return (value);
+
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_PAGESZ, value, sizeof(value));
+ else
+ error = ENOSYS;
+ if (error == 0 value != 0)
+ return (value);
+
+ mib[0] = CTL_HW;
+ mib[1] = HW_PAGESIZE;
+ size = sizeof value;
+ if (sysctl(mib, 2, value, size, NULL, 0) == -1)
+ return (-1);
- if (!value) {
- mib[0] = CTL_HW;
- mib[1] = HW_PAGESIZE;
- size = sizeof value;
- if (sysctl(mib, 2, value, size, NULL, 0) == -1)
- return (-1);
- }
return (value);
}
diff --git a/lib/libc/stdlib/malloc.c b/lib/libc/stdlib/malloc.c
index 270d641..e479abe 100644
--- a/lib/libc/stdlib/malloc.c
+++ b/lib/libc/stdlib/malloc.c
@@ -179,6 +179,7 @@ __FBSDID($FreeBSD$);
#include errno.h
#include limits.h
+#include link.h
#include pthread.h
#include sched.h
#include stdarg.h
@@ -4769,7 +4770,10 @@ malloc_init_hard(void)
unsigned i;
int linklen;
char buf[PATH_MAX + 1];
+ int mib[2];
+ size_t len;
const char *opts;
+ int error;
malloc_mutex_lock(init_lock);
if (malloc_initialized) {
@@ -4782,10 +4786,11 @@ malloc_init_hard(void)
}
/* Get number of CPUs. */
- {
- int mib[2];
- size_t len;
-
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_NCPUS, ncpus, sizeof(ncpus));
+ else
+ error = ENOSYS;
+ if (error != 0 || ncpus == 0) {
mib[0] = CTL_HW;
mib[1] = HW_NCPU;
len = sizeof(ncpus);
diff --git a/lib/libc/sys/stack_protector.c b/lib/libc/sys/stack_protector.c
index 63beebc..571f63c 100644
--- a/lib/libc/sys/stack_protector.c
+++ b/lib/libc/sys/stack_protector.c
@@ -34,6 +34,8 @@ __FBSDID($FreeBSD$);
#include sys/param.h
#include sys/sysctl.h
#include sys/types.h
+#include errno.h
+#include link.h
#include signal.h
#include string.h
#include syslog.h
@@ -54,9 +56,17 @@ __guard_setup(void)
{
int mib[2];
size_t len;
+ int error;
if (__stack_chk_guard[0] != 0)
return;
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_CANARY, __stack_chk_guard,
+ sizeof(__stack_chk_guard));
+ else
+ error = ENOSYS;
+ if (error == 0 __stack_chk_guard[0] != 0)
+ return;
mib[0] = CTL_KERN;
mib[1] = KERN_ARND;
diff --git a/libexec/rtld-elf/Symbol.map b/libexec/rtld-elf/Symbol.map
index ce1e3e5..f45f955 100644
--- a/libexec/rtld-elf/Symbol.map
+++ b/libexec/rtld-elf/Symbol.map
@@ -24,4 +24,5 @@ FBSDprivate_1.0 {
_rtld_free_tls;
_rtld_atfork_pre;
_rtld_atfork_post;
+_rtld_aux_info;
};
diff --git a/libexec/rtld-elf/amd64/reloc.c b/libexec/rtld-elf/amd64/reloc.c
index 8a32adf..a510884 100644
--- a/libexec/rtld-elf/amd64/reloc.c
+++ b/libexec/rtld-elf/amd64/reloc.c
@@ -113,7
Re: concurrent sysctl implementation
On Sat, 25 Jul 2009 00:29:16 +0300 Kostik Belousov kostik...@gmail.com wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. [patch deleted] I'm currently running the patch under AMD64 and haven't noticed any regressiosn yet. --- Gary Jennejohn ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
On Sat, Jul 25, 2009 at 03:01:54PM +0200, Gary Jennejohn wrote: On Sat, 25 Jul 2009 00:29:16 +0300 Kostik Belousov kostik...@gmail.com wrote: Below is the prototype that seems to work for me both with patched and old rtld on i386. Patch also contains bits for amd64 that I did not tested yet. All other arches are not buildable for now. [patch deleted] I'm currently running the patch under AMD64 and haven't noticed any regressiosn yet. Thank you. It is interesting how much sysctls are issued for startup. Easier way to see that is to ktrace /bin/ls and then do kdump | grep SCTL. pgpCJLYBhSWmi.pgp Description: PGP signature
Re: concurrent sysctl implementation
Hi Ed, Sorry for the late reply. On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote: We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. Can you elaborate a little? Are you talking about elf-hints.h? I don't see where we can get randomness from it. Thanks. Regards, -- Jeremie Le Hen jeremie at le-hen dot org ttz at chchile dot org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
On Fri, Jul 24, 2009 at 09:34:51AM +0200, Jeremie Le Hen wrote: Hi Ed, Sorry for the late reply. On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote: We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. Can you elaborate a little? Are you talking about elf-hints.h? I don't see where we can get randomness from it. The thing is called ELF auxillary information vector. It is used to supply some useful information for interpreter from the kernel, see include/machine/elf.h for AT_* entries. pgp7Qk5S94jFN.pgp Description: PGP signature
Re: concurrent sysctl implementation
On Fri, Jul 24, 2009 at 11:18:42AM +0300, Kostik Belousov wrote: On Fri, Jul 24, 2009 at 09:34:51AM +0200, Jeremie Le Hen wrote: Hi Ed, Sorry for the late reply. On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote: We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. Can you elaborate a little? Are you talking about elf-hints.h? I don't see where we can get randomness from it. The thing is called ELF auxillary information vector. It is used to supply some useful information for interpreter from the kernel, see include/machine/elf.h for AT_* entries. Ah ok, so the idea is to generate a new hint, for instance AT_RANDOM, generated at link time, that will be used to fill the canary at exec(2) time? Regards, -- Jeremie Le Hen jeremie at le-hen dot org ttz at chchile dot org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
* Jeremie Le Hen jere...@le-hen.org wrote: On Fri, Jul 24, 2009 at 11:18:42AM +0300, Kostik Belousov wrote: On Fri, Jul 24, 2009 at 09:34:51AM +0200, Jeremie Le Hen wrote: Hi Ed, Sorry for the late reply. On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote: We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. Can you elaborate a little? Are you talking about elf-hints.h? I don't see where we can get randomness from it. The thing is called ELF auxillary information vector. It is used to supply some useful information for interpreter from the kernel, see include/machine/elf.h for AT_* entries. Ah ok, so the idea is to generate a new hint, for instance AT_RANDOM, generated at link time, that will be used to fill the canary at exec(2) time? Very short answer: yes! -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ pgpXiKMuleZca.pgp Description: PGP signature
Re: concurrent sysctl implementation
On Fri, Jul 24, 2009 at 01:54:04PM +0200, Jeremie Le Hen wrote: On Fri, Jul 24, 2009 at 11:18:42AM +0300, Kostik Belousov wrote: On Fri, Jul 24, 2009 at 09:34:51AM +0200, Jeremie Le Hen wrote: Hi Ed, Sorry for the late reply. On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote: We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. Can you elaborate a little? Are you talking about elf-hints.h? I don't see where we can get randomness from it. The thing is called ELF auxillary information vector. It is used to supply some useful information for interpreter from the kernel, see include/machine/elf.h for AT_* entries. Ah ok, so the idea is to generate a new hint, for instance AT_RANDOM, generated at link time, that will be used to fill the canary at exec(2) time? The aux entries are not hints, and they are put on the new image stack when execve() activates the image. Aux entries has nothing to do with static link time, they are supplied to the dynamic linker (ELF interpreter). pgpFY5GAPaI2S.pgp Description: PGP signature
Re: concurrent sysctl implementation
On Fri, Jul 24, 2009 at 01:56:49PM +0200, Ed Schouten wrote: * Jeremie Le Hen jere...@le-hen.org wrote: On Fri, Jul 24, 2009 at 11:18:42AM +0300, Kostik Belousov wrote: On Fri, Jul 24, 2009 at 09:34:51AM +0200, Jeremie Le Hen wrote: Hi Ed, Sorry for the late reply. On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote: We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. Can you elaborate a little? Are you talking about elf-hints.h? I don't see where we can get randomness from it. The thing is called ELF auxillary information vector. It is used to supply some useful information for interpreter from the kernel, see include/machine/elf.h for AT_* entries. Ah ok, so the idea is to generate a new hint, for instance AT_RANDOM, generated at link time, that will be used to fill the canary at exec(2) time? Very short answer: yes! Ok thanks. But this would make stack protection useless for local attacks on suid binaries that are world-readable since the attacker could read the ELF aux vector and compute the canary. Upon each invocation, the canary would stay the same which makes the repeat-until-success attack feasible for daemons that are automatically respawned. This saves one syscall per exec(2) but reduce security for the two cases described above. In the performance test I've run with and without -fstack-protector to build world, I saw only around 1 percent penalty. I must admit this was on a UP system which wasn't loaded though. I know that the sysctl system may be redesigned in the future to allow more concurrency. Is it something that could prevent from changing the way the canary is initialized? Regards, -- Jeremie Le Hen jeremie at le-hen dot org ttz at chchile dot org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
* Jeremie Le Hen jere...@le-hen.org wrote: On Fri, Jul 24, 2009 at 01:56:49PM +0200, Ed Schouten wrote: * Jeremie Le Hen jere...@le-hen.org wrote: On Fri, Jul 24, 2009 at 11:18:42AM +0300, Kostik Belousov wrote: On Fri, Jul 24, 2009 at 09:34:51AM +0200, Jeremie Le Hen wrote: Hi Ed, Sorry for the late reply. On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote: We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. Can you elaborate a little? Are you talking about elf-hints.h? I don't see where we can get randomness from it. The thing is called ELF auxillary information vector. It is used to supply some useful information for interpreter from the kernel, see include/machine/elf.h for AT_* entries. Ah ok, so the idea is to generate a new hint, for instance AT_RANDOM, generated at link time, that will be used to fill the canary at exec(2) time? Very short answer: yes! Ok thanks. But this would make stack protection useless for local attacks on suid binaries that are world-readable since the attacker could read the ELF aux vector and compute the canary. Wait wait wait. It seems you were only partially right (and Kostik corrected you): We could add AT_RANDOM, but this value will be filled in by the kernel when starting the process. This means the random value is not stored in the binary. -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ pgpkPBdmMkXBL.pgp Description: PGP signature
Re: concurrent sysctl implementation
On Fri, Jul 24, 2009 at 03:49:53PM +0200, Ed Schouten wrote:
* Jeremie Le Hen jere...@le-hen.org wrote:
On Fri, Jul 24, 2009 at 01:56:49PM +0200, Ed Schouten wrote:
* Jeremie Le Hen jere...@le-hen.org wrote:
On Fri, Jul 24, 2009 at 11:18:42AM +0300, Kostik Belousov wrote:
On Fri, Jul 24, 2009 at 09:34:51AM +0200, Jeremie Le Hen wrote:
Hi Ed,
Sorry for the late reply.
On Sat, May 09, 2009 at 02:13:13PM +0200, Ed Schouten wrote:
We probably could. I think I discussed this with Robert Watson
some time
ago and we could use things like ELF hints. But still, that
doesn't
prevent us from reaching this limitation later on.
Can you elaborate a little? Are you talking about elf-hints.h?
I don't see where we can get randomness from it.
The thing is called ELF auxillary information vector. It is used to
supply some useful information for interpreter from the kernel,
see include/machine/elf.h for AT_* entries.
Ah ok, so the idea is to generate a new hint, for instance AT_RANDOM,
generated at link time, that will be used to fill the canary at exec(2)
time?
Very short answer: yes!
Ok thanks. But this would make stack protection useless for local
attacks on suid binaries that are world-readable since the attacker
could read the ELF aux vector and compute the canary.
Wait wait wait. It seems you were only partially right (and Kostik
corrected you):
We could add AT_RANDOM, but this value will be filled in by the kernel
when starting the process. This means the random value is not stored in
the binary.
Below is the prototype that seems to work for me both with patched and
old rtld on i386. Patch also contains bits for amd64 that I did not
tested yet. All other arches are not buildable for now.
Patch completely eliminates sysctl syscalls from the rtld and libc
startup. Without the patch, a single run of /bin/ls did 6 sysctls,
with the patch, no sysctls is queried at all.
diff --git a/lib/libc/gen/__getosreldate.c b/lib/libc/gen/__getosreldate.c
index 69aac07..6a4e5ea 100644
--- a/lib/libc/gen/__getosreldate.c
+++ b/lib/libc/gen/__getosreldate.c
@@ -29,6 +29,8 @@ __FBSDID($FreeBSD$);
#include sys/param.h
#include sys/sysctl.h
+#include errno.h
+#include link.h
/*
* This is private to libc. It is intended for wrapping syscall stubs in order
@@ -49,7 +51,15 @@ __getosreldate(void)
if (osreldate != 0)
return (osreldate);
-
+
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_OSRELDATE, osreldate,
+ sizeof(osreldate));
+ else
+ error = ENOSYS;
+ if (error == 0 osreldate != 0)
+ return (osreldate);
+
oid[0] = CTL_KERN;
oid[1] = KERN_OSRELDATE;
osrel = 0;
diff --git a/lib/libc/gen/getpagesize.c b/lib/libc/gen/getpagesize.c
index d796b9d..b8f0ec1 100644
--- a/lib/libc/gen/getpagesize.c
+++ b/lib/libc/gen/getpagesize.c
@@ -36,6 +36,8 @@ __FBSDID($FreeBSD$);
#include sys/param.h
#include sys/sysctl.h
+#include errno.h
+#include link.h
#include unistd.h
/*
@@ -52,13 +54,23 @@ getpagesize()
int mib[2];
static int value;
size_t size;
+ int error;
+
+ if (value != 0)
+ return (value);
+
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_PAGESZ, value, sizeof(value));
+ else
+ error = ENOSYS;
+ if (error == 0 value != 0)
+ return (value);
+
+ mib[0] = CTL_HW;
+ mib[1] = HW_PAGESIZE;
+ size = sizeof value;
+ if (sysctl(mib, 2, value, size, NULL, 0) == -1)
+ return (-1);
- if (!value) {
- mib[0] = CTL_HW;
- mib[1] = HW_PAGESIZE;
- size = sizeof value;
- if (sysctl(mib, 2, value, size, NULL, 0) == -1)
- return (-1);
- }
return (value);
}
diff --git a/lib/libc/stdlib/malloc.c b/lib/libc/stdlib/malloc.c
index 270d641..e479abe 100644
--- a/lib/libc/stdlib/malloc.c
+++ b/lib/libc/stdlib/malloc.c
@@ -179,6 +179,7 @@ __FBSDID($FreeBSD$);
#include errno.h
#include limits.h
+#include link.h
#include pthread.h
#include sched.h
#include stdarg.h
@@ -4769,7 +4770,10 @@ malloc_init_hard(void)
unsigned i;
int linklen;
char buf[PATH_MAX + 1];
+ int mib[2];
+ size_t len;
const char *opts;
+ int error;
malloc_mutex_lock(init_lock);
if (malloc_initialized) {
@@ -4782,10 +4786,11 @@ malloc_init_hard(void)
}
/* Get number of CPUs. */
- {
- int mib[2];
- size_t len;
-
+ if (_rtld_aux_info != NULL)
+ error = _rtld_aux_info(AT_NCPUS, ncpus, sizeof(ncpus));
+ else
+ error = ENOSYS;
+ if (error != 0 || ncpus == 0) {
Re: concurrent sysctl implementation
* John Baldwin j...@freebsd.org wrote: Well, in theory a bunch of small requests to SYSCTL_PROC() nodes that used sysctl_wire_old() (or whatever it is called) could cause the amount of user memory wired for sysctls to grow unbounded. Thus, allowing this limited concurrency is a tradeoff as there is a minimal (perhaps only theoretical at the moment) risk of removing the safety net. The patch is quite small, btw, because the locking for the sysctl tree already exists, and by using read locks, one can already allow concurrent sysctl requests. There is no need to add any new locks or restructure the sysctl tree, just to adjust the locking that is already present. It might be clearer, in fact, to split the sysctl memory lock back out into a separate lock. This would allow small sysctl requests to run concurrently with a single large request whereas in my suggestion in the earlier e-mail, the large request will block all other user requests until it finishes. I've actually gone ahead and done this below. Boohoo. I actually wanted jt to work on this, as a small exercise to figure out the way locking primitives work in the kernel. No problem, because I can think of dozens of other things. Is there a chance we can see this patch in 8.0? I like it that the memlock is being picked up before we pick up the sysctl lock itself, which makes a lot of sense. -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ pgpoN9a9RYCSv.pgp Description: PGP signature
Re: concurrent sysctl implementation
On Thursday 14 May 2009 5:34:26 pm Ed Schouten wrote: * John Baldwin j...@freebsd.org wrote: Well, in theory a bunch of small requests to SYSCTL_PROC() nodes that used sysctl_wire_old() (or whatever it is called) could cause the amount of user memory wired for sysctls to grow unbounded. Thus, allowing this limited concurrency is a tradeoff as there is a minimal (perhaps only theoretical at the moment) risk of removing the safety net. The patch is quite small, btw, because the locking for the sysctl tree already exists, and by using read locks, one can already allow concurrent sysctl requests. There is no need to add any new locks or restructure the sysctl tree, just to adjust the locking that is already present. It might be clearer, in fact, to split the sysctl memory lock back out into a separate lock. This would allow small sysctl requests to run concurrently with a single large request whereas in my suggestion in the earlier e-mail, the large request will block all other user requests until it finishes. I've actually gone ahead and done this below. Boohoo. I actually wanted jt to work on this, as a small exercise to figure out the way locking primitives work in the kernel. No problem, because I can think of dozens of other things. Is there a chance we can see this patch in 8.0? I like it that the memlock is being picked up before we pick up the sysctl lock itself, which makes a lot of sense. Yes, I can commit it. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
On Friday 08 May 2009 5:41:17 pm Ed Schouten wrote: A solution would be to solve it as follows: - Use a semaphore, initialized to some insane high value to put an upper limit on the amount of concurrent sysctl calls. I'm not sure whether this is really needed. Maybe this issue is not as serious as we think it is. Well, one compromise might be to allow concurrent userland requests if the buffer size is small (say 1 page). This would be a quite simple change and would cover many common syscalls like fetching an int which don't wire memory anyway. - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up the lock when we traverse parts of the sysctl tree that has dynamically created entries. I don't think further work is needed here for the tree, notice that in-kernel sysctls are already concurrent and use a read lock on the tree. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
John, Thank you for your input on this matter, I'm excited to write some software for this project since its given me great code to learn from as i've grown up (still a kid though :). My questions are a bit more detailed below. On Mon, May 11, 2009 at 12:24 PM, John Baldwin j...@freebsd.org wrote: On Friday 08 May 2009 5:41:17 pm Ed Schouten wrote: A solution would be to solve it as follows: - Use a semaphore, initialized to some insane high value to put an upper limit on the amount of concurrent sysctl calls. I'm not sure whether this is really needed. Maybe this issue is not as serious as we think it is. Well, one compromise might be to allow concurrent userland requests if the buffer size is small (say 1 page). This would be a quite simple change and would cover many common syscalls like fetching an int which don't wire memory anyway. Why is this a compromise? Isn't concurrent sysctl calls from userland a good thing? What materials would be good to read other than the code and the sysctl manual pages? You said it would be relatively easy to implement this; what methods should I be considering to do this in and what part of the code specifically should I be looking at? - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up the lock when we traverse parts of the sysctl tree that has dynamically created entries. I don't think further work is needed here for the tree, notice that in-kernel sysctls are already concurrent and use a read lock on the tree. yes i've seen the locking mechanism, it reminds me of a monitor type system... though from my understanding monitors appear seldom compared to semaphores in the kernel. I assume the lock will need a bit of twiddling with in some areas of the code if I'm going to enable concurrency from userland, when its said that we should consider the things that are dynamic would it be better to implement this with more than one queue or list? Instead perhaps break them up into several lists or, more fundamentally, two lists -- those that are dynamically created entries and those that are not -- is this even feasible to distinguish between the two originally and then on the fly later? Thanks a lot! Respectfully, /jt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
On Monday 11 May 2009 2:27:48 pm j...@0xabadba.be wrote:
John,
Thank you for your input on this matter, I'm excited to write
some software for this project since its given me great code to learn
from as i've grown up (still a kid though :). My questions are a bit
more detailed below.
On Mon, May 11, 2009 at 12:24 PM, John Baldwin j...@freebsd.org wrote:
On Friday 08 May 2009 5:41:17 pm Ed Schouten wrote:
A solution would be to solve it as follows:
- Use a semaphore, initialized to some insane high value to put an upper
limit on the amount of concurrent sysctl calls. I'm not sure whether
this is really needed. Maybe this issue is not as serious as we think
it is.
Well, one compromise might be to allow concurrent userland requests if the
buffer size is small (say 1 page). This would be a quite simple change
and
would cover many common syscalls like fetching an int which don't wire
memory
anyway.
Why is this a compromise? Isn't concurrent sysctl calls from userland
a good thing? What materials would be good to read other than the
code and the sysctl manual pages? You said it would be relatively
easy to implement this; what methods should I be considering to do
this in and what part of the code specifically should I be looking at?
Well, in theory a bunch of small requests to SYSCTL_PROC() nodes that used
sysctl_wire_old() (or whatever it is called) could cause the amount of user
memory wired for sysctls to grow unbounded. Thus, allowing this limited
concurrency is a tradeoff as there is a minimal (perhaps only theoretical at
the moment) risk of removing the safety net.
The patch is quite small, btw, because the locking for the sysctl tree already
exists, and by using read locks, one can already allow concurrent sysctl
requests. There is no need to add any new locks or restructure the sysctl
tree, just to adjust the locking that is already present. It might be
clearer, in fact, to split the sysctl memory lock back out into a separate
lock. This would allow small sysctl requests to run concurrently with a
single large request whereas in my suggestion in the earlier e-mail,
the large request will block all other user requests until it finishes.
I've actually gone ahead and done this below.
--- //depot/projects/smpng/sys/kern/kern_sysctl.c 2009/05/08 11:53:25
+++ //depot/user/jhb/lock/kern/kern_sysctl.c2009/05/11 21:58:08
@@ -77,11 +77,12 @@
* API rather than using the dynamic API. Use of the dynamic API is
* strongly encouraged for most code.
*
- * This lock is also used to serialize userland sysctl requests. Some
- * sysctls wire user memory, and serializing the requests limits the
- * amount of wired user memory in use.
+ * The sysctlmemlock is used to limit the amount of user memory wired for
+ * sysctl requests. This is implemented by serializing any userland
+ * sysctl requests larger than a single page via an exclusive lock.
*/
static struct sx sysctllock;
+static struct sx sysctlmemlock;
#defineSYSCTL_SLOCK() sx_slock(sysctllock)
#defineSYSCTL_SUNLOCK()sx_sunlock(sysctllock)
@@ -543,6 +544,7 @@
{
struct sysctl_oid **oidp;
+ sx_init(sysctlmemlock, sysctl mem);
SYSCTL_INIT();
SYSCTL_XLOCK();
SET_FOREACH(oidp, sysctl_set)
@@ -1563,7 +1565,7 @@
size_t *oldlenp, int inkernel, void *new, size_t newlen, size_t *retval,
int flags)
{
- int error = 0;
+ int error = 0, memlocked;
struct sysctl_req req;
bzero(req, sizeof req);
@@ -1603,14 +1605,20 @@
if (KTRPOINT(curthread, KTR_SYSCTL))
ktrsysctl(name, namelen);
#endif
-
- SYSCTL_XLOCK();
+
+ if (req.oldlen PAGE_SIZE) {
+ memlocked = 1;
+ sx_xlock(sysctlmemlock);
+ } else
+ memlocked = 0;
CURVNET_SET(TD_TO_VNET(curthread));
for (;;) {
req.oldidx = 0;
req.newidx = 0;
+ SYSCTL_SLOCK();
error = sysctl_root(0, name, namelen, req);
+ SYSCTL_SUNLOCK();
if (error != EAGAIN)
break;
uio_yield();
@@ -1620,7 +1628,8 @@
if (req.lock == REQ_WIRED req.validlen 0)
vsunlock(req.oldptr, req.validlen);
- SYSCTL_XUNLOCK();
+ if (memlocked)
+ sx_xunlock(sysctlmemlock);
if (error error != ENOMEM)
return (error);
--
John Baldwin
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
On Fri, May 08, 2009 at 11:41:17PM +0200, Ed Schouten wrote: Hi, * vasanth raonaik vasanth.raon...@gmail.com wrote: Hello Jt, I am a newbee in this alias. I am having a very basic question. It would be really good if you could give me some of this information. Could you please elaborate on what is the current architecture of sysctl implementation and How the concurrency would benefit us. Right now sysctl is synchronized using the sysctl lock. The problem is that certain sysctls just block for a very long time (especially some of the GEOM ones). We also call sysctl when we execute new processes, to obtain a random number for the stack protector. This means we have quite a lot of contention on it. This lock needs to be there, because sysctls can be added and removed on demand. Why is sysctl used to get a random number? Can't we get a different source for it? pgpy3bE29OnYg.pgp Description: PGP signature
Re: concurrent sysctl implementation
Hi Lars, * Lars Engels lars.eng...@0x20.net wrote: Why is sysctl used to get a random number? Can't we get a different source for it? We probably could. I think I discussed this with Robert Watson some time ago and we could use things like ELF hints. But still, that doesn't prevent us from reaching this limitation later on. I think other people (Rink Springer) also reported this issue years ago, even before we had the stack protector enabled. -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ pgpVkyVsCo8A2.pgp Description: PGP signature
Re: concurrent sysctl implementation
Hi, * vasanth raonaik vasanth.raon...@gmail.com wrote: Hello Jt, I am a newbee in this alias. I am having a very basic question. It would be really good if you could give me some of this information. Could you please elaborate on what is the current architecture of sysctl implementation and How the concurrency would benefit us. Right now sysctl is synchronized using the sysctl lock. The problem is that certain sysctls just block for a very long time (especially some of the GEOM ones). We also call sysctl when we execute new processes, to obtain a random number for the stack protector. This means we have quite a lot of contention on it. This lock needs to be there, because sysctls can be added and removed on demand. I think I discussed this with John Baldwin (right?) and I think we also have the issue that we cannot allow an unbounded amount of concurrent calls to sysctl, because sysctl wires memory. A solution would be to solve it as follows: - Use a semaphore, initialized to some insane high value to put an upper limit on the amount of concurrent sysctl calls. I'm not sure whether this is really needed. Maybe this issue is not as serious as we think it is. - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up the lock when we traverse parts of the sysctl tree that has dynamically created entries. -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ pgpVFgnEuH1Cq.pgp Description: PGP signature
Re: concurrent sysctl implementation
Ed, Thanks :) I'll be implementing this as discussed over the next few months thanks for the technical detail I've been extremely busy with finals. I will write the list with my thoughts within the next week, sorry for the delay. =jt On May 8, 2009, at 17:41, Ed Schouten e...@80386.nl wrote: Hi, * vasanth raonaik vasanth.raon...@gmail.com wrote: Hello Jt, I am a newbee in this alias. I am having a very basic question. It would be really good if you could give me some of this information. Could you please elaborate on what is the current architecture of sysctl implementation and How the concurrency would benefit us. Right now sysctl is synchronized using the sysctl lock. The problem is that certain sysctls just block for a very long time (especially some of the GEOM ones). We also call sysctl when we execute new processes, to obtain a random number for the stack protector. This means we have quite a lot of contention on it. This lock needs to be there, because sysctls can be added and removed on demand. I think I discussed this with John Baldwin (right?) and I think we also have the issue that we cannot allow an unbounded amount of concurrent calls to sysctl, because sysctl wires memory. A solution would be to solve it as follows: - Use a semaphore, initialized to some insane high value to put an upper limit on the amount of concurrent sysctl calls. I'm not sure whether this is really needed. Maybe this issue is not as serious as we think it is. - Use an rw/rm/sxlock to protect the sysctl tree, but only pick up the lock when we traverse parts of the sysctl tree that has dynamically created entries. -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
concurrent sysctl implementation
Hackers, I've been using FreeBSD since a boy and now its time for me to give back. I will be doing my final projects in university concerning concurrency in the FreeBSD Kernel. I've been discussing with Ed@ methods of implementing sysctl concurrently since we use sysctl quite a lot for _everything_ essentially (make, general kernel information, etc). I've been reading the sysctl man pages and some of the code in the kernel; however, I wanted to shoot this out to the public since many of you know better than I do about where I should be looking to do the required reading to get the job done correctly. I'm also open to implementing other things once this is done. I hope everyone is doing well. respectfully, =jt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: concurrent sysctl implementation
Hello Jt, I am a newbee in this alias. I am having a very basic question. It would be really good if you could give me some of this information. Could you please elaborate on what is the current architecture of sysctl implementation and How the concurrency would benefit us. Thanks in advance, Vasanth On Tue, May 5, 2009 at 1:37 PM, j...@0xabadba.be wrote: Hackers, I've been using FreeBSD since a boy and now its time for me to give back. I will be doing my final projects in university concerning concurrency in the FreeBSD Kernel. I've been discussing with Ed@ methods of implementing sysctl concurrently since we use sysctl quite a lot for _everything_ essentially (make, general kernel information, etc). I've been reading the sysctl man pages and some of the code in the kernel; however, I wanted to shoot this out to the public since many of you know better than I do about where I should be looking to do the required reading to get the job done correctly. I'm also open to implementing other things once this is done. I hope everyone is doing well. respectfully, =jt ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
