Re: [head tinderbox] failure on i386/i386
on 12/12/2011 07:58 FreeBSD Tinderbox said the following: In file included from /src/sys/kern/kern_racct.c:53: /src/sys/sys/sx.h: In function '__sx_xlock': /src/sys/sys/sx.h:154: warning: implicit declaration of function 'SCHEDULER_STOPPED' /src/sys/sys/sx.h:154: warning: nested extern declaration of 'SCHEDULER_STOPPED' [-Wnested-externs] *** Error code 1 This should be fixed in r228430. Apologies. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
[head tinderbox] failure on i386/i386
TB --- 2011-12-12 09:10:00 - tinderbox 2.8 running on freebsd-current.sentex.ca TB --- 2011-12-12 09:10:00 - starting HEAD tinderbox run for i386/i386 TB --- 2011-12-12 09:10:00 - cleaning the object tree TB --- 2011-12-12 09:10:23 - cvsupping the source tree TB --- 2011-12-12 09:10:23 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/HEAD/i386/i386/supfile TB --- 2011-12-12 09:10:39 - building world TB --- 2011-12-12 09:10:39 - CROSS_BUILD_TESTING=YES TB --- 2011-12-12 09:10:39 - MAKEOBJDIRPREFIX=/obj TB --- 2011-12-12 09:10:39 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2011-12-12 09:10:39 - SRCCONF=/dev/null TB --- 2011-12-12 09:10:39 - TARGET=i386 TB --- 2011-12-12 09:10:39 - TARGET_ARCH=i386 TB --- 2011-12-12 09:10:39 - TZ=UTC TB --- 2011-12-12 09:10:39 - __MAKE_CONF=/dev/null TB --- 2011-12-12 09:10:39 - cd /src TB --- 2011-12-12 09:10:39 - /usr/bin/make -B buildworld World build started on Mon Dec 12 09:10:40 UTC 2011 Rebuilding the temporary build tree stage 1.1: legacy release compatibility shims stage 1.2: bootstrap tools stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3: cross tools stage 4.1: building includes stage 4.2: building libraries stage 4.3: make dependencies stage 4.4: building everything World build completed on Mon Dec 12 11:17:28 UTC 2011 TB --- 2011-12-12 11:17:28 - generating LINT kernel config TB --- 2011-12-12 11:17:28 - cd /src/sys/i386/conf TB --- 2011-12-12 11:17:28 - /usr/bin/make -B LINT TB --- 2011-12-12 11:17:28 - cd /src/sys/i386/conf TB --- 2011-12-12 11:17:28 - /usr/sbin/config -m LINT-NOINET TB --- 2011-12-12 11:17:28 - building LINT-NOINET kernel TB --- 2011-12-12 11:17:28 - CROSS_BUILD_TESTING=YES TB --- 2011-12-12 11:17:28 - MAKEOBJDIRPREFIX=/obj TB --- 2011-12-12 11:17:28 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2011-12-12 11:17:28 - SRCCONF=/dev/null TB --- 2011-12-12 11:17:28 - TARGET=i386 TB --- 2011-12-12 11:17:28 - TARGET_ARCH=i386 TB --- 2011-12-12 11:17:28 - TZ=UTC TB --- 2011-12-12 11:17:28 - __MAKE_CONF=/dev/null TB --- 2011-12-12 11:17:28 - cd /src TB --- 2011-12-12 11:17:28 - /usr/bin/make -B buildkernel KERNCONF=LINT-NOINET Kernel build for LINT-NOINET started on Mon Dec 12 11:17:28 UTC 2011 stage 1: configuring the kernel stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3.1: making dependencies stage 3.2: building everything [...] cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -DGPROF -falign-functions=16 -DGPROF4 -DGUPROF -fno-builtin -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-sse -mno-mmx -msoft-float -ffreestanding -fstack-protector -Werror -pg -mprofiler-epilogue /src/sys/kern/kern_sig.c cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -DGPROF -falign-functions=16 -DGPROF4 -DGUPROF -fno-builtin -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-sse -mno-mmx -msoft-float -ffreestanding -fstack-protector -Werror -pg -mprofiler-epilogue /src/sys/kern/kern_switch.c cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -DGPROF -falign-functions=16 -DGPROF4 -DGUPROF -fno-builtin -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-sse -mno-mmx -msoft-float -ffreestanding -fstack-protector -Werror -pg -mprofiler-epilogue /src/sys/kern/kern_sx.c cc1: warnings being treated as errors In file included from /src/sys/kern/kern_sx.c:52: /src/sys/sys/sx.h: In function '__sx_xlock': /src/sys/sys/sx.h:154: warning: implicit declaration of function 'SCHEDULER_STOPPED' /src/sys/sys/sx.h:154: warning: nested extern declaration of 'SCHEDULER_STOPPED' [-Wnested-externs] *** Error code 1 Stop in
[head tinderbox] failure on amd64/amd64
TB --- 2011-12-12 09:10:00 - tinderbox 2.8 running on freebsd-current.sentex.ca TB --- 2011-12-12 09:10:00 - starting HEAD tinderbox run for amd64/amd64 TB --- 2011-12-12 09:10:00 - cleaning the object tree TB --- 2011-12-12 09:10:30 - cvsupping the source tree TB --- 2011-12-12 09:10:30 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/HEAD/amd64/amd64/supfile TB --- 2011-12-12 09:16:04 - building world TB --- 2011-12-12 09:16:04 - CROSS_BUILD_TESTING=YES TB --- 2011-12-12 09:16:04 - MAKEOBJDIRPREFIX=/obj TB --- 2011-12-12 09:16:04 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2011-12-12 09:16:04 - SRCCONF=/dev/null TB --- 2011-12-12 09:16:04 - TARGET=amd64 TB --- 2011-12-12 09:16:04 - TARGET_ARCH=amd64 TB --- 2011-12-12 09:16:04 - TZ=UTC TB --- 2011-12-12 09:16:04 - __MAKE_CONF=/dev/null TB --- 2011-12-12 09:16:04 - cd /src TB --- 2011-12-12 09:16:04 - /usr/bin/make -B buildworld World build started on Mon Dec 12 09:16:04 UTC 2011 Rebuilding the temporary build tree stage 1.1: legacy release compatibility shims stage 1.2: bootstrap tools stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3: cross tools stage 4.1: building includes stage 4.2: building libraries stage 4.3: make dependencies stage 4.4: building everything stage 5.1: building 32 bit shim libraries World build completed on Mon Dec 12 11:53:40 UTC 2011 TB --- 2011-12-12 11:53:41 - generating LINT kernel config TB --- 2011-12-12 11:53:41 - cd /src/sys/amd64/conf TB --- 2011-12-12 11:53:41 - /usr/bin/make -B LINT TB --- 2011-12-12 11:53:41 - cd /src/sys/amd64/conf TB --- 2011-12-12 11:53:41 - /usr/sbin/config -m LINT-NOINET TB --- 2011-12-12 11:53:41 - building LINT-NOINET kernel TB --- 2011-12-12 11:53:41 - CROSS_BUILD_TESTING=YES TB --- 2011-12-12 11:53:41 - MAKEOBJDIRPREFIX=/obj TB --- 2011-12-12 11:53:41 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2011-12-12 11:53:41 - SRCCONF=/dev/null TB --- 2011-12-12 11:53:41 - TARGET=amd64 TB --- 2011-12-12 11:53:41 - TARGET_ARCH=amd64 TB --- 2011-12-12 11:53:41 - TZ=UTC TB --- 2011-12-12 11:53:41 - __MAKE_CONF=/dev/null TB --- 2011-12-12 11:53:41 - cd /src TB --- 2011-12-12 11:53:41 - /usr/bin/make -B buildkernel KERNCONF=LINT-NOINET Kernel build for LINT-NOINET started on Mon Dec 12 11:53:41 UTC 2011 stage 1: configuring the kernel stage 2.1: cleaning up the object tree stage 2.2: rebuilding the object tree stage 2.3: build tools stage 3.1: making dependencies stage 3.2: building everything [...] cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -DGPROF -falign-functions=16 -DGPROF4 -DGUPROF -fno-builtin -fno-omit-frame-pointer -mno-sse -mcmodel=kernel -mno-red-zone -mno-mmx -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror -pg -mprofiler-epilogue /src/sys/kern/kern_sig.c cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -DGPROF -falign-functions=16 -DGPROF4 -DGUPROF -fno-builtin -fno-omit-frame-pointer -mno-sse -mcmodel=kernel -mno-red-zone -mno-mmx -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror -pg -mprofiler-epilogue /src/sys/kern/kern_switch.c cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/src/sys -I/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -DGPROF -falign-functions=16 -DGPROF4 -DGUPROF -fno-builtin -fno-omit-frame-pointer -mno-sse -mcmodel=kernel -mno-red-zone -mno-mmx -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror -pg -mprofiler-epilogue /src/sys/kern/kern_sx.c cc1: warnings being treated as errors In file included from /src/sys/kern/kern_sx.c:52: /src/sys/sys/sx.h: In function '__sx_xlock':
Re: SCHED_ULE should not be the default
Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. O. signature.asc Description: OpenPGP digital signature
Re: SCHED_ULE should not be the default
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/12/2011 13:47, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. It all a little old now but some if the stuff in http://people.freebsd.org/~kris/scaling/ covers improvements that were seen. http://jeffr-tech.livejournal.com/5705.html shows a little too, reading though Jeffs blog is worth it as it has some interesting stuff on SHED_ULE. I thought there were some more benchmarks floating round but cant find any with a quick google. Vince Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. O. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJO5hn7AAoJEF4mgOY1fXowOLAP/2EjhAFPb88NgKM0ieBb4X7R NSw/9HTiwcshkfEdvYjAzYZ0cUWetEuRfnPVnh+abwfJEmMzZkwA0KIz8UYGHHik 22Z2SWSVDiwZAluz0ca7Xc931ojbzrK/zVMbivqW3cvnz8P4oEnASiENnsoa89Jy Oskjd4QpAyIpB/AsYgc9FLT3kPX13fXC5bzw/zAPDsaupOYssRRlZu8nnqsEc1i1 IanLIPKLnIbpZTx75ehWxxRW8IjiQRvIe+7eBaDMhXO/Kvftotf0JzknrBnJezDQ ZdhiOTq7F1Pm3dxra+DNKD+Dw+xUCYPFq/kuyqrZNz44H3qwT60vDhvw0yDz6422 nNP11z2+G4M85sahBak5AmSHuyek7HWb6uIHHnfvwNKSX4ZsdS8MVBViNJjmCYtL PwuHDU3WdCes/vvKRNDopSp/s6RSLK9w3RT7jlMkaTu2Mmtw0BwGziDJ2pGaCQ14 68R5eO/SfNxoVp0g4lIzObyQR+//0OmALzElVK3VmHM9NoL3qZGCwBRLqjN5re82 dX6nsBr/DFJOpaFfdFLwPNyCNdNpg/WVegRkq2BEL/BaMISNiKzoVbM0Psh9gnb3 LW1j3LP2fOHhuN1bW3S31JmbNzvAnlRNynoNMldrwj5PWJY2HPk+mMFRjmRwdDTJ 9mhscz8++WRPvDZQXefl =XqaR -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, 12 Dec 2011 15:13:00 + Vincent Hoffman vi...@unsane.co.uk wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/12/2011 13:47, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. It all a little old now but some if the stuff in http://people.freebsd.org/~kris/scaling/ covers improvements that were seen. http://jeffr-tech.livejournal.com/5705.html shows a little too, reading though Jeffs blog is worth it as it has some interesting stuff on SHED_ULE. I thought there were some more benchmarks floating round but cant find any with a quick google. Vince Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. These observations are not scientific, but I have a CPU from AMD with 6 cores (AMD Phenom(tm) II X6 1090T Processor). My simple test was ``make buildkernel'' while watching the core usage with gkrellm. With SCHED_4BSD all 6 cores are loaded to 97% during the build phase. I've never seen any value above 97% with gkrellm. With SCHED_ULE I never saw all 6 cores loaded this heavily. Usually 2 or more cores were at or below 90%. Not really that significant, but still a noticeable difference in apparent scheduling behavior. Whether the observed difference is due to some change in data from the kernel to gkrellm is beyond me. -- Gary Jennejohn ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, Dec 12, 2011 at 02:47:57PM +0100, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. This comes up every 9 months or so, and must be approaching FAQ status. In a HPC environment, I recommend 4BSD. Depending on the workload, ULE can cause a severe increase in turn around time when doing already long computations. If you have an MPI application, simply launching greater than ncpu+1 jobs can show the problem. PS: search the list archives for kargl and ULE. -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: 9.0-RC1 panic in tcp_input: negative winow.
On Monday, October 24, 2011 8:14:22 am John Baldwin wrote: On Sunday, October 23, 2011 11:58:28 am Pawel Jakub Dawidek wrote: On Sun, Oct 23, 2011 at 11:44:45AM +0300, Kostik Belousov wrote: On Sun, Oct 23, 2011 at 08:10:38AM +0200, Pawel Jakub Dawidek wrote: My suggestion would be that if we won't be able to fix it before 9.0, we should turn this assertion off, as the system seems to be able to recover. Shipped kernels have all assertions turned off. Yes, I'm aware of that, but many people compile their production kernels with INVARIANTS/INVARIANT_SUPPORT to fail early instead of eg. corrupting data. I'd be fine in moving this under DIAGNOSTIC or changing it into a printf, so it will be visible. No, the kernel is corrupting things in other places when this is true, so if you are running with INVARIANTS, we want to know about it. Specifically, in several places in TCP we assume that rcv_adv = rcv_nxt, and depend on being able to do 'rcv_adv - rcv_nxt'. In this case, it looks like the difference is consistently less than one frame. I suspect the other end of the connection is sending just beyond the end of the advertised window (it probably assumes it is better to send a full frame if it has that much pending data even though part of it is beyond the window edge vs sending a truncated packet that just fills the window) and that that frame is accepted ok in the header prediction case and it's ACK is delayed, but the next packet to arrive then trips over this assumption. Since 'win' is guaranteed to be non-negative and we explicitly cast 'rcv_adv - rcv_nxt' to (int) in the following line that the assert is checking for: tp-rcv_wnd = imax(win, (int)(tp-rcv_adv - tp-rcv_nxt)); I think we already handle this case ok and perhaps the assertion can just be removed? Not sure if others feel that it warrants a comment to note that this is the case being handled. Also, I'm not sure if this case can leak into the timewait code? If so, we will need to fix this case: /* * Recover last window size sent. */ KASSERT(SEQ_GEQ(tp-rcv_adv, tp-rcv_nxt), (tcp_twstart negative window: tp %p rcv_nxt %u rcv_adv %u, tp, tp-rcv_nxt, tp-rcv_adv)); tw-last_win = (tp-rcv_adv - tp-rcv_nxt) tp-rcv_scale; So that it sets last_win to 0 instead of some really huge value. An update. I've sent Pawel a testing patch to see if my hypothesis is correct (www.freebsd.org/~jhb/patches/tcp_negwin_test.patch). If it is then I intend to commit www.freebsd.org/~jhb/patches/tcp_negwin2.patch as the fix. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, Dec 12, 2011 at 7:32 AM, Gary Jennejohn gljennj...@googlemail.com wrote: On Mon, 12 Dec 2011 15:13:00 + Vincent Hoffman vi...@unsane.co.uk wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/12/2011 13:47, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. It all a little old now but some if the stuff in http://people.freebsd.org/~kris/scaling/ covers improvements that were seen. http://jeffr-tech.livejournal.com/5705.html shows a little too, reading though Jeffs blog is worth it as it has some interesting stuff on SHED_ULE. I thought there were some more benchmarks floating round but cant find any with a quick google. Vince Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. These observations are not scientific, but I have a CPU from AMD with 6 cores (AMD Phenom(tm) II X6 1090T Processor). My simple test was ``make buildkernel'' while watching the core usage with gkrellm. With SCHED_4BSD all 6 cores are loaded to 97% during the build phase. I've never seen any value above 97% with gkrellm. With SCHED_ULE I never saw all 6 cores loaded this heavily. Usually 2 or more cores were at or below 90%. Not really that significant, but still a noticeable difference in apparent scheduling behavior. Whether the observed difference is due to some change in data from the kernel to gkrellm is beyond me. SCHED_ULE is much sloppier about calculating which thread used a timeslice -- unless the timeslice went 100% to a thread, the fraction it used may get attributed elsewhere. So top's reporting of thread usage is not a useful metric. Total buildworld time is, potentially. Thanks, matthew ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
Did you use -jX to build the world? _ Von: Gary Jennejohn gljennj...@googlemail.com Versendet am: Mon Dec 12 16:32:21 MEZ 2011 An: Vincent Hoffman vi...@unsane.co.uk CC: O. Hartmann ohart...@mail.zedat.fu-berlin.de, Current FreeBSD freebsd-current@freebsd.org, freebsd-sta...@freebsd.org, freebsd-performa...@freebsd.org Betreff: Re: SCHED_ULE should not be the default On Mon, 12 Dec 2011 15:13:00 + Vincent Hoffman vi...@unsane.co.uk wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/12/2011 13:47, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. It all a little old now but some if the stuff in http://people.freebsd.org/~kris/scaling/ covers improvements that were seen. http://jeffr-tech.livejournal.com/5705.html shows a little too, reading though Jeffs blog is worth it as it has some interesting stuff on SHED_ULE. I thought there were some more benchmarks floating round but cant find any with a quick google. Vince Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. These observations are not scientific, but I have a CPU from AMD with 6 cores (AMD Phenom(tm) II X6 1090T Processor). My simple test was ``make buildkernel'' while watching the core usage with gkrellm. With SCHED_4BSD all 6 cores are loaded to 97% during the build phase. I've never seen any value above 97% with gkrellm. With SCHED_ULE I never saw all 6 cores loaded this heavily. Usually 2 or more cores were at or below 90%. Not really that significant, but still a noticeable difference in apparent scheduling behavior. Whether the observed difference is due to some change in data from the kernel to gkrellm is beyond me. -- Gary Jennejohn _ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
Would it be possible to implement a mechanism that lets one change the scheduler on the fly? Afaik Solaris can do that. _ Von: Steve Kargl s...@troutmask.apl.washington.edu Versendet am: Mon Dec 12 16:51:59 MEZ 2011 An: O. Hartmann ohart...@mail.zedat.fu-berlin.de CC: freebsd-performa...@freebsd.org, Current FreeBSD freebsd-current@freebsd.org, freebsd-sta...@freebsd.org Betreff: Re: SCHED_ULE should not be the default On Mon, Dec 12, 2011 at 02:47:57PM +0100, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. This comes up every 9 months or so, and must be approaching FAQ status. In a HPC environment, I recommend 4BSD. Depending on the workload, ULE can cause a severe increase in turn around time when doing already long computations. If you have an MPI application, simply launching greater than ncpu+1 jobs can show the problem. PS: search the list archives for kargl and ULE. -- Steve _ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On 12/12/2011 15:51, Steve Kargl wrote: This comes up every 9 months or so, and must be approaching FAQ status. In a HPC environment, I recommend 4BSD. Depending on the workload, ULE can cause a severe increase in turn around time when doing already long computations. If you have an MPI application, simply launching greater than ncpu+1 jobs can show the problem. PS: search the list archives for kargl and ULE. This isn't something that can be fixed by tuning ULE? For example for desktop applications kern.sched.preempt_thresh should be set to 224 from its default. I'm wondering if the installer should ask people what the typical use will be, and tune the scheduler appropriately. -- Bruce Cran ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
В Mon, 12 Dec 2011 16:18:35 + Bruce Cran br...@cran.org.uk пишет: On 12/12/2011 15:51, Steve Kargl wrote: This comes up every 9 months or so, and must be approaching FAQ status. In a HPC environment, I recommend 4BSD. Depending on the workload, ULE can cause a severe increase in turn around time when doing already long computations. If you have an MPI application, simply launching greater than ncpu+1 jobs can show the problem. PS: search the list archives for kargl and ULE. This isn't something that can be fixed by tuning ULE? For example for desktop applications kern.sched.preempt_thresh should be set to 224 from its default. I'm wondering if the installer should ask people what the typical use will be, and tune the scheduler appropriately. This is by and large does not help in certain situations ... ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Monday 12 December 2011 14:47:57 O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. O. In my spare time I do some stuff which can be considered HPC. If I recall correctly the most loud supporters of the notion that SCHED_BSD is faster than SCHED_ULE are using more threads than there are cores, causing CPU core contention and more importantly unevenly distributed runtimes among threads, resulting in suboptimal execution times for their programs. Since I've never actually seen that code in question it's hard to say whether or not this unfair distribution actually results in lower throughput or that it simply violates an assumption in the code that each thread takes about as long to finish its task. Although I haven't actually benchmarked the two schedulers directly, I have no reason to suspect SCHED_ULE of suboptimal performance because: 1) A program model where there are N threads on N cores which take work items from a shared queue until it is empty has almost perfect scaling on SCHED_ULE (I get 398% CPU usage on a quadcore) 2) The same program on Linux (dual boot) compiled with exactly the same compiler and flags runs slightly slower. I think this has to do with VM differences. What I'm trying to say is that until someone actually shows some code which has demonstrably lower performance on SCHED_ULE and this is not caused by IMHO improper timing dependencies between threads I'd say that there is no cause for concern here. I actually expect performance differences between the two schedulers to show in problems which cause a lot more contention on the CPU cores and use lots of locks internally so threads are frequently waiting on each other, for instance the MySQL benchmarks done a couple of years ago by Kris Kennaway. Aside from algorithmic limitations (SCHED_BSD doesn't really scale all that well), there will always exist some problems in which SCHED_BSD is faster because it by chance has a better execution order for these problems... The good thing is people have a choice :-). I'm looking forward to the results of your benchmark. -- Pieter de Goeje ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, 12 Dec 2011 17:10:46 +0100 Lars Engels lars.eng...@0x20.net wrote: Did you use -jX to build the world? I'm top posting since Lars did. It was buildkernel, not buildworld. Yes, -j6. _ Von: Gary Jennejohn gljennj...@googlemail.com Versendet am: Mon Dec 12 16:32:21 MEZ 2011 An: Vincent Hoffman vi...@unsane.co.uk CC: O. Hartmann ohart...@mail.zedat.fu-berlin.de, Current FreeBSD freebsd-current@freebsd.org, freebsd-sta...@freebsd.org, freebsd-performa...@freebsd.org Betreff: Re: SCHED_ULE should not be the default On Mon, 12 Dec 2011 15:13:00 + Vincent Hoffman vi...@unsane.co.uk wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/12/2011 13:47, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. It all a little old now but some if the stuff in http://people.freebsd.org/~kris/scaling/ covers improvements that were seen. http://jeffr-tech.livejournal.com/5705.html shows a little too, reading though Jeffs blog is worth it as it has some interesting stuff on SHED_ULE. I thought there were some more benchmarks floating round but cant find any with a quick google. Vince Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. These observations are not scientific, but I have a CPU from AMD with 6 cores (AMD Phenom(tm) II X6 1090T Processor). My simple test was ``make buildkernel'' while watching the core usage with gkrellm. With SCHED_4BSD all 6 cores are loaded to 97% during the build phase. I've never seen any value above 97% with gkrellm. With SCHED_ULE I never saw all 6 cores loaded this heavily. Usually 2 or more cores were at or below 90%. Not really that significant, but still a noticeable difference in apparent scheduling behavior. Whether the observed difference is due to some change in data from the kernel to gkrellm is beyond me. -- Gary Jennejohn _ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Gary Jennejohn ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, 12 Dec 2011 08:04:37 -0800 m...@freebsd.org wrote: On Mon, Dec 12, 2011 at 7:32 AM, Gary Jennejohn gljennj...@googlemail.com wrote: On Mon, 12 Dec 2011 15:13:00 + Vincent Hoffman vi...@unsane.co.uk wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/12/2011 13:47, O. Hartmann wrote: Not fully right, boinc defaults to run on idprio 31 so this isn't an issue. And yes, there are cases where SCHED_ULE shows much better performance then SCHED_4BSD. [...] Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? Whenever the subject comes up, it is mentioned, that SCHED_ULE has better performance on boxes with a ncpu 2. But in the end I see here contradictionary statements. People complain about poor performance (especially in scientific environments), and other give contra not being the case. It all a little old now but some if the stuff in http://people.freebsd.org/~kris/scaling/ covers improvements that were seen. http://jeffr-tech.livejournal.com/5705.html shows a little too, reading though Jeffs blog is worth it as it has some interesting stuff on SHED_ULE. I thought there were some more benchmarks floating round but cant find any with a quick google. Vince Within our department, we developed a highly scalable code for planetary science purposes on imagery. It utilizes present GPUs via OpenCL if present. Otherwise it grabs as many cores as it can. By the end of this year I'll get a new desktop box based on Intels new Sandy Bridge-E architecture with plenty of memory. If the colleague who developed the code is willing performing some benchmarks on the same hardware platform, we'll benchmark bot FreeBSD 9.0/10.0 and the most recent Suse. For FreeBSD I intent also to look for performance with both different schedulers available. These observations are not scientific, but I have a CPU from AMD with 6 cores (AMD Phenom(tm) II X6 1090T Processor). My simple test was ``make buildkernel'' while watching the core usage with gkrellm. With SCHED_4BSD all 6 cores are loaded to 97% during the build phase. I've never seen any value above 97% with gkrellm. With SCHED_ULE I never saw all 6 cores loaded this heavily. Usually 2 or more cores were at or below 90%. Not really that significant, but still a noticeable difference in apparent scheduling behavior. Whether the observed difference is due to some change in data from the kernel to gkrellm is beyond me. SCHED_ULE is much sloppier about calculating which thread used a timeslice -- unless the timeslice went 100% to a thread, the fraction it used may get attributed elsewhere. So top's reporting of thread usage is not a useful metric. Total buildworld time is, potentially. I suspect you're right since the buildworld time, a much better test, was pretty much the same with 4BSD and ULE. -- Gary Jennejohn ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, Dec 12, 2011 at 04:18:35PM +, Bruce Cran wrote: On 12/12/2011 15:51, Steve Kargl wrote: This comes up every 9 months or so, and must be approaching FAQ status. In a HPC environment, I recommend 4BSD. Depending on the workload, ULE can cause a severe increase in turn around time when doing already long computations. If you have an MPI application, simply launching greater than ncpu+1 jobs can show the problem. PS: search the list archives for kargl and ULE. This isn't something that can be fixed by tuning ULE? For example for desktop applications kern.sched.preempt_thresh should be set to 224 from its default. I'm wondering if the installer should ask people what the typical use will be, and tune the scheduler appropriately. Tuning kern.sched.preempt_thresh did not seem to help for my workload. My code is a classic master-slave OpenMPI application where the master runs on one node and all cpu-bound slaves are sent to a second node. If I send send ncpu+1 jobs to the 2nd node with ncpu's, then ncpu-1 jobs are assigned to the 1st ncpu-1 cpus. The last two jobs are assigned to the ncpu'th cpu, and these ping-pong on the this cpu. AFAICT, it is a cpu affinity issue, where ULE is trying to keep each job associated with its initially assigned cpu. While one might suggest that starting ncpu+1 jobs is not prudent, my example is just that. It is an example showing that ULE has performance issues. So, I now can start only ncpu jobs on each node in the cluster and send emails to all other users to not use those node, or use 4BSD and not worry about loading issues. -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD/amd64 on machine without ACPI BIOS?
On Friday, December 09, 2011 5:10:18 am Lev Serebryakov wrote: Hello, Freebsd-current. Soekris (famous developer of small x86-compatible appliance-like hardware) released net6501 some time ago, which is based on Atom (E6xx) CPU. It seems, that 64-bit version of Linux could run on it without problems. But FreeBSD/amd64 can not. It stops after kernel detect some devices without any errors or panics. This box has one big difference from billions other Intel-based boxes on market: it has very special BIOS without ACPI at all. Someone says, that it could be reason why FreeBSD/amd64 could not be boot on this box. Is it true? Is it possible to have FreeBSD/amd64 without ACPI? Hmm, amd64 pretty much requires ACPI currently. It might mostly work if you add 'device mptable', but it won't be able to find PNPBIOS devices (though hints will mostly work for that stuff). If you don't have an mptable it won't be able to route PCI interrupts, etc. It also requires an APIC, though you might be able to use 'device atpic' to work around that. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: r227487 breaks C++ programs that use __isthreaded
On Thursday, December 01, 2011 4:23:11 pm David Schultz wrote: On Thu, Dec 01, 2011, George Liaskos wrote: Hello One example is Google's tcmalloc [1], is this behaviour intended? [1] http://code.google.com/p/google- perftools/source/browse/trunk/src/maybe_threads.cc This code uses an unportable workaround for a bug that I believe was fixed in r227999. Using internal names starting with a double underscore isn't supported. I still think 227999 is wrong and would prefer that once actually worked, but that has fallout for libstdc++. libc has an internal _once() that always works. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Monday, December 12, 2011 12:06:04 pm Steve Kargl wrote: On Mon, Dec 12, 2011 at 04:18:35PM +, Bruce Cran wrote: On 12/12/2011 15:51, Steve Kargl wrote: This comes up every 9 months or so, and must be approaching FAQ status. In a HPC environment, I recommend 4BSD. Depending on the workload, ULE can cause a severe increase in turn around time when doing already long computations. If you have an MPI application, simply launching greater than ncpu+1 jobs can show the problem. PS: search the list archives for kargl and ULE. This isn't something that can be fixed by tuning ULE? For example for desktop applications kern.sched.preempt_thresh should be set to 224 from its default. I'm wondering if the installer should ask people what the typical use will be, and tune the scheduler appropriately. Tuning kern.sched.preempt_thresh did not seem to help for my workload. My code is a classic master-slave OpenMPI application where the master runs on one node and all cpu-bound slaves are sent to a second node. If I send send ncpu+1 jobs to the 2nd node with ncpu's, then ncpu-1 jobs are assigned to the 1st ncpu-1 cpus. The last two jobs are assigned to the ncpu'th cpu, and these ping-pong on the this cpu. AFAICT, it is a cpu affinity issue, where ULE is trying to keep each job associated with its initially assigned cpu. While one might suggest that starting ncpu+1 jobs is not prudent, my example is just that. It is an example showing that ULE has performance issues. So, I now can start only ncpu jobs on each node in the cluster and send emails to all other users to not use those node, or use 4BSD and not worry about loading issues. This is a case where 4BSD's naive algorithm will spread out the load more evenly because all the threads are on a single, shared queue and each CPU just grabs the head of the queue when it finishes a timeslice. ULE always assigns threads to a single CPU (even if they aren't pinned to a single CPU using cpuset, etc.) and then tries to balance the load across cores later, but I believe in this case it's rebalancer won't have anything to really do as no matter what it does with the N+1 job it's going to be sharing a CPU with another job. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
amd64 packages
Hello! My question is quite short and stupid -- why there is still no packages for 10-current? Did I miss something? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, Dec 12, 2011 at 09:06:04AM -0800, Steve Kargl wrote: Tuning kern.sched.preempt_thresh did not seem to help for my workload. My code is a classic master-slave OpenMPI application where the master runs on one node and all cpu-bound slaves are sent to a second node. If I send send ncpu+1 jobs to the 2nd node with ncpu's, then ncpu-1 jobs are assigned to the 1st ncpu-1 cpus. The last two jobs are assigned to the ncpu'th cpu, and these ping-pong on the this cpu. AFAICT, it is a cpu affinity issue, where ULE is trying to keep each job associated with its initially assigned cpu. While one might suggest that starting ncpu+1 jobs is not prudent, my example is just that. It is an example showing that ULE has performance issues. So, I now can start only ncpu jobs on each node in the cluster and send emails to all other users to not use those node, or use 4BSD and not worry about loading issues. Does it meet your expectations if you start (j modulo ncpu) = 0 jobs on a node? -- Scott LambertKC5MLE Unix SysAdmin lamb...@lambertfam.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On Mon, Dec 12, 2011 at 01:03:30PM -0600, Scott Lambert wrote: On Mon, Dec 12, 2011 at 09:06:04AM -0800, Steve Kargl wrote: Tuning kern.sched.preempt_thresh did not seem to help for my workload. My code is a classic master-slave OpenMPI application where the master runs on one node and all cpu-bound slaves are sent to a second node. If I send send ncpu+1 jobs to the 2nd node with ncpu's, then ncpu-1 jobs are assigned to the 1st ncpu-1 cpus. The last two jobs are assigned to the ncpu'th cpu, and these ping-pong on the this cpu. AFAICT, it is a cpu affinity issue, where ULE is trying to keep each job associated with its initially assigned cpu. While one might suggest that starting ncpu+1 jobs is not prudent, my example is just that. It is an example showing that ULE has performance issues. So, I now can start only ncpu jobs on each node in the cluster and send emails to all other users to not use those node, or use 4BSD and not worry about loading issues. Does it meet your expectations if you start (j modulo ncpu) = 0 jobs on a node? I've never tried to launch more than ncpu + 1 (or + 2) jobs. I suppose at the time I was investigating the issue, it was determined that 4BSD allowed me to get my work done in a more timely manner. So, I took the path of least resistance. -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] winbond watchdog driver for FreeBSD/i386 and FreeBSD/amd64
On 12/7/2011 02:17, Bjoern A. Zeeb wrote: On 7. Dec 2011, at 09:29 , Pawel Jakub Dawidek wrote: On Tue, Jun 28, 2011 at 03:32:41PM -0700, Xin LI wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, I'd like to request for comments on the attached driver, which supports watchdogs on several Winbond super I/O chip models and have been tested on a few of recent Supermicro motherboards. Is there any reason this is not yet committed? Please commit, pretty please. Yes we have 2+ of them and are trying to merge. The other one sits here and allows you even longer timeouts as well as time setting the timeout independent of watchdogd in case you have two watchdogs and want to do tricks like NMI/reset with the other one... but is lacking the man page yet. I have added another one or two IC revisions I think that I had found and tested privately. http://people.freebsd.org/~bz/patch-20110710-03-wbwd.diff I've been using 20110718-02-wbwd.diff for a few months now on a project with PC Engines Alix 1.d boards (http://pcengines.ch/alix1d.htm). They have a Winbond W83627HG chip. I don't see any probing/attach messages on boot but the driver seems to be properly configuring the chip - if I kill watchdogd with -9 the board reboots with watchdog timeout. I'm also trying to use the above winbond chip for GPIO (userland bit banging at this point). /bz -Keith ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: amd64 packages
Hi! My question is quite short and stupid -- why there is still no packages for 10-current? Did I miss something? Because the ports people are busy with getting 9.0-REL out of the door. -- p...@opsec.eu+49 171 3101372 9 years to go ! ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] winbond watchdog driver for FreeBSD/i386 and FreeBSD/amd64
On 12/12/2011 2:49 PM, Keith Simonsen wrote: I've been using 20110718-02-wbwd.diff for a few months now on a project with PC Engines Alix 1.d boards (http://pcengines.ch/alix1d.htm). They have a Winbond W83627HG chip. I don't see any probing/attach messages on boot but the driver seems to be properly configuring the chip - if I kill watchdogd with -9 the board reboots with watchdog timeout. Are you sure thats the watchdog thats doing the 'killing' so to speak ? If you have option CPU_GEODE in your kernel config, you will get the watchdog code there no ? ( /usr/src/sys/i386/i386/geode.c) ---Mike -- --- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, m...@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On 12/12/11 18:06, Steve Kargl wrote: On Mon, Dec 12, 2011 at 04:18:35PM +, Bruce Cran wrote: On 12/12/2011 15:51, Steve Kargl wrote: This comes up every 9 months or so, and must be approaching FAQ status. In a HPC environment, I recommend 4BSD. Depending on the workload, ULE can cause a severe increase in turn around time when doing already long computations. If you have an MPI application, simply launching greater than ncpu+1 jobs can show the problem. PS: search the list archives for kargl and ULE. This isn't something that can be fixed by tuning ULE? For example for desktop applications kern.sched.preempt_thresh should be set to 224 from its default. I'm wondering if the installer should ask people what the typical use will be, and tune the scheduler appropriately. Is the tuning of kern.sched.preempt_thresh and a proper method of estimating its correct value for the intended to use workload documented in the manpages, maybe tuning()? I find it hard to crawl a lot of pros and cons of mailing lists for evaluating a correct value of this, seemingly, important tunable. Tuning kern.sched.preempt_thresh did not seem to help for my workload. My code is a classic master-slave OpenMPI application where the master runs on one node and all cpu-bound slaves are sent to a second node. If I send send ncpu+1 jobs to the 2nd node with ncpu's, then ncpu-1 jobs are assigned to the 1st ncpu-1 cpus. The last two jobs are assigned to the ncpu'th cpu, and these ping-pong on the this cpu. AFAICT, it is a cpu affinity issue, where ULE is trying to keep each job associated with its initially assigned cpu. While one might suggest that starting ncpu+1 jobs is not prudent, my example is just that. It is an example showing that ULE has performance issues. So, I now can start only ncpu jobs on each node in the cluster and send emails to all other users to not use those node, or use 4BSD and not worry about loading issues. signature.asc Description: OpenPGP digital signature
Re: amd64 packages
On 12/12/11 19:46, siur wrote: Hello! My question is quite short and stupid -- why there is still no packages for 10-current? Did I miss something? 10.0-CURRENT is at this very moment the bloody edge development and 9.0-REL isn't out yet. So do not expect personell dedicating themselfs on building packages. Building packages/software has still some issues on 10.0-CUR, and I guess there will no packeage building until those issues get fixed. The beste way is to build the software via ports yourself - one of the really great benefits of having FreeBSD and its ports collection. Regards, Oliver signature.asc Description: OpenPGP digital signature
Re: [RFC] winbond watchdog driver for FreeBSD/i386 and FreeBSD/amd64
On 12/12/2011 12:25, Mike Tancsa wrote: On 12/12/2011 2:49 PM, Keith Simonsen wrote: I've been using 20110718-02-wbwd.diff for a few months now on a project with PC Engines Alix 1.d boards (http://pcengines.ch/alix1d.htm). They have a Winbond W83627HG chip. I don't see any probing/attach messages on boot but the driver seems to be properly configuring the chip - if I kill watchdogd with -9 the board reboots with watchdog timeout. Are you sure thats the watchdog thats doing the 'killing' so to speak ? If you have option CPU_GEODE in your kernel config, you will get the watchdog code there no ? ( /usr/src/sys/i386/i386/geode.c) Yes I do have CPU_GEODE in my kernel and I see the geode MFGPT probed in the verbose dmesg output. I'm not sure how I can tell what piece of hardware /dev/fido is linked to but I think you're correct and I'm using the geode watchdog and not the winbond chip. Maybe this has something do with me not having 'device eisa' in my kernel config! I'm going to start compiling a new nanobsd image right now with eisa and the newer wbwd.c driver and see how it goes Thanks ---Mike -Keith ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On 12/12/2011 23:48, O. Hartmann wrote: Is the tuning of kern.sched.preempt_thresh and a proper method of estimating its correct value for the intended to use workload documented in the manpages, maybe tuning()? I find it hard to crawl a lot of pros and cons of mailing lists for evaluating a correct value of this, seemingly, important tunable. Note that I said for example :) I was suggesting that there may be sysctl's that can be tweaked to improve performance. -- Bruce Cran ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: SCHED_ULE should not be the default
On 12/12/2011 05:47, O. Hartmann wrote: Do we have any proof at hand for such cases where SCHED_ULE performs much better than SCHED_4BSD? I complained about poor interactive performance of ULE in a desktop environment for years. I had numerous people try to help, including Jeff, with various tunables, dtrace'ing, etc. The cause of the problem was never found. I switched to 4BSD, problem gone. This is on 2 separate systems with core 2 duos. hth, Doug -- [^L] Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: amd64 packages
On Mon, Dec 12, 2011 at 10:46:08PM +0400, siur wrote: why there is still no packages for 10-current? Did I miss something? We're trying to debug multiple package building problems, and currently using i386-10 for that. Until we get farther along with that process, we aren't doing amd64-10 yet. mcl ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
multihomed nfs server - NLM lock failure on additional interfaces
Hi Folks, I have a 9-prerelease system where I've been testing nfs/zfs. The system has been working quite well until moving the server to a multihomed configuration. Given the following: nfsd: master (nfsd) nfsd: server (nfsd) /usr/sbin/rpcbind -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 /usr/sbin/mountd -r -l -h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 /usr/sbin/rpc.statd-h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 /usr/sbin/rpc.lockd-h 10.24.6.38 -h 172.1.1.2 -h 172.21.201.1 -h 172.21.202.1 -h 172.21.203.1 -h 172.21.204.1 -h 172.21.205.1 -h 10.24.6.34 -h 10.24.6.33 10.24.6.38 is the default interface on 1G. The 172 nets are 10G connected to compute systems. ifconfig_bce0=' inet 10.24.6.38netmask 255.255.0.0 -rxcsum -txcsum' _c='physical addr which never changes' ifconfig_bce1=' inet 172.1.1.2 netmask 255.255.255.0' _c='physcial addr on crossover cable' ifconfig_cxgb2='inet 172.21.21.129 netmask 255.255.255.0' _c='physical backside 10g compute net' ifconfig_cxgb3='inet 172.21.201.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb6='inet 172.21.202.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb8='inet 172.21.203.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb4='inet 172.21.204.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' ifconfig_cxgb0='inet 172.21.205.1 netmask 255.255.255.0 mtu 9000' _c='physical backside 10g compute net' The 10.24.6.34 and 10.24.6.33 are alias addresses for the system. DestinationGatewayFlagsRefs Use Netif Expire default10.24.0.1 UGS 0 1049 bce0 The server works correctly (and quite well) for both udp tcp mounts. Basically, all nfs traffic is great! However, locking only works for clients connected to the 10.24.6.38 interface. A tcpdump file from good bad runs: http://www.freebsd.org/~jwd/lockgood.pcap http://www.freebsd.org/~jwd/lockbad.pcap Basically, the clients (both FreeBSD Linux) query the servers rpcbind for the address of the nlm which is returned correctly. For the good run, the NLM is then called. For the bad call, it is not. I've started digging through code, but I do not claim to be an rpc expert. If anyone has suggestions I would appreciate any pointers. Thanks! John ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: NFS + SVN problem?
On 2011-11-23 19:26, Sean Bruno wrote: On Wed, 2011-11-23 at 09:58 -0800, Rick Macklem wrote: I don't know if Dimitry tried this, but you could also try the nolockd option, so that byte range locking is done locally in the client and avoids the NLM. Good luck with it and please let us know how it goes, rick This seems to allow SVN 1.7 to do whatever nonsense it is trying to do. I've modified my fstab on the test host in the cluster to: dumpster:/vol/volshscratch /dumpster/scratch nfs rw,soft,intr,bg,nolockd,nosuid 0 0 Removing soft,intr had no effect. This, I suspect will be problematic for clusteradm@ if we start updating hosts in the cluster. A very late addition to this: I got Subversion 1.7 to work properly over NFSv3, by making sure rpc.lockd runs on both server and client. E.g, set rpc_lockd_enable to YES in rc.conf; this is off by default, even if you have nfs_client_enable/nfs_server_enable set to YES. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
