Re: Bumping MAXCPU on amd64?
On Wed, 22 Sep 2010, Maxim Sobolev wrote: On 9/22/2010 6:37 AM, John Baldwin wrote: Unfortunately this can't be MFC'd to 7 as it would destroy the ABI for existing klds. Ah, ok, sorry, I did only check RELENG_7. Can we make it a kernel option then? In principle, yes, but MAXCPU is used to size various kernel data structures inspected by userspace crash post-mortem tools, etc. I've done a bit of work to teach some of those tools (in particular, vmstat -z and vmstat -m) to extract the version of maxcpu compiled into the kernel instead just relying on the version of MAXCPU present when the command line tool was compiled. However, I think a better long-term approach here is to generally eliminate sizing based on MAXCPU and instead size based on the number of CPUs present. Certain kernel subsystems already do this (UMA, netisr, ...) but others don't (malloc(9), ...). Additional hands on this project would probably help :-). As John mentioned, the other issue is the use of fixed-width types instead of variable-length CPU bitmasks to name cores for IPIs, etc. There are people actively working on this, but it's a non-trivial project as kernel code likes to do things like cpumask othermask. My expectation is that this problem will be solved in 9.0 but I don't see any obvious MFC paths for 8.x due to KBI issues. It could be that this forces our hand in terms of breaking the KBI at some point in the 8.x series, unclear... Robert ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Bumping MAXCPU on amd64?
2010/9/22 Maxim Sobolev sobo...@freebsd.org: Hi, Is there any reason to keep MAXCPU at 16 in the default kernel config? There are quite few servers on the market today that have 24 or even 32 physical cores. With hyper-threading this can even go as high as 48 or 64 virtual cpus. People who buy such hardware might get very disappointed finding out that the FreeBSD is not going to use such hardware to its full potential. Does anybody object if I'd bump MAXCPU to 32, which is still low but might me more reasonable default these days, or at least make it an kernel configuration option documented in the NOTES? Please correct me, if I'm about smth. different, but isn't it already? /sys/amd64/include/param.h:#define MAXCPU 32 /sys/arm/include/param.h:#defineMAXCPU 2 /sys/i386/include/param.h:#define MAXCPU32 /sys/ia64/include/param.h:#define MAXCPU 32 /sys/mips/include/param.h:#define MAXCPU MAXSMPCPU /sys/powerpc/include/param.h:#defineMAXCPU 2 /sys/sparc64/include/param.h:#define MAXCPU 16 /sys/sun4v/include/param.h:#define MAXCPU 32 (almost 2y ago for x86) -- wbr, pluknet ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Bumping MAXCPU on amd64?
On Wednesday, September 22, 2010 6:36:56 am Maxim Sobolev wrote: Hi, Is there any reason to keep MAXCPU at 16 in the default kernel config? There are quite few servers on the market today that have 24 or even 32 physical cores. With hyper-threading this can even go as high as 48 or 64 virtual cpus. People who buy such hardware might get very disappointed finding out that the FreeBSD is not going to use such hardware to its full potential. Does anybody object if I'd bump MAXCPU to 32, which is still low but might me more reasonable default these days, or at least make it an kernel configuration option documented in the NOTES? ? % grep MAXCPU ~/work/freebsd/svn/head/sys/amd64/include/param.h #define MAXCPU 32 #define MAXCPU 1 In fact: % grep MAXCPU ~/work/freebsd/svn/stable/8/sys/amd64/include/param.h #define MAXCPU 32 #define MAXCPU 1 Unfortunately this can't be MFC'd to 7 as it would destroy the ABI for existing klds. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Bumping MAXCPU on amd64?
MAXCPU at 32 has been good in the 32bit days. Soon there will be (if not already) systems that will have 16cores/socket or more, and motherboards that have 4 sockets or more. Combining this with hyper-threading, you have gone significantly beyond the limits of feasible server. Bumping the number now is not feasible. But in release 9+ or 10, this number could be bumped to something in the order of 1024 or more. This will not be easy as there are considerable performance and compatibility problems. But with Moore's law it will happen, and freeBSD will need to adapt to stay relevant. Curtis Penner On 09/22/2010 06:37 AM, John Baldwin wrote: On Wednesday, September 22, 2010 6:36:56 am Maxim Sobolev wrote: Hi, Is there any reason to keep MAXCPU at 16 in the default kernel config? There are quite few servers on the market today that have 24 or even 32 physical cores. With hyper-threading this can even go as high as 48 or 64 virtual cpus. People who buy such hardware might get very disappointed finding out that the FreeBSD is not going to use such hardware to its full potential. Does anybody object if I'd bump MAXCPU to 32, which is still low but might me more reasonable default these days, or at least make it an kernel configuration option documented in the NOTES? ? % grep MAXCPU ~/work/freebsd/svn/head/sys/amd64/include/param.h #define MAXCPU 32 #define MAXCPU 1 In fact: % grep MAXCPU ~/work/freebsd/svn/stable/8/sys/amd64/include/param.h #define MAXCPU 32 #define MAXCPU 1 Unfortunately this can't be MFC'd to 7 as it would destroy the ABI for existing klds. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Bumping MAXCPU on amd64?
2010/9/22 Curtis Penner curtis.penn...@gmail.com: MAXCPU at 32 has been good in the 32bit days. Soon there will be (if not already) systems that will have 16cores/socket or more, and motherboards that have 4 sockets or more. Combining this with hyper-threading, you have gone significantly beyond the limits of feasible server. There is a one (16cores per socket, up to 4 sockets, 512 way). http://www.oracle.com/us/corporate/press/173536 -- wbr, pluknet ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Bumping MAXCPU on amd64?
On Wednesday, September 22, 2010 1:08:30 pm Curtis Penner wrote: MAXCPU at 32 has been good in the 32bit days. Soon there will be (if not already) systems that will have 16cores/socket or more, and motherboards that have 4 sockets or more. Combining this with hyper-threading, you have gone significantly beyond the limits of feasible server. My point was in response to Maxim's mail about bumping it from 16. Going higher than 32 is a bigger project (but in progress-ish) as it involves transitioning away from a simple int to hold CPU ID bitmasks (cpumask_t) and using cpuset_t instead. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Bumping MAXCPU on amd64?
On 9/22/2010 6:37 AM, John Baldwin wrote: Unfortunately this can't be MFC'd to 7 as it would destroy the ABI for existing klds. Ah, ok, sorry, I did only check RELENG_7. Can we make it a kernel option then? Regards, -- Maksym Sobolyev Sippy Software, Inc. Internet Telephony (VoIP) Experts T/F: +1-646-651-1110 Web: http://www.sippysoft.com MSN: sa...@sippysoft.com Skype: SippySoft ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
