Re: [PATCH] x86: Allow NR_CPUS=1024
On 2013/11/5 14:25, Ingo Molnar wrote: > > * H. Peter Anvin wrote: > >> On 11/04/2013 12:11 PM, Ingo Molnar wrote: >>> >>> * H. Peter Anvin wrote: >>> 8192 maybe? >>> >>> Yeah, that makes more sense I guess. >>> >> >> However, I still have serious issues with crap like this because >> randconfig is basically broken. If nothing else we need to get that >> feedback to the kconfig maintainers. > > The problem with that is that there are no kconfig maintainers: > Yes, there is. https://lkml.org/lkml/2013/10/30/124 Though I don't know if the new maintainer will help in this issue. > KCONFIG > ... > S: Odd Fixes > > The kconfig code is still a hard to maintain, scarcely documented mess. It > took us almost a decade to rescue the NTP code from a similar obfuscation > trap and make it maintainable. It had the same author as the original > Kconfig code - and the Kconfig code is 10 times larger than the NTP code. > > Thanks, > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* H. Peter Anvin wrote: > On 11/04/2013 12:11 PM, Ingo Molnar wrote: > > > > * H. Peter Anvin wrote: > > > >> 8192 maybe? > > > > Yeah, that makes more sense I guess. > > > > However, I still have serious issues with crap like this because > randconfig is basically broken. If nothing else we need to get that > feedback to the kconfig maintainers. The problem with that is that there are no kconfig maintainers: KCONFIG ... S: Odd Fixes The kconfig code is still a hard to maintain, scarcely documented mess. It took us almost a decade to rescue the NTP code from a similar obfuscation trap and make it maintainable. It had the same author as the original Kconfig code - and the Kconfig code is 10 times larger than the NTP code. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/04/2013 12:11 PM, Ingo Molnar wrote: > > * H. Peter Anvin wrote: > >> 8192 maybe? > > Yeah, that makes more sense I guess. > However, I still have serious issues with crap like this because randconfig is basically broken. If nothing else we need to get that feedback to the kconfig maintainers. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* H. Peter Anvin wrote: > 8192 maybe? Yeah, that makes more sense I guess. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
8192 maybe? Ingo Molnar wrote: > >* Russ Anderson wrote: > >> > Russ, does SGI (or anyone else that you know of) have x86 hardware >> > with more than 4096 CPUs? >> >> Yes. We have a system in the lab with 254 12-core IVB sockets for a >> total of 3048 cores. With HT is it 6096 cpus. > >It would then be nice to up MAXSMP to 6144 or so. > >Thanks, > > Ingo -- Sent from my mobile phone. Please pardon brevity and lack of formatting. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Russ Anderson wrote: > > Russ, does SGI (or anyone else that you know of) have x86 hardware > > with more than 4096 CPUs? > > Yes. We have a system in the lab with 254 12-core IVB sockets for a > total of 3048 cores. With HT is it 6096 cpus. It would then be nice to up MAXSMP to 6144 or so. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/03/2013 10:51 PM, Ingo Molnar wrote: > > randconfig will not randomize numeric Kconfig ranges, so there's no other > mechanism right now to trigger those large config kernels. > Sounds like the real problem... -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Mon, Nov 04, 2013 at 09:16:16AM -0500, Josh Boyer wrote: > On Mon, Nov 04, 2013 at 03:10:51PM +0100, Ingo Molnar wrote: > > * Josh Boyer wrote: > > > > > > Why touch MAXSMP at all? It's really just a shortcut for 'configure > > > > the kernel silly large', via a single option, nothing else. You are > > > > not forced to use it and it should not affect configurability of > > > > NR_CPUS. > > > > > > > > What we _really_ want here is to fix NR_CPUS setting: to extend its > > > > range and to enforce that NR_CPUS cannot be set larger than 512 > > > > without setting CONFIG_CPUMASK_OFFSTACK. > > > > > > OK. I was just thinking that if we've come to the conclusion that 4096 > > > CPUs isn't silly large anymore, we should make MAXSMP be something we > > > consider silly large. [...] > > > > MAXSMP is also supposed to track the real hardware max as well on x86 - > > i.e. we should only increase it to 8192 etc. if such hardware exists. > > Russ, does SGI (or anyone else that you know of) have x86 hardware with > more than 4096 CPUs? Yes. We have a system in the lab with 254 12-core IVB sockets for a total of 3048 cores. With HT is it 6096 cpus. > If so, I can actually make a bump to the MAXSMP count a separate patch. > > josh -- Russ Anderson, OS RAS/Partitioning Project Lead SGI - Silicon Graphics Inc r...@sgi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/04/2013 09:16 AM, Josh Boyer wrote: > On Mon, Nov 04, 2013 at 03:10:51PM +0100, Ingo Molnar wrote: >> >> * Josh Boyer wrote: >> Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. >>> >>> OK. I was just thinking that if we've come to the conclusion that 4096 >>> CPUs isn't silly large anymore, we should make MAXSMP be something we >>> consider silly large. [...] >> >> MAXSMP is also supposed to track the real hardware max as well on x86 - >> i.e. we should only increase it to 8192 etc. if such hardware exists. > > Russ, does SGI (or anyone else that you know of) have x86 hardware with > more than 4096 CPUs? I can answer this for Russ. Yes, SGI has boxes that hit 5120. P. > > If so, I can actually make a bump to the MAXSMP count a separate patch. > > josh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Mon, Nov 04, 2013 at 03:10:51PM +0100, Ingo Molnar wrote: > > * Josh Boyer wrote: > > > > Why touch MAXSMP at all? It's really just a shortcut for 'configure > > > the kernel silly large', via a single option, nothing else. You are > > > not forced to use it and it should not affect configurability of > > > NR_CPUS. > > > > > > What we _really_ want here is to fix NR_CPUS setting: to extend its > > > range and to enforce that NR_CPUS cannot be set larger than 512 > > > without setting CONFIG_CPUMASK_OFFSTACK. > > > > OK. I was just thinking that if we've come to the conclusion that 4096 > > CPUs isn't silly large anymore, we should make MAXSMP be something we > > consider silly large. [...] > > MAXSMP is also supposed to track the real hardware max as well on x86 - > i.e. we should only increase it to 8192 etc. if such hardware exists. Russ, does SGI (or anyone else that you know of) have x86 hardware with more than 4096 CPUs? If so, I can actually make a bump to the MAXSMP count a separate patch. josh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Josh Boyer wrote: > > Why touch MAXSMP at all? It's really just a shortcut for 'configure > > the kernel silly large', via a single option, nothing else. You are > > not forced to use it and it should not affect configurability of > > NR_CPUS. > > > > What we _really_ want here is to fix NR_CPUS setting: to extend its > > range and to enforce that NR_CPUS cannot be set larger than 512 > > without setting CONFIG_CPUMASK_OFFSTACK. > > OK. I was just thinking that if we've come to the conclusion that 4096 > CPUs isn't silly large anymore, we should make MAXSMP be something we > consider silly large. [...] MAXSMP is also supposed to track the real hardware max as well on x86 - i.e. we should only increase it to 8192 etc. if such hardware exists. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Mon, Nov 04, 2013 at 07:53:43AM +0100, Ingo Molnar wrote: > > * Josh Boyer wrote: > > > On Sun, Nov 03, 2013 at 11:21:32AM +0100, Ingo Molnar wrote: > > > > > > * Ingo Molnar wrote: > > > > > > > > > > > * Josh Boyer wrote: > > > > > > > > > The current range for SMP configs is 2 - 512, or a full 4096 in the > > > > > case > > > > > of MAXSMP. There are machines that have 1024 CPUs in them today and > > > > > configuring a kernel for that means you are forced to set MAXSMP. > > > > > This > > > > > adds additional unnecessary overhead. While that overhead might be > > > > > considered tiny for large machines, it isn't necessarily so if you > > > > > are > > > > > building a kernel that runs across a wide variety of machines. We > > > > > increase the range to 1024 to help with this. > > > > > > > > > > Signed-off-by: Josh Boyer > > > > > --- > > > > > arch/x86/Kconfig | 2 +- > > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > > > > > index f67e839..d726b2d 100644 > > > > > --- a/arch/x86/Kconfig > > > > > +++ b/arch/x86/Kconfig > > > > > @@ -825,7 +825,7 @@ config MAXSMP > > > > > config NR_CPUS > > > > > int "Maximum number of CPUs" if SMP && !MAXSMP > > > > > range 2 8 if SMP && X86_32 && !X86_BIGSMP > > > > > - range 2 512 if SMP && !MAXSMP > > > > > + range 2 1024 if SMP && !MAXSMP > > > > > default "1" if !SMP > > > > > default "4096" if MAXSMP > > > > > default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP > > > > > || X86_ES7000) > > > > > > > > Any reason not to allow it to go up to 4096? The original concern was > > > > that CPUS=4096 wasn't working very well and you had to select MAXSMP > > > > deliberately and keep all the pieces. > > > > No real reason to not allow all the way to 4096, no. I just started > > small as I wanted 1024 specifically, and this is the simplest way to > > achieve that. > > > > > The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is > > > 512 bytes, too large to be kept on the kernel stack. > > > > > > MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. > > > > > > With 1024 CPUs a single cpumask is 128 bytes - rather significant as > > > well. > > > With 512 CPUs it's 64 bytes - borderline. > > > > > > So I think a better solution would be to allow an increase above 512 CPUs > > > only if CONFIG_CPUMASK_OFFSTACK is also enabled. > > > > OK, that makes sense. So in this scenario, we could probably either: > > > > a) do away with MAXSMP entirely and just depend on > > CONFIG_CPUMASK_OFFSTACK. > > > > b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. > > > > Which would you prefer? Either is easy enough to code up, I just need > > to know which I should shoot for. > > Why touch MAXSMP at all? It's really just a shortcut for 'configure the > kernel silly large', via a single option, nothing else. You are not forced > to use it and it should not affect configurability of NR_CPUS. > > What we _really_ want here is to fix NR_CPUS setting: to extend its range > and to enforce that NR_CPUS cannot be set larger than 512 without setting > CONFIG_CPUMASK_OFFSTACK. OK. I was just thinking that if we've come to the conclusion that 4096 CPUs isn't silly large anymore, we should make MAXSMP be something we consider silly large. Anyway, I'm fine with leaving MAXSMP as it is. I'll work on the rest today. josh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/03/2013 10:51 PM, Ingo Molnar wrote: randconfig will not randomize numeric Kconfig ranges, so there's no other mechanism right now to trigger those large config kernels. Sounds like the real problem... -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Russ Anderson r...@sgi.com wrote: Russ, does SGI (or anyone else that you know of) have x86 hardware with more than 4096 CPUs? Yes. We have a system in the lab with 254 12-core IVB sockets for a total of 3048 cores. With HT is it 6096 cpus. It would then be nice to up MAXSMP to 6144 or so. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
8192 maybe? Ingo Molnar mi...@kernel.org wrote: * Russ Anderson r...@sgi.com wrote: Russ, does SGI (or anyone else that you know of) have x86 hardware with more than 4096 CPUs? Yes. We have a system in the lab with 254 12-core IVB sockets for a total of 3048 cores. With HT is it 6096 cpus. It would then be nice to up MAXSMP to 6144 or so. Thanks, Ingo -- Sent from my mobile phone. Please pardon brevity and lack of formatting. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* H. Peter Anvin h...@zytor.com wrote: 8192 maybe? Yeah, that makes more sense I guess. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/04/2013 12:11 PM, Ingo Molnar wrote: * H. Peter Anvin h...@zytor.com wrote: 8192 maybe? Yeah, that makes more sense I guess. However, I still have serious issues with crap like this because randconfig is basically broken. If nothing else we need to get that feedback to the kconfig maintainers. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* H. Peter Anvin h...@zytor.com wrote: On 11/04/2013 12:11 PM, Ingo Molnar wrote: * H. Peter Anvin h...@zytor.com wrote: 8192 maybe? Yeah, that makes more sense I guess. However, I still have serious issues with crap like this because randconfig is basically broken. If nothing else we need to get that feedback to the kconfig maintainers. The problem with that is that there are no kconfig maintainers: KCONFIG ... S: Odd Fixes The kconfig code is still a hard to maintain, scarcely documented mess. It took us almost a decade to rescue the NTP code from a similar obfuscation trap and make it maintainable. It had the same author as the original Kconfig code - and the Kconfig code is 10 times larger than the NTP code. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 2013/11/5 14:25, Ingo Molnar wrote: * H. Peter Anvin h...@zytor.com wrote: On 11/04/2013 12:11 PM, Ingo Molnar wrote: * H. Peter Anvin h...@zytor.com wrote: 8192 maybe? Yeah, that makes more sense I guess. However, I still have serious issues with crap like this because randconfig is basically broken. If nothing else we need to get that feedback to the kconfig maintainers. The problem with that is that there are no kconfig maintainers: Yes, there is. https://lkml.org/lkml/2013/10/30/124 Though I don't know if the new maintainer will help in this issue. KCONFIG ... S: Odd Fixes The kconfig code is still a hard to maintain, scarcely documented mess. It took us almost a decade to rescue the NTP code from a similar obfuscation trap and make it maintainable. It had the same author as the original Kconfig code - and the Kconfig code is 10 times larger than the NTP code. Thanks, -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Mon, Nov 04, 2013 at 07:53:43AM +0100, Ingo Molnar wrote: * Josh Boyer jwbo...@redhat.com wrote: On Sun, Nov 03, 2013 at 11:21:32AM +0100, Ingo Molnar wrote: * Ingo Molnar mi...@kernel.org wrote: * Josh Boyer jwbo...@redhat.com wrote: The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP - range 2 512 if SMP !MAXSMP + range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. No real reason to not allow all the way to 4096, no. I just started small as I wanted 1024 specifically, and this is the simplest way to achieve that. The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is 512 bytes, too large to be kept on the kernel stack. MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. With 512 CPUs it's 64 bytes - borderline. So I think a better solution would be to allow an increase above 512 CPUs only if CONFIG_CPUMASK_OFFSTACK is also enabled. OK, that makes sense. So in this scenario, we could probably either: a) do away with MAXSMP entirely and just depend on CONFIG_CPUMASK_OFFSTACK. b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. Which would you prefer? Either is easy enough to code up, I just need to know which I should shoot for. Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. OK. I was just thinking that if we've come to the conclusion that 4096 CPUs isn't silly large anymore, we should make MAXSMP be something we consider silly large. Anyway, I'm fine with leaving MAXSMP as it is. I'll work on the rest today. josh -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Josh Boyer jwbo...@redhat.com wrote: Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. OK. I was just thinking that if we've come to the conclusion that 4096 CPUs isn't silly large anymore, we should make MAXSMP be something we consider silly large. [...] MAXSMP is also supposed to track the real hardware max as well on x86 - i.e. we should only increase it to 8192 etc. if such hardware exists. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Mon, Nov 04, 2013 at 03:10:51PM +0100, Ingo Molnar wrote: * Josh Boyer jwbo...@redhat.com wrote: Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. OK. I was just thinking that if we've come to the conclusion that 4096 CPUs isn't silly large anymore, we should make MAXSMP be something we consider silly large. [...] MAXSMP is also supposed to track the real hardware max as well on x86 - i.e. we should only increase it to 8192 etc. if such hardware exists. Russ, does SGI (or anyone else that you know of) have x86 hardware with more than 4096 CPUs? If so, I can actually make a bump to the MAXSMP count a separate patch. josh -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/04/2013 09:16 AM, Josh Boyer wrote: On Mon, Nov 04, 2013 at 03:10:51PM +0100, Ingo Molnar wrote: * Josh Boyer jwbo...@redhat.com wrote: Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. OK. I was just thinking that if we've come to the conclusion that 4096 CPUs isn't silly large anymore, we should make MAXSMP be something we consider silly large. [...] MAXSMP is also supposed to track the real hardware max as well on x86 - i.e. we should only increase it to 8192 etc. if such hardware exists. Russ, does SGI (or anyone else that you know of) have x86 hardware with more than 4096 CPUs? I can answer this for Russ. Yes, SGI has boxes that hit 5120. P. If so, I can actually make a bump to the MAXSMP count a separate patch. josh -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Mon, Nov 04, 2013 at 09:16:16AM -0500, Josh Boyer wrote: On Mon, Nov 04, 2013 at 03:10:51PM +0100, Ingo Molnar wrote: * Josh Boyer jwbo...@redhat.com wrote: Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. OK. I was just thinking that if we've come to the conclusion that 4096 CPUs isn't silly large anymore, we should make MAXSMP be something we consider silly large. [...] MAXSMP is also supposed to track the real hardware max as well on x86 - i.e. we should only increase it to 8192 etc. if such hardware exists. Russ, does SGI (or anyone else that you know of) have x86 hardware with more than 4096 CPUs? Yes. We have a system in the lab with 254 12-core IVB sockets for a total of 3048 cores. With HT is it 6096 cpus. If so, I can actually make a bump to the MAXSMP count a separate patch. josh -- Russ Anderson, OS RAS/Partitioning Project Lead SGI - Silicon Graphics Inc r...@sgi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Josh Boyer wrote: > On Sun, Nov 03, 2013 at 11:21:32AM +0100, Ingo Molnar wrote: > > > > * Ingo Molnar wrote: > > > > > > > > * Josh Boyer wrote: > > > > > > > The current range for SMP configs is 2 - 512, or a full 4096 in the > > > > case > > > > of MAXSMP. There are machines that have 1024 CPUs in them today and > > > > configuring a kernel for that means you are forced to set MAXSMP. This > > > > adds additional unnecessary overhead. While that overhead might be > > > > considered tiny for large machines, it isn't necessarily so if you are > > > > building a kernel that runs across a wide variety of machines. We > > > > increase the range to 1024 to help with this. > > > > > > > > Signed-off-by: Josh Boyer > > > > --- > > > > arch/x86/Kconfig | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > > > > index f67e839..d726b2d 100644 > > > > --- a/arch/x86/Kconfig > > > > +++ b/arch/x86/Kconfig > > > > @@ -825,7 +825,7 @@ config MAXSMP > > > > config NR_CPUS > > > > int "Maximum number of CPUs" if SMP && !MAXSMP > > > > range 2 8 if SMP && X86_32 && !X86_BIGSMP > > > > - range 2 512 if SMP && !MAXSMP > > > > + range 2 1024 if SMP && !MAXSMP > > > > default "1" if !SMP > > > > default "4096" if MAXSMP > > > > default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP > > > > || X86_ES7000) > > > > > > Any reason not to allow it to go up to 4096? The original concern was > > > that CPUS=4096 wasn't working very well and you had to select MAXSMP > > > deliberately and keep all the pieces. > > No real reason to not allow all the way to 4096, no. I just started > small as I wanted 1024 specifically, and this is the simplest way to > achieve that. > > > The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is > > 512 bytes, too large to be kept on the kernel stack. > > > > MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. > > > > With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. > > With 512 CPUs it's 64 bytes - borderline. > > > > So I think a better solution would be to allow an increase above 512 CPUs > > only if CONFIG_CPUMASK_OFFSTACK is also enabled. > > OK, that makes sense. So in this scenario, we could probably either: > > a) do away with MAXSMP entirely and just depend on > CONFIG_CPUMASK_OFFSTACK. > > b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. > > Which would you prefer? Either is easy enough to code up, I just need > to know which I should shoot for. Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* H. Peter Anvin wrote: > On 11/03/2013 07:57 AM, Josh Boyer wrote: > > > > OK, that makes sense. So in this scenario, we could probably either: > > > > a) do away with MAXSMP entirely and just depend on > > CONFIG_CPUMASK_OFFSTACK. > > > > b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. > > > > Which would you prefer? Either is easy enough to code up, I just need > > to know which I should shoot for. > > > > Let's get rid of MAXSMP. I'd rather not, because it has caught a number of regressions in the past, because randconfig can wander over it and trigger those large configs. randconfig will not randomize numeric Kconfig ranges, so there's no other mechanism right now to trigger those large config kernels. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/03/2013 07:57 AM, Josh Boyer wrote: > > OK, that makes sense. So in this scenario, we could probably either: > > a) do away with MAXSMP entirely and just depend on > CONFIG_CPUMASK_OFFSTACK. > > b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. > > Which would you prefer? Either is easy enough to code up, I just need > to know which I should shoot for. > Let's get rid of MAXSMP. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Sun, Nov 03, 2013 at 11:21:32AM +0100, Ingo Molnar wrote: > > * Ingo Molnar wrote: > > > > > * Josh Boyer wrote: > > > > > The current range for SMP configs is 2 - 512, or a full 4096 in the case > > > of MAXSMP. There are machines that have 1024 CPUs in them today and > > > configuring a kernel for that means you are forced to set MAXSMP. This > > > adds additional unnecessary overhead. While that overhead might be > > > considered tiny for large machines, it isn't necessarily so if you are > > > building a kernel that runs across a wide variety of machines. We > > > increase the range to 1024 to help with this. > > > > > > Signed-off-by: Josh Boyer > > > --- > > > arch/x86/Kconfig | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > > > index f67e839..d726b2d 100644 > > > --- a/arch/x86/Kconfig > > > +++ b/arch/x86/Kconfig > > > @@ -825,7 +825,7 @@ config MAXSMP > > > config NR_CPUS > > > int "Maximum number of CPUs" if SMP && !MAXSMP > > > range 2 8 if SMP && X86_32 && !X86_BIGSMP > > > - range 2 512 if SMP && !MAXSMP > > > + range 2 1024 if SMP && !MAXSMP > > > default "1" if !SMP > > > default "4096" if MAXSMP > > > default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || > > > X86_ES7000) > > > > Any reason not to allow it to go up to 4096? The original concern was > > that CPUS=4096 wasn't working very well and you had to select MAXSMP > > deliberately and keep all the pieces. No real reason to not allow all the way to 4096, no. I just started small as I wanted 1024 specifically, and this is the simplest way to achieve that. > The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is > 512 bytes, too large to be kept on the kernel stack. > > MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. > > With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. > With 512 CPUs it's 64 bytes - borderline. > > So I think a better solution would be to allow an increase above 512 CPUs > only if CONFIG_CPUMASK_OFFSTACK is also enabled. OK, that makes sense. So in this scenario, we could probably either: a) do away with MAXSMP entirely and just depend on CONFIG_CPUMASK_OFFSTACK. b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. Which would you prefer? Either is easy enough to code up, I just need to know which I should shoot for. josh -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Sun, Nov 03, 2013 at 09:29:16AM -0500, Prarit Bhargava wrote: > On 11/03/2013 05:18 AM, Ingo Molnar wrote: > > > > * Josh Boyer wrote: > > > >> The current range for SMP configs is 2 - 512, or a full 4096 in the case > >> of MAXSMP. There are machines that have 1024 CPUs in them today and > >> configuring a kernel for that means you are forced to set MAXSMP. This > >> adds additional unnecessary overhead. While that overhead might be > >> considered tiny for large machines, it isn't necessarily so if you are > >> building a kernel that runs across a wide variety of machines. We > >> increase the range to 1024 to help with this. > >> > >> Signed-off-by: Josh Boyer > >> --- > >> arch/x86/Kconfig | 2 +- > >> 1 file changed, 1 insertion(+), 1 deletion(-) > >> > >> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > >> index f67e839..d726b2d 100644 > >> --- a/arch/x86/Kconfig > >> +++ b/arch/x86/Kconfig > >> @@ -825,7 +825,7 @@ config MAXSMP > >> config NR_CPUS > >>int "Maximum number of CPUs" if SMP && !MAXSMP > >>range 2 8 if SMP && X86_32 && !X86_BIGSMP > >> - range 2 512 if SMP && !MAXSMP > >> + range 2 1024 if SMP && !MAXSMP > >>default "1" if !SMP > >>default "4096" if MAXSMP > >>default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || > >> X86_ES7000) > > > > Any reason not to allow it to go up to 4096? The original concern was that > > CPUS=4096 wasn't working very well and you had to select MAXSMP > > deliberately and keep all the pieces. > > > > But today it's all pretty robust so I see no reason why not to allow up to > > 4096 CPUs. > > Adding Russ from SGI as they are one of the consumers of a large CPU count. > > I have no objections to raising this to 4096 FWIW. I think it is a good idea, > and it is long overdue. I obviously agree with increasing to 4096. The bigger the better. -- Russ Anderson, OS RAS/Partitioning Project Lead SGI - Silicon Graphics Inc r...@sgi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/03/2013 05:18 AM, Ingo Molnar wrote: > > * Josh Boyer wrote: > >> The current range for SMP configs is 2 - 512, or a full 4096 in the case >> of MAXSMP. There are machines that have 1024 CPUs in them today and >> configuring a kernel for that means you are forced to set MAXSMP. This >> adds additional unnecessary overhead. While that overhead might be >> considered tiny for large machines, it isn't necessarily so if you are >> building a kernel that runs across a wide variety of machines. We >> increase the range to 1024 to help with this. >> >> Signed-off-by: Josh Boyer >> --- >> arch/x86/Kconfig | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig >> index f67e839..d726b2d 100644 >> --- a/arch/x86/Kconfig >> +++ b/arch/x86/Kconfig >> @@ -825,7 +825,7 @@ config MAXSMP >> config NR_CPUS >> int "Maximum number of CPUs" if SMP && !MAXSMP >> range 2 8 if SMP && X86_32 && !X86_BIGSMP >> -range 2 512 if SMP && !MAXSMP >> +range 2 1024 if SMP && !MAXSMP >> default "1" if !SMP >> default "4096" if MAXSMP >> default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || >> X86_ES7000) > > Any reason not to allow it to go up to 4096? The original concern was that > CPUS=4096 wasn't working very well and you had to select MAXSMP > deliberately and keep all the pieces. > > But today it's all pretty robust so I see no reason why not to allow up to > 4096 CPUs. Adding Russ from SGI as they are one of the consumers of a large CPU count. I have no objections to raising this to 4096 FWIW. I think it is a good idea, and it is long overdue. P. > > Thanks, > > Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Ingo Molnar wrote: > > * Josh Boyer wrote: > > > The current range for SMP configs is 2 - 512, or a full 4096 in the case > > of MAXSMP. There are machines that have 1024 CPUs in them today and > > configuring a kernel for that means you are forced to set MAXSMP. This > > adds additional unnecessary overhead. While that overhead might be > > considered tiny for large machines, it isn't necessarily so if you are > > building a kernel that runs across a wide variety of machines. We > > increase the range to 1024 to help with this. > > > > Signed-off-by: Josh Boyer > > --- > > arch/x86/Kconfig | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > > index f67e839..d726b2d 100644 > > --- a/arch/x86/Kconfig > > +++ b/arch/x86/Kconfig > > @@ -825,7 +825,7 @@ config MAXSMP > > config NR_CPUS > > int "Maximum number of CPUs" if SMP && !MAXSMP > > range 2 8 if SMP && X86_32 && !X86_BIGSMP > > - range 2 512 if SMP && !MAXSMP > > + range 2 1024 if SMP && !MAXSMP > > default "1" if !SMP > > default "4096" if MAXSMP > > default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || > > X86_ES7000) > > Any reason not to allow it to go up to 4096? The original concern was > that CPUS=4096 wasn't working very well and you had to select MAXSMP > deliberately and keep all the pieces. The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is 512 bytes, too large to be kept on the kernel stack. MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. With 512 CPUs it's 64 bytes - borderline. So I think a better solution would be to allow an increase above 512 CPUs only if CONFIG_CPUMASK_OFFSTACK is also enabled. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Josh Boyer wrote: > The current range for SMP configs is 2 - 512, or a full 4096 in the case > of MAXSMP. There are machines that have 1024 CPUs in them today and > configuring a kernel for that means you are forced to set MAXSMP. This > adds additional unnecessary overhead. While that overhead might be > considered tiny for large machines, it isn't necessarily so if you are > building a kernel that runs across a wide variety of machines. We > increase the range to 1024 to help with this. > > Signed-off-by: Josh Boyer > --- > arch/x86/Kconfig | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > index f67e839..d726b2d 100644 > --- a/arch/x86/Kconfig > +++ b/arch/x86/Kconfig > @@ -825,7 +825,7 @@ config MAXSMP > config NR_CPUS > int "Maximum number of CPUs" if SMP && !MAXSMP > range 2 8 if SMP && X86_32 && !X86_BIGSMP > - range 2 512 if SMP && !MAXSMP > + range 2 1024 if SMP && !MAXSMP > default "1" if !SMP > default "4096" if MAXSMP > default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || > X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. But today it's all pretty robust so I see no reason why not to allow up to 4096 CPUs. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Josh Boyer jwbo...@redhat.com wrote: The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP - range 2 512 if SMP !MAXSMP + range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. But today it's all pretty robust so I see no reason why not to allow up to 4096 CPUs. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Ingo Molnar mi...@kernel.org wrote: * Josh Boyer jwbo...@redhat.com wrote: The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP - range 2 512 if SMP !MAXSMP + range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is 512 bytes, too large to be kept on the kernel stack. MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. With 512 CPUs it's 64 bytes - borderline. So I think a better solution would be to allow an increase above 512 CPUs only if CONFIG_CPUMASK_OFFSTACK is also enabled. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/03/2013 05:18 AM, Ingo Molnar wrote: * Josh Boyer jwbo...@redhat.com wrote: The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP -range 2 512 if SMP !MAXSMP +range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. But today it's all pretty robust so I see no reason why not to allow up to 4096 CPUs. Adding Russ from SGI as they are one of the consumers of a large CPU count. I have no objections to raising this to 4096 FWIW. I think it is a good idea, and it is long overdue. P. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Sun, Nov 03, 2013 at 09:29:16AM -0500, Prarit Bhargava wrote: On 11/03/2013 05:18 AM, Ingo Molnar wrote: * Josh Boyer jwbo...@redhat.com wrote: The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP - range 2 512 if SMP !MAXSMP + range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. But today it's all pretty robust so I see no reason why not to allow up to 4096 CPUs. Adding Russ from SGI as they are one of the consumers of a large CPU count. I have no objections to raising this to 4096 FWIW. I think it is a good idea, and it is long overdue. I obviously agree with increasing to 4096. The bigger the better. -- Russ Anderson, OS RAS/Partitioning Project Lead SGI - Silicon Graphics Inc r...@sgi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On Sun, Nov 03, 2013 at 11:21:32AM +0100, Ingo Molnar wrote: * Ingo Molnar mi...@kernel.org wrote: * Josh Boyer jwbo...@redhat.com wrote: The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP - range 2 512 if SMP !MAXSMP + range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. No real reason to not allow all the way to 4096, no. I just started small as I wanted 1024 specifically, and this is the simplest way to achieve that. The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is 512 bytes, too large to be kept on the kernel stack. MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. With 512 CPUs it's 64 bytes - borderline. So I think a better solution would be to allow an increase above 512 CPUs only if CONFIG_CPUMASK_OFFSTACK is also enabled. OK, that makes sense. So in this scenario, we could probably either: a) do away with MAXSMP entirely and just depend on CONFIG_CPUMASK_OFFSTACK. b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. Which would you prefer? Either is easy enough to code up, I just need to know which I should shoot for. josh -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
On 11/03/2013 07:57 AM, Josh Boyer wrote: OK, that makes sense. So in this scenario, we could probably either: a) do away with MAXSMP entirely and just depend on CONFIG_CPUMASK_OFFSTACK. b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. Which would you prefer? Either is easy enough to code up, I just need to know which I should shoot for. Let's get rid of MAXSMP. -hpa -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* H. Peter Anvin h...@zytor.com wrote: On 11/03/2013 07:57 AM, Josh Boyer wrote: OK, that makes sense. So in this scenario, we could probably either: a) do away with MAXSMP entirely and just depend on CONFIG_CPUMASK_OFFSTACK. b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. Which would you prefer? Either is easy enough to code up, I just need to know which I should shoot for. Let's get rid of MAXSMP. I'd rather not, because it has caught a number of regressions in the past, because randconfig can wander over it and trigger those large configs. randconfig will not randomize numeric Kconfig ranges, so there's no other mechanism right now to trigger those large config kernels. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] x86: Allow NR_CPUS=1024
* Josh Boyer jwbo...@redhat.com wrote: On Sun, Nov 03, 2013 at 11:21:32AM +0100, Ingo Molnar wrote: * Ingo Molnar mi...@kernel.org wrote: * Josh Boyer jwbo...@redhat.com wrote: The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP - range 2 512 if SMP !MAXSMP + range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) Any reason not to allow it to go up to 4096? The original concern was that CPUS=4096 wasn't working very well and you had to select MAXSMP deliberately and keep all the pieces. No real reason to not allow all the way to 4096, no. I just started small as I wanted 1024 specifically, and this is the simplest way to achieve that. The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is 512 bytes, too large to be kept on the kernel stack. MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there. With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. With 512 CPUs it's 64 bytes - borderline. So I think a better solution would be to allow an increase above 512 CPUs only if CONFIG_CPUMASK_OFFSTACK is also enabled. OK, that makes sense. So in this scenario, we could probably either: a) do away with MAXSMP entirely and just depend on CONFIG_CPUMASK_OFFSTACK. b) make MAXSMP something even higher than 4096. Like 5120 or 6144, etc. Which would you prefer? Either is easy enough to code up, I just need to know which I should shoot for. Why touch MAXSMP at all? It's really just a shortcut for 'configure the kernel silly large', via a single option, nothing else. You are not forced to use it and it should not affect configurability of NR_CPUS. What we _really_ want here is to fix NR_CPUS setting: to extend its range and to enforce that NR_CPUS cannot be set larger than 512 without setting CONFIG_CPUMASK_OFFSTACK. Thanks, Ingo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH] x86: Allow NR_CPUS=1024
The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int "Maximum number of CPUs" if SMP && !MAXSMP range 2 8 if SMP && X86_32 && !X86_BIGSMP - range 2 512 if SMP && !MAXSMP + range 2 1024 if SMP && !MAXSMP default "1" if !SMP default "4096" if MAXSMP default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH] x86: Allow NR_CPUS=1024
The current range for SMP configs is 2 - 512, or a full 4096 in the case of MAXSMP. There are machines that have 1024 CPUs in them today and configuring a kernel for that means you are forced to set MAXSMP. This adds additional unnecessary overhead. While that overhead might be considered tiny for large machines, it isn't necessarily so if you are building a kernel that runs across a wide variety of machines. We increase the range to 1024 to help with this. Signed-off-by: Josh Boyer jwbo...@fedoraproject.org --- arch/x86/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index f67e839..d726b2d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,7 +825,7 @@ config MAXSMP config NR_CPUS int Maximum number of CPUs if SMP !MAXSMP range 2 8 if SMP X86_32 !X86_BIGSMP - range 2 512 if SMP !MAXSMP + range 2 1024 if SMP !MAXSMP default 1 if !SMP default 4096 if MAXSMP default 32 if SMP (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000) -- 1.8.3.1 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
