Re: svn commit: r242029 - head/sys/kern
Peter, I agree. It's certainly not perfect, however it's not nearly as bogus as what was there previously. I know maxusers is wrong, however what it really means, if you think about it, is give me a scaling factor that is relative to physical ram, BUT capped at some value so as to not exhaust KVA. Yes, I grok that on certain architectures that mbufs clusters aren't pulled from KVA, but, that seems much less important than how broken it is currently This fix is good enough for the general case, and a far greater improvement than what was there previously which would make FreeBSD blow chunks on any sort of 10gigE load. I think what needs to happen here, is that the people requiring perfection think about what mess it was prior and if they themselves do not have time to make it 100% perfect, allowing someone to step in and move something a step in the right direction without overly complicating it. What is there is crap, it's old, crufty and broken, it really is. It needs to be fixed, it needs to be given a nice fat band-aid now, and when someone interested in perfection comes along, then they can make it even more awesome. I am not saying that my fix is PERFECT or the be all and end all, however it serves as a good step in the right direction on our tier 1 platforms and is easily modifiable (just replace VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS with some form of MD magic sauce.) Would you like me to do that? Replace the hardline calculation with some constant that each platform can configure? I'm thinking this might suffice to make purists a bit more happy: #if defined(i386) || defined(amd64) #define MAX_KERNEL_ADDRESS_SPACE (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS) #else #define MAX_KERNEL_ADDRESS_SPACE (1024*1024*1024) #endif Given my algorithm this should result in pretty much the same for other platforms than amd64 which will then be able to grow maxusers some. I'm basically running out of time on this and I'm worried that I'll have to back it out indefinitely so that FreeBSD can't do 10gigE out of the box. -Alfred On 11/7/12 11:46 PM, Peter Wemm wrote: On Wed, Nov 7, 2012 at 10:24 PM, Alfred Perlstein bri...@mu.org wrote: [[ + peter ]] Folks, I spent quite a bit of time trying to figure out how to resolve maxusers scaling in a happy way for all. I think I came up with a solution. This solution should work for i386, and other 32 bit platforms, as well as scaling well for 64 bit (or higher) platforms that have virtually unlimited AND 64bit with limited kernel address space. Here is how it works: We calculate the maxusers value based on physical memory, and then clamp it down if physical memory exceeds kernel addressable memory. The algorithm actually remains the same for all architectures, with the exception that machines with large kernel address space it is no longer clamped at 384. I've attached a test program that lets you play with various values for VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS and physpages. (argv[1, 2, 3] respectively.) Please give me your feedback. This is still bogus. VM_MIN_KERNEL_ADDRESS and VM_MAX_KERNEL_ADDRESS have no bearing on how much space should be allocated for mbuf clusters on amd64. If anything, you want dmapbase / dmapend if you want a practical cap for amd64. Even then, jumbo clusters are 4K so they come out of kva rather than direct map. maxusers is the wrong thing for this. maxusers should, if anything, be used to set things like kern.maxproc. Preferably it should be deleted entirely and sysctl.conf should be used to change kern.maxproc. Setting limits for the mbuf / cluster pool should be a MD parameter. Trying to scale maxusers based on physical ram in order to get mbuf cluster limits set as a side effect is just plain wrong. It makes no more sense than trying to set nmbclusters based on PRINTF_BUFR_SIZE, and then trying to scale PRINTF_BUFR_SIZE in order to get desirable second and third order side effects. Scale nmbclusters based on physical ram, with a MD method for capping it for when there are MD limits (eg: disproportionately small kva on an i386 PAE machine). Don't use maxusers. ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r242029 - head/sys/kern
On Nov 8, 2012, at 1:13 AM, Alfred Perlstein wrote: Peter, I agree. It's certainly not perfect, however it's not nearly as bogus as what was there previously. I know maxusers is wrong, however what it really means, if you think about it, is give me a scaling factor that is relative to physical ram, BUT capped at some value so as to not exhaust KVA. Yes, I grok that on certain architectures that mbufs clusters aren't pulled from KVA, but, that seems much less important than how broken it is currently This fix is good enough for the general case, and a far greater improvement than what was there previously which would make FreeBSD blow chunks on any sort of 10gigE load. I think what needs to happen here, is that the people requiring perfection think about what mess it was prior and if they themselves do not have time to make it 100% perfect, allowing someone to step in and move something a step in the right direction without overly complicating it. What is there is crap, it's old, crufty and broken, it really is. It needs to be fixed, it needs to be given a nice fat band-aid now, and when someone interested in perfection comes along, then they can make it even more awesome. I am not saying that my fix is PERFECT or the be all and end all, however it serves as a good step in the right direction on our tier 1 platforms and is easily modifiable (just replace VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS with some form of MD magic sauce.) Would you like me to do that? Replace the hardline calculation with some constant that each platform can configure? I'm thinking this might suffice to make purists a bit more happy: #if defined(i386) || defined(amd64) #define MAX_KERNEL_ADDRESS_SPACE (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS) #else #define MAX_KERNEL_ADDRESS_SPACE (1024*1024*1024) Only 1GB for KVA on 64-bit platforms too...? #endif Given my algorithm this should result in pretty much the same for other platforms than amd64 which will then be able to grow maxusers some. I'm basically running out of time on this and I'm worried that I'll have to back it out indefinitely so that FreeBSD can't do 10gigE out of the box. I agree with the philosophy being taken. Perfection shouldn't be the enemy of the good as long as it gets close enough that things are more useful than they are today. Thanks for championing this Alfred. The less voodoo required for FreeBSD to function on commodity hardware, the better! -Garrett ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r242029 - head/sys/kern
On 08.11.2012 08:46, Peter Wemm wrote: On Wed, Nov 7, 2012 at 10:24 PM, Alfred Perlstein bri...@mu.org wrote: [[ + peter ]] Folks, I spent quite a bit of time trying to figure out how to resolve maxusers scaling in a happy way for all. I think I came up with a solution. This solution should work for i386, and other 32 bit platforms, as well as scaling well for 64 bit (or higher) platforms that have virtually unlimited AND 64bit with limited kernel address space. Here is how it works: We calculate the maxusers value based on physical memory, and then clamp it down if physical memory exceeds kernel addressable memory. The algorithm actually remains the same for all architectures, with the exception that machines with large kernel address space it is no longer clamped at 384. I've attached a test program that lets you play with various values for VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS and physpages. (argv[1, 2, 3] respectively.) Please give me your feedback. This is still bogus. VM_MIN_KERNEL_ADDRESS and VM_MAX_KERNEL_ADDRESS have no bearing on how much space should be allocated for mbuf clusters on amd64. If anything, you want dmapbase / dmapend if you want a practical cap for amd64. Even then, jumbo clusters are 4K so they come out of kva rather than direct map. maxusers is the wrong thing for this. maxusers should, if anything, be used to set things like kern.maxproc. Preferably it should be deleted entirely and sysctl.conf should be used to change kern.maxproc. Setting limits for the mbuf / cluster pool should be a MD parameter. Trying to scale maxusers based on physical ram in order to get mbuf cluster limits set as a side effect is just plain wrong. It makes no more sense than trying to set nmbclusters based on PRINTF_BUFR_SIZE, and then trying to scale PRINTF_BUFR_SIZE in order to get desirable second and third order side effects. Scale nmbclusters based on physical ram, with a MD method for capping it for when there are MD limits (eg: disproportionately small kva on an i386 PAE machine). Don't use maxusers. ACK -- Andre ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r242029 - head/sys/kern
On 11/8/12 1:22 AM, Garrett Cooper wrote: On Nov 8, 2012, at 1:13 AM, Alfred Perlstein wrote: Peter, I agree. It's certainly not perfect, however it's not nearly as bogus as what was there previously. I know maxusers is wrong, however what it really means, if you think about it, is give me a scaling factor that is relative to physical ram, BUT capped at some value so as to not exhaust KVA. Yes, I grok that on certain architectures that mbufs clusters aren't pulled from KVA, but, that seems much less important than how broken it is currently This fix is good enough for the general case, and a far greater improvement than what was there previously which would make FreeBSD blow chunks on any sort of 10gigE load. I think what needs to happen here, is that the people requiring perfection think about what mess it was prior and if they themselves do not have time to make it 100% perfect, allowing someone to step in and move something a step in the right direction without overly complicating it. What is there is crap, it's old, crufty and broken, it really is. It needs to be fixed, it needs to be given a nice fat band-aid now, and when someone interested in perfection comes along, then they can make it even more awesome. I am not saying that my fix is PERFECT or the be all and end all, however it serves as a good step in the right direction on our tier 1 platforms and is easily modifiable (just replace VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS with some form of MD magic sauce.) Would you like me to do that? Replace the hardline calculation with some constant that each platform can configure? I'm thinking this might suffice to make purists a bit more happy: #if defined(i386) || defined(amd64) #define MAX_KERNEL_ADDRESS_SPACE (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS) #else #define MAX_KERNEL_ADDRESS_SPACE (1024*1024*1024) Only 1GB for KVA on 64-bit platforms too...? Sure, if there's any people paying attention for these platforms they can quickly provide the necessary #define for the platform. (To make it MD as Peter suggested.) I guess I could #error until people catch up? That seems rude though. The way this works is that platforms that aren't listed get the old 384 cap, if they want more, then they can give me the #defines needed for better autotuning. Truth be told this is basically what Peter said to do, however just keeping the name maxusers. Yes maxusers is gross, but a step forward which unfortunately retains it needs to happen now. #endif Given my algorithm this should result in pretty much the same for other platforms than amd64 which will then be able to grow maxusers some. I'm basically running out of time on this and I'm worried that I'll have to back it out indefinitely so that FreeBSD can't do 10gigE out of the box. I agree with the philosophy being taken. Perfection shouldn't be the enemy of the good as long as it gets close enough that things are more useful than they are today. Thanks for championing this Alfred. The less voodoo required for FreeBSD to function on commodity hardware, the better! The less voodoo required to commit simple patches that take us forward the better as well. One shouldn't be tasked with redesigning rewiring an entire house just because they want to change one outlet. -Alfred ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r242029 - head/sys/kern
Peter, can you let me know what you think?
-Alfred
On 11/7/12 11:46 PM, Peter Wemm wrote:
On Wed, Nov 7, 2012 at 10:24 PM, Alfred Perlstein bri...@mu.org wrote:
[[ + peter ]]
Folks, I spent quite a bit of time trying to figure out how to resolve
maxusers scaling in a happy way for all.
I think I came up with a solution.
This solution should work for i386, and other 32 bit platforms, as well as
scaling well for 64 bit (or higher) platforms that have virtually unlimited
AND 64bit with limited kernel address space.
Here is how it works:
We calculate the maxusers value based on physical memory, and then clamp it
down if physical memory exceeds kernel addressable memory.
The algorithm actually remains the same for all architectures, with the
exception that machines with large kernel address space it is no longer
clamped at 384.
I've attached a test program that lets you play with various values for
VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS and physpages. (argv[1, 2, 3]
respectively.)
Please give me your feedback.
This is still bogus. VM_MIN_KERNEL_ADDRESS and VM_MAX_KERNEL_ADDRESS
have no bearing on how much space should be allocated for mbuf
clusters on amd64. If anything, you want dmapbase / dmapend if you
want a practical cap for amd64. Even then, jumbo clusters are 4K so
they come out of kva rather than direct map.
maxusers is the wrong thing for this. maxusers should, if anything,
be used to set things like kern.maxproc. Preferably it should be
deleted entirely and sysctl.conf should be used to change
kern.maxproc.
Setting limits for the mbuf / cluster pool should be a MD parameter.
Trying to scale maxusers based on physical ram in order to get mbuf
cluster limits set as a side effect is just plain wrong.
It makes no more sense than trying to set nmbclusters based on
PRINTF_BUFR_SIZE, and then trying to scale PRINTF_BUFR_SIZE in order
to get desirable second and third order side effects.
Scale nmbclusters based on physical ram, with a MD method for capping
it for when there are MD limits (eg: disproportionately small kva on
an i386 PAE machine). Don't use maxusers.
---BeginMessage---
Author: alfred
Date: Thu Nov 8 20:15:12 2012
New Revision: 242783
URL: http://svnweb.freebsd.org/changeset/base/242783
Log:
Divorce autotune nmbclusters from maxusers.
Provide arch specific override maximum.
Suggested by: peter
Modified:
user/alfred/9-alfred/sys/i386/include/vmparam.h
user/alfred/9-alfred/sys/kern/kern_mbuf.c
Modified: user/alfred/9-alfred/sys/i386/include/vmparam.h
==
--- user/alfred/9-alfred/sys/i386/include/vmparam.h Thu Nov 8 18:11:31
2012(r242782)
+++ user/alfred/9-alfred/sys/i386/include/vmparam.h Thu Nov 8 20:15:12
2012(r242783)
@@ -202,4 +202,9 @@
#defineZERO_REGION_SIZE(64 * 1024) /* 64KB */
+#ifndef MAX_AUTOTUNE_NMBCLUSTERS
+/* old maxusers max value. */
+#define MAX_AUTOTUNE_NMBCLUSTERS (1024 + 384 * 64)
+#endif
+
#endif /* _MACHINE_VMPARAM_H_ */
Modified: user/alfred/9-alfred/sys/kern/kern_mbuf.c
==
--- user/alfred/9-alfred/sys/kern/kern_mbuf.c Thu Nov 8 18:11:31 2012
(r242782)
+++ user/alfred/9-alfred/sys/kern/kern_mbuf.c Thu Nov 8 20:15:12 2012
(r242783)
@@ -102,6 +102,30 @@ int nmbjumbo9; /* limits number of 9k
int nmbjumbo16;/* limits number of 16k jumbo clusters
*/
struct mbstat mbstat;
+static int
+nmbclusters_from_physpages(void)
+{
+ long factor;
+ long rv;
+
+ factor = physmem / (2 * 1024 * 1024 / PAGE_SIZE);
+ if (factor 32)
+ factor = 32;
+ /* after 384, switch scale to 1/4 */
+ if (factor 384)
+ factor = 384 + (factor - 384) / 4;
+ rv = 1024 + factor * 64;
+ /*
+* allow a platform specific override to prevent exhausting
+* kernel memory on large memory + small address space machines.
+*/
+#ifdef MAX_AUTOTUNE_NMBCLUSTERS
+ if (rv MAX_AUTOTUNE_NMBCLUSTERS)
+ rv = MAX_AUTOTUNE_NMBCLUSTERS
+#endif
+ return (rv);
+}
+
/*
* tunable_mbinit() has to be run before init_maxsockets() thus
* the SYSINIT order below is SI_ORDER_MIDDLE while init_maxsockets()
@@ -114,7 +138,7 @@ tunable_mbinit(void *dummy)
/* This has to be done before VM init. */
TUNABLE_INT_FETCH(kern.ipc.nmbclusters, nmbclusters);
if (nmbclusters == 0)
- nmbclusters = 1024 + maxusers * 64;
+ nmbclusters = nmbclusters_from_physpages();
TUNABLE_INT_FETCH(kern.ipc.nmbjumbop, nmbjumbop);
if (nmbjumbop == 0)
---End Message---
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to
Re: svn commit: r242029 - head/sys/kern
[[ + peter ]]
Folks, I spent quite a bit of time trying to figure out how to resolve
maxusers scaling in a happy way for all.
I think I came up with a solution.
This solution should work for i386, and other 32 bit platforms, as well
as scaling well for 64 bit (or higher) platforms that have virtually
unlimited AND 64bit with limited kernel address space.
Here is how it works:
We calculate the maxusers value based on physical memory, and then clamp
it down if physical memory exceeds kernel addressable memory.
The algorithm actually remains the same for all architectures, with the
exception that machines with large kernel address space it is no longer
clamped at 384.
I've attached a test program that lets you play with various values for
VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS and physpages. (argv[1, 2,
3] respectively.)
Please give me your feedback.
Note a lot of the ugly types/casting is due to building this test
program on i386 darwin. I will clean it up before committing.
I just want to make sure this the right direction.
I would like to commit this shortly.
-Alfred
On 10/25/12 6:50 AM, John Baldwin wrote:
On Thursday, October 25, 2012 4:05:51 am Konstantin Belousov wrote:
On Thu, Oct 25, 2012 at 01:46:21AM +, Alfred Perlstein wrote:
Author: alfred
Date: Thu Oct 25 01:46:20 2012
New Revision: 242029
URL: http://svn.freebsd.org/changeset/base/242029
Log:
Allow autotune maxusers 384 on 64 bit machines
A default install on large memory machines with multiple 10gigE interfaces
were not being given enough mbufs to do full bandwidth TCP or NFS traffic.
To keep the value somewhat reasonable, we scale back the number of
maxuers by 1/6 past the 384 point. This gives us enough mbufs for most
of our pretty basic 10gigE line-speed tests to complete.
Modified:
head/sys/kern/subr_param.c
Modified: head/sys/kern/subr_param.c
==
--- head/sys/kern/subr_param.c Thu Oct 25 01:27:01 2012(r242028)
+++ head/sys/kern/subr_param.c Thu Oct 25 01:46:20 2012(r242029)
@@ -278,8 +278,16 @@ init_param2(long physpages)
maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
if (maxusers 32)
maxusers = 32;
- if (maxusers 384)
- maxusers = 384;
+ /*
+* Clips maxusers to 384 on machines with = 4GB RAM or 32bit.
+* Scales it down 6x for large memory machines.
+*/
+ if (maxusers 384) {
+ if (sizeof(void *) = 4)
+ maxusers = 384;
+ else
+ maxusers = 384 + ((maxusers - 384) / 6);
+ }
This is unbelievably weird way to express the '64bit'. The
#ifdef _LP64 is enough there instead of the runtime check.
Also, are you sure that all our 64bit arches do not have KVA limitations ?
There is an active thread on current@ with a different patch that uses a KVA
constant to derive 384 instead. When we have updated tuning in the past
(e.g. adjusting the cap on maxvnodes), there was a bit of discussion rather
than a random drive by commit. I think we should probably hold off on making
changes here and figure out what the right way to fix this is in the thread on
current@ instead. Andre has already suggested divorcing mbuf tuning from
maxusers entirely for example.
#include stdio.h
#include stdlib.h
#include sys/types.h
#ifndef __FreeBSD__
#include machine/types.h
#endif
#define VM_MIN_KERNEL_ADDRESS atoull(argv[1])
#define VM_MAX_KERNEL_ADDRESS atoull(argv[2])
#define physpages atol(argv[3])
#define PAGE_SIZE (long long)4096
int maxusers;
unsigned long long
atoull(const char *str)
{
unsigned long long ret;
if (sscanf(str, %llu, ret) != 1) {
fprintf(stderr, sscanf: error\n);
exit(1);
}
return ret;
}
int
main(int argc, char **argv) {
u_int64_t maxkvapages;
long maxusers_kva;
long maxusers_phys;
printf(min: %llu, max: %llu\n, VM_MIN_KERNEL_ADDRESS,
VM_MAX_KERNEL_ADDRESS);
maxkvapages = (VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS)
/ (long long)PAGE_SIZE;
printf(VM_ADDRESS_SPACE: %llu\n, (long long)((long
long)VM_MAX_KERNEL_ADDRESS - (long long)VM_MIN_KERNEL_ADDRESS));
printf(maxkvapages: %ld\n, (long)maxkvapages);
maxusers_kva = maxkvapages * 3 / 4 / 512;
printf(maxusers_kva: %ld\n, maxusers_kva);
maxusers_phys = physpages / 512;
printf(maxusers_phys: %ld\n, maxusers_phys);
if (maxusers_phys maxusers_kva) {
maxusers = maxusers_kva;
} else {
Re: svn commit: r242029 - head/sys/kern
On Wed, Nov 7, 2012 at 10:24 PM, Alfred Perlstein bri...@mu.org wrote: [[ + peter ]] Folks, I spent quite a bit of time trying to figure out how to resolve maxusers scaling in a happy way for all. I think I came up with a solution. This solution should work for i386, and other 32 bit platforms, as well as scaling well for 64 bit (or higher) platforms that have virtually unlimited AND 64bit with limited kernel address space. Here is how it works: We calculate the maxusers value based on physical memory, and then clamp it down if physical memory exceeds kernel addressable memory. The algorithm actually remains the same for all architectures, with the exception that machines with large kernel address space it is no longer clamped at 384. I've attached a test program that lets you play with various values for VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS and physpages. (argv[1, 2, 3] respectively.) Please give me your feedback. This is still bogus. VM_MIN_KERNEL_ADDRESS and VM_MAX_KERNEL_ADDRESS have no bearing on how much space should be allocated for mbuf clusters on amd64. If anything, you want dmapbase / dmapend if you want a practical cap for amd64. Even then, jumbo clusters are 4K so they come out of kva rather than direct map. maxusers is the wrong thing for this. maxusers should, if anything, be used to set things like kern.maxproc. Preferably it should be deleted entirely and sysctl.conf should be used to change kern.maxproc. Setting limits for the mbuf / cluster pool should be a MD parameter. Trying to scale maxusers based on physical ram in order to get mbuf cluster limits set as a side effect is just plain wrong. It makes no more sense than trying to set nmbclusters based on PRINTF_BUFR_SIZE, and then trying to scale PRINTF_BUFR_SIZE in order to get desirable second and third order side effects. Scale nmbclusters based on physical ram, with a MD method for capping it for when there are MD limits (eg: disproportionately small kva on an i386 PAE machine). Don't use maxusers. -- Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV All of this is for nothing if we don't go to the stars - JMS/B5 If Java had true garbage collection, most programs would delete themselves upon execution. -- Robert Sewell ___ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r242029 - head/sys/kern
On Thu, Oct 25, 2012 at 01:46:21AM +, Alfred Perlstein wrote:
Author: alfred
Date: Thu Oct 25 01:46:20 2012
New Revision: 242029
URL: http://svn.freebsd.org/changeset/base/242029
Log:
Allow autotune maxusers 384 on 64 bit machines
A default install on large memory machines with multiple 10gigE interfaces
were not being given enough mbufs to do full bandwidth TCP or NFS traffic.
To keep the value somewhat reasonable, we scale back the number of
maxuers by 1/6 past the 384 point. This gives us enough mbufs for most
of our pretty basic 10gigE line-speed tests to complete.
Modified:
head/sys/kern/subr_param.c
Modified: head/sys/kern/subr_param.c
==
--- head/sys/kern/subr_param.cThu Oct 25 01:27:01 2012
(r242028)
+++ head/sys/kern/subr_param.cThu Oct 25 01:46:20 2012
(r242029)
@@ -278,8 +278,16 @@ init_param2(long physpages)
maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
if (maxusers 32)
maxusers = 32;
- if (maxusers 384)
- maxusers = 384;
+ /*
+ * Clips maxusers to 384 on machines with = 4GB RAM or 32bit.
+ * Scales it down 6x for large memory machines.
+ */
+ if (maxusers 384) {
+ if (sizeof(void *) = 4)
+ maxusers = 384;
+ else
+ maxusers = 384 + ((maxusers - 384) / 6);
+ }
This is unbelievably weird way to express the '64bit'. The
#ifdef _LP64 is enough there instead of the runtime check.
Also, are you sure that all our 64bit arches do not have KVA limitations ?
pgpKF20hTnNOJ.pgp
Description: PGP signature
Re: svn commit: r242029 - head/sys/kern
On Wed, Oct 24, 2012 at 6:46 PM, Alfred Perlstein alf...@freebsd.org wrote:
Author: alfred
Date: Thu Oct 25 01:46:20 2012
New Revision: 242029
URL: http://svn.freebsd.org/changeset/base/242029
Log:
Allow autotune maxusers 384 on 64 bit machines
A default install on large memory machines with multiple 10gigE interfaces
were not being given enough mbufs to do full bandwidth TCP or NFS traffic.
To keep the value somewhat reasonable, we scale back the number of
maxuers by 1/6 past the 384 point. This gives us enough mbufs for most
of our pretty basic 10gigE line-speed tests to complete.
Modified:
head/sys/kern/subr_param.c
Modified: head/sys/kern/subr_param.c
==
--- head/sys/kern/subr_param.c Thu Oct 25 01:27:01 2012(r242028)
+++ head/sys/kern/subr_param.c Thu Oct 25 01:46:20 2012(r242029)
@@ -278,8 +278,16 @@ init_param2(long physpages)
maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
if (maxusers 32)
maxusers = 32;
- if (maxusers 384)
- maxusers = 384;
+ /*
+* Clips maxusers to 384 on machines with = 4GB RAM or 32bit.
+* Scales it down 6x for large memory machines.
+*/
+ if (maxusers 384) {
+ if (sizeof(void *) = 4)
+ maxusers = 384;
+ else
+ maxusers = 384 + ((maxusers - 384) / 6);
Why `/ 6` (other than the fact that it makes the value presumably
a multiple of 64)? Also, shouldn't some kind of clamping be taking
place to ensure that this the end result of the calculation is a
multiple of a power of two, e.g. 16, 32, 64, etc?
And as usual, got ministat :)?
Thanks!
-Garrett
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r242029 - head/sys/kern
On Thu, 25 Oct 2012, Konstantin Belousov wrote:
On Thu, Oct 25, 2012 at 01:46:21AM +, Alfred Perlstein wrote:
...
Modified: head/sys/kern/subr_param.c
==
--- head/sys/kern/subr_param.c Thu Oct 25 01:27:01 2012(r242028)
+++ head/sys/kern/subr_param.c Thu Oct 25 01:46:20 2012(r242029)
@@ -278,8 +278,16 @@ init_param2(long physpages)
maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
if (maxusers 32)
maxusers = 32;
- if (maxusers 384)
- maxusers = 384;
+ /*
+* Clips maxusers to 384 on machines with = 4GB RAM or 32bit.
+* Scales it down 6x for large memory machines.
+*/
+ if (maxusers 384) {
+ if (sizeof(void *) = 4)
+ maxusers = 384;
+ else
+ maxusers = 384 + ((maxusers - 384) / 6);
+ }
This is unbelievably weird way to express the '64bit'. The
#ifdef _LP64 is enough there instead of the runtime check.
Such runtime checks are well optimized by compilers, giving nicer code than
ifdefs.
However, there are lots of other style bugs:
- comments writtens not in English is
- after deciphering the comments, we see that they match the code only in
the unbelievably weird ways:
(1) = 4GB RAM isn't necessarily related to the size of void *.
In fact, the size of physical memory certainly isn't related. The
size of virtual memory is more closely related. But it is the
physical memory size that is most relevant here.
(2) 32 bitness isn't necessaril related to the size of void *.
- after fixing the comments, they would just echo the code, and thus
shouldn't be made. No comments are made for the other maxusers
initializations, although it would be easy to write ones 10 times
longer than this mail to describe all the historical bogus tuning
given by maxusers.
- half a level of indentation for the maxusers lines
- excessive parentheses.
Bruce
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
Re: svn commit: r242029 - head/sys/kern
On Thursday, October 25, 2012 4:05:51 am Konstantin Belousov wrote:
On Thu, Oct 25, 2012 at 01:46:21AM +, Alfred Perlstein wrote:
Author: alfred
Date: Thu Oct 25 01:46:20 2012
New Revision: 242029
URL: http://svn.freebsd.org/changeset/base/242029
Log:
Allow autotune maxusers 384 on 64 bit machines
A default install on large memory machines with multiple 10gigE interfaces
were not being given enough mbufs to do full bandwidth TCP or NFS traffic.
To keep the value somewhat reasonable, we scale back the number of
maxuers by 1/6 past the 384 point. This gives us enough mbufs for most
of our pretty basic 10gigE line-speed tests to complete.
Modified:
head/sys/kern/subr_param.c
Modified: head/sys/kern/subr_param.c
==
--- head/sys/kern/subr_param.c Thu Oct 25 01:27:01 2012
(r242028)
+++ head/sys/kern/subr_param.c Thu Oct 25 01:46:20 2012
(r242029)
@@ -278,8 +278,16 @@ init_param2(long physpages)
maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
if (maxusers 32)
maxusers = 32;
- if (maxusers 384)
- maxusers = 384;
+ /*
+* Clips maxusers to 384 on machines with = 4GB RAM or 32bit.
+* Scales it down 6x for large memory machines.
+*/
+ if (maxusers 384) {
+ if (sizeof(void *) = 4)
+ maxusers = 384;
+ else
+ maxusers = 384 + ((maxusers - 384) / 6);
+ }
This is unbelievably weird way to express the '64bit'. The
#ifdef _LP64 is enough there instead of the runtime check.
Also, are you sure that all our 64bit arches do not have KVA limitations ?
There is an active thread on current@ with a different patch that uses a KVA
constant to derive 384 instead. When we have updated tuning in the past
(e.g. adjusting the cap on maxvnodes), there was a bit of discussion rather
than a random drive by commit. I think we should probably hold off on making
changes here and figure out what the right way to fix this is in the thread on
current@ instead. Andre has already suggested divorcing mbuf tuning from
maxusers entirely for example.
--
John Baldwin
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
svn commit: r242029 - head/sys/kern
Author: alfred
Date: Thu Oct 25 01:46:20 2012
New Revision: 242029
URL: http://svn.freebsd.org/changeset/base/242029
Log:
Allow autotune maxusers 384 on 64 bit machines
A default install on large memory machines with multiple 10gigE interfaces
were not being given enough mbufs to do full bandwidth TCP or NFS traffic.
To keep the value somewhat reasonable, we scale back the number of
maxuers by 1/6 past the 384 point. This gives us enough mbufs for most
of our pretty basic 10gigE line-speed tests to complete.
Modified:
head/sys/kern/subr_param.c
Modified: head/sys/kern/subr_param.c
==
--- head/sys/kern/subr_param.c Thu Oct 25 01:27:01 2012(r242028)
+++ head/sys/kern/subr_param.c Thu Oct 25 01:46:20 2012(r242029)
@@ -278,8 +278,16 @@ init_param2(long physpages)
maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE);
if (maxusers 32)
maxusers = 32;
- if (maxusers 384)
- maxusers = 384;
+ /*
+* Clips maxusers to 384 on machines with = 4GB RAM or 32bit.
+* Scales it down 6x for large memory machines.
+*/
+ if (maxusers 384) {
+ if (sizeof(void *) = 4)
+ maxusers = 384;
+ else
+ maxusers = 384 + ((maxusers - 384) / 6);
+ }
}
/*
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to svn-src-all-unsubscr...@freebsd.org
