Re: FW: Filesystem gets a huge performance boost
Doug Barton wrote:
Alfred Perlstein wrote:
I'm figuring the only time when it may be a problem is on machines
with a small amount of memory. Since memory is cheap, I plan on
turning it on within the next couple of days unless a stability
issue comes up.
I'll leave it to those people with low memory to remember to turn
it off.
OK... this brings up the question of what other cool optimizations are
there that may have been disabled in the past for reasons that are no
longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file
with commented out examples of configurations for various systems. For
example,
# For more modern systems that have a reasonable amount of RAM
#vfs.vmiodirenable=1
# Low memory systems
# Systems that need lots of randomness
# Low resource systems that need less randomness
# Super high performance TCP options for various situations
etc. I'm sure y'all can come up with more.
It might also be desirable to put these in etc/defautls/rc.conf, but I
think something of this nature might be better suited in a freer format.
What do you think about attached patch?
-Maxim
Index: Makefile
===
RCS file: /home/ncvs/src/etc/Makefile,v
retrieving revision 1.248
diff -d -u -r1.248 Makefile
--- Makefile2001/03/29 14:02:59 1.248
+++ Makefile2001/04/19 12:44:09
@@ -15,8 +15,8 @@
printcap profile protocols \
rc rc.atm rc.devfs rc.diskless1 rc.diskless2 rc.firewall rc.firewall6 \
rc.isdn rc.network rc.network6 rc.pccard rc.serial rc.shutdown \
- rc.syscons rc.sysctl remote rpc security services shells syslog.conf \
- usbd.conf \
+ rc.syscons rc.sysctl remote rpc security services shells sysctl.conf \
+ syslog.conf usbd.conf \
etc.${MACHINE_ARCH}/disktab \
etc.${MACHINE_ARCH}/rc.${MACHINE_ARCH} \
etc.${MACHINE_ARCH}/ttys \
Index: sysctl.conf
===
RCS file: /home/ncvs/src/etc/sysctl.conf,v
retrieving revision 1.1
diff -d -u -r1.1 sysctl.conf
--- sysctl.conf 2000/07/27 22:53:42 1.1
+++ sysctl.conf 2001/04/19 12:44:09
@@ -3,3 +3,6 @@
# This file is read when going to multi-user and its contents piped thru
# ``sysctl -w'' to adjust kernel values. ``man 5 sysctl.conf'' for details.
#
+
+vfs.vmiodirenable=0# Set to 1 to enable a new directory allocation policy
+ # (codenamed "dirpref")
Re: FW: Filesystem gets a huge performance boost
Alfred Perlstein wrote:
* Maxim Sobolev [EMAIL PROTECTED] [010419 05:48] wrote:
Doug Barton wrote:
Alfred Perlstein wrote:
I'm figuring the only time when it may be a problem is on machines
with a small amount of memory. Since memory is cheap, I plan on
turning it on within the next couple of days unless a stability
issue comes up.
I'll leave it to those people with low memory to remember to turn
it off.
OK... this brings up the question of what other cool optimizations are
there that may have been disabled in the past for reasons that are no
longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file
with commented out examples of configurations for various systems. For
example,
# For more modern systems that have a reasonable amount of RAM
#vfs.vmiodirenable=1
# Low memory systems
# Systems that need lots of randomness
# Low resource systems that need less randomness
# Super high performance TCP options for various situations
etc. I'm sure y'all can come up with more.
It might also be desirable to put these in etc/defautls/rc.conf, but I
think something of this nature might be better suited in a freer format.
What do you think about attached patch?
I think it's a step in the right direction, however
OOPS, I see. See updated patch.
-Maxim
Index: Makefile
===
RCS file: /home/ncvs/src/etc/Makefile,v
retrieving revision 1.248
diff -d -u -r1.248 Makefile
--- Makefile2001/03/29 14:02:59 1.248
+++ Makefile2001/04/19 13:18:01
@@ -15,8 +15,8 @@
printcap profile protocols \
rc rc.atm rc.devfs rc.diskless1 rc.diskless2 rc.firewall rc.firewall6 \
rc.isdn rc.network rc.network6 rc.pccard rc.serial rc.shutdown \
- rc.syscons rc.sysctl remote rpc security services shells syslog.conf \
- usbd.conf \
+ rc.syscons rc.sysctl remote rpc security services shells sysctl.conf \
+ syslog.conf usbd.conf \
etc.${MACHINE_ARCH}/disktab \
etc.${MACHINE_ARCH}/rc.${MACHINE_ARCH} \
etc.${MACHINE_ARCH}/ttys \
Index: sysctl.conf
===
RCS file: /home/ncvs/src/etc/sysctl.conf,v
retrieving revision 1.1
diff -d -u -r1.1 sysctl.conf
--- sysctl.conf 2000/07/27 22:53:42 1.1
+++ sysctl.conf 2001/04/19 13:18:01
@@ -3,3 +3,10 @@
# This file is read when going to multi-user and its contents piped thru
# ``sysctl -w'' to adjust kernel values. ``man 5 sysctl.conf'' for details.
#
+
+vfs.vmiodirenable=0# Set to 1 to enable the use of the VM subsystem to
+ # back UFS directory memory requirements. Because of
+ # the amount of wasted memory this causes it's not
+ # advised for machines with less than 64MB of RAM, but
+ # on machines with more than 64MB it can provide a
+ # substantial benefit related to directory caching
Re: FW: Filesystem gets a huge performance boost
* Maxim Sobolev [EMAIL PROTECTED] [010419 06:20] wrote:
OOPS, I see. See updated patch.
Looks ok.
Index: Makefile
===
RCS file: /home/ncvs/src/etc/Makefile,v
retrieving revision 1.248
diff -d -u -r1.248 Makefile
--- Makefile 2001/03/29 14:02:59 1.248
+++ Makefile 2001/04/19 13:18:01
@@ -15,8 +15,8 @@
printcap profile protocols \
rc rc.atm rc.devfs rc.diskless1 rc.diskless2 rc.firewall rc.firewall6 \
rc.isdn rc.network rc.network6 rc.pccard rc.serial rc.shutdown \
- rc.syscons rc.sysctl remote rpc security services shells syslog.conf \
- usbd.conf \
+ rc.syscons rc.sysctl remote rpc security services shells sysctl.conf \
+ syslog.conf usbd.conf \
etc.${MACHINE_ARCH}/disktab \
etc.${MACHINE_ARCH}/rc.${MACHINE_ARCH} \
etc.${MACHINE_ARCH}/ttys \
Index: sysctl.conf
===
RCS file: /home/ncvs/src/etc/sysctl.conf,v
retrieving revision 1.1
diff -d -u -r1.1 sysctl.conf
--- sysctl.conf 2000/07/27 22:53:42 1.1
+++ sysctl.conf 2001/04/19 13:18:01
@@ -3,3 +3,10 @@
# This file is read when going to multi-user and its contents piped thru
# ``sysctl -w'' to adjust kernel values. ``man 5 sysctl.conf'' for details.
#
+
+vfs.vmiodirenable=0 # Set to 1 to enable the use of the VM subsystem to
+ # back UFS directory memory requirements. Because of
+ # the amount of wasted memory this causes it's not
+ # advised for machines with less than 64MB of RAM, but
+ # on machines with more than 64MB it can provide a
+ # substantial benefit related to directory caching
--
-Alfred Perlstein - [[EMAIL PROTECTED]]
Represent yourself, show up at BABUG http://www.babug.org/
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Thu, Apr 19, 2001 at 03:46:39PM +0300, Maxim Sobolev wrote:
What do you think about attached patch?
-Maxim
Index: Makefile
===
RCS file: /home/ncvs/src/etc/Makefile,v
retrieving revision 1.248
diff -d -u -r1.248 Makefile
--- Makefile 2001/03/29 14:02:59 1.248
+++ Makefile 2001/04/19 12:44:09
@@ -15,8 +15,8 @@
printcap profile protocols \
rc rc.atm rc.devfs rc.diskless1 rc.diskless2 rc.firewall rc.firewall6 \
rc.isdn rc.network rc.network6 rc.pccard rc.serial rc.shutdown \
- rc.syscons rc.sysctl remote rpc security services shells syslog.conf \
- usbd.conf \
+ rc.syscons rc.sysctl remote rpc security services shells sysctl.conf \
+ syslog.conf usbd.conf \
etc.${MACHINE_ARCH}/disktab \
etc.${MACHINE_ARCH}/rc.${MACHINE_ARCH} \
etc.${MACHINE_ARCH}/ttys \
Index: sysctl.conf
===
RCS file: /home/ncvs/src/etc/sysctl.conf,v
retrieving revision 1.1
diff -d -u -r1.1 sysctl.conf
--- sysctl.conf 2000/07/27 22:53:42 1.1
+++ sysctl.conf 2001/04/19 12:44:09
@@ -3,3 +3,6 @@
# This file is read when going to multi-user and its contents piped thru
# ``sysctl -w'' to adjust kernel values. ``man 5 sysctl.conf'' for details.
#
+
+vfs.vmiodirenable=0 # Set to 1 to enable a new directory allocation policy
+ # (codenamed "dirpref")
Oops, the comment is wrong. Dirpref is totally another thing which has
nothing to do with that sysctl.
--
Alex Kapranoff, Voice: +7(0832)791845
We've lived 2608 hours in the brand new millenium...
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Maxim Sobolev wrote: What do you think about attached patch? Definitely the right idea, however I'm waiting on input from a couple people on some additional suggestions, so if you'd hold off I'd appreciate it. -- "One thing they don't tell you about doing experimental physics is that sometimes you must work under adverse conditions ... like a state of sheer terror." -- W. K. Hartmann Do YOU Yahoo!? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Doug Barton wrote: Maxim Sobolev wrote: What do you think about attached patch? Definitely the right idea, however I'm waiting on input from a couple people on some additional suggestions, so if you'd hold off I'd appreciate it. Unfortunately I've already cvs ci it. :( Hovewer, it is a cvs, so it should not be big deal, really. -Maxim To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
: :What do you think about attached patch? : :-Maxim mmm.. I think it would just confuse the issue and prevent us from being able to change the kernel default trivially. 99.5% of the FreeBSD boxes out there are just going to want it to be on by default. We could provide a commented-out rc.conf variable and support in rc to allow the sysadmin to set it, but we should not provide a default in the rc system or try to create yet another configuration file. The issue doesn't require that level of sophistication. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Matt Dillon wrote: : :What do you think about attached patch? : :-Maxim mmm.. I think it would just confuse the issue and prevent us from being able to change the kernel default trivially. 99.5% of the FreeBSD boxes out there are just going to want it to be on by default. Point taken and issues fixed, thank you! We could provide a commented-out rc.conf variable and support in rc to allow the sysadmin to set it, but we should not provide a default in the rc system or try to create yet another configuration file. The issue doesn't require that level of sophistication. But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's wrong with putting some useful payload into it? -Maxim To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
:But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim If it's commented out, it's fine. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
:But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim Let me explain a little more. If it's commented out, it's fine. But if you are actually setting a value in there you will override whatever is set in the kernel. When people install new systems they do not usually override their /etc/ files, so an uncommented value will set that default in stone and prevent us from being able to change it with a new kernel rev. This being a *kernel* specific feature, we need to have control over the default in the kernel itself. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Thu, Apr 19, 2001 at 10:39:58 -0700, Matt Dillon wrote: :But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim Let me explain a little more. If it's commented out, it's fine. But if you are actually setting a value in there you will override whatever is set in the kernel. When people install new systems they do not usually override their /etc/ files, so an uncommented value will set that default in stone and prevent us from being able to change it with a new kernel rev. This being a *kernel* specific feature, we need to have control over the default in the kernel itself. What about simple check in the kernel: if total memory is above 64Mb, then enable this mode by default, else disable it. -- Andrey A. Chernov http://ache.pp.ru/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
: set that default in stone and prevent us from being able to change : it with a new kernel rev. This being a *kernel* specific feature, : we need to have control over the default in the kernel itself. : :What about simple check in the kernel: if total memory is above 64Mb, then :enable this mode by default, else disable it. : :-- :Andrey A. Chernov You are assuming that turning on vmiodirenable is detrimental on a low-memory machine. I don't think it is, because on a low memory machine you aren't going to be able to cache very much anyway. But if anyone thinks it is please be my guest and test it on a low-memory machine on verses off. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Thu, Apr 19, 2001 at 10:47:20 -0700, Matt Dillon wrote: : set that default in stone and prevent us from being able to change : it with a new kernel rev. This being a *kernel* specific feature, : we need to have control over the default in the kernel itself. : :What about simple check in the kernel: if total memory is above 64Mb, then :enable this mode by default, else disable it. : :-- :Andrey A. Chernov You are assuming that turning on vmiodirenable is detrimental on a low-memory machine. I don't think it is, because on a low memory machine you aren't going to be able to cache very much anyway. But if anyone thinks it is please be my guest and test it on a low-memory machine on verses off. Personally I not test this thing, just assume that current etc/sysctl.conf comment about 64Mb limit is correct, i.e. assume that sysctl.conf commiter already test it. -- Andrey A. Chernov http://ache.pp.ru/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Thu, 19 Apr 2001, Andrey A. Chernov wrote: On Thu, Apr 19, 2001 at 10:39:58 -0700, Matt Dillon wrote: Let me explain a little more. If it's commented out, it's fine. But if you are actually setting a value in there you will override whatever is set in the kernel. When people install new systems they do not usually override their /etc/ files, so an uncommented value will set that default in stone and prevent us from being able to change it with a new kernel rev. This being a *kernel* specific feature, we need to have control over the default in the kernel itself. What about simple check in the kernel: if total memory is above 64Mb, then enable this mode by default, else disable it. I believe it should be enabled at more like 640MB than 64MB. It slows down makeworld (immediately after rebooting to get a consistent enviroment) a little here with 256MB. Whether it is good depends more on the application mix than the memory size. Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
-On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: Testing it 'on' in stable on production systems and observing the relative change in performance is a worthy experiment. Testing it 'on' in current is just an experiment. I have been running vfs.vmiodirenable=1 on two STABLE boxes for the last week or so. Still no problems. Been doing massive cvsups and all that. This is not in combination with softupdates. That's next on the agenda. I think Dan Langille enabled it on a cvsupd server he has set up after I mentioned this sysctl to him. Dan? -- Jeroen Ruigrok van der Werven/Asmodai .oUo. asmodai@[wxs.nl|freebsd.org] Documentation nutter/C-rated Coder BSD: Technical excellence at its best D78D D0AD 244D 1D12 C9CA 7152 035C 1138 546A B867 We decide who's crazy or not... To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
-On [20010418 01:00], Alfred Perlstein ([EMAIL PROTECTED]) wrote: (although afaik we're basing it on both Solaris and BSD/os's implementation so... well I'm not going to bother defending it.) You just scared the shit out of me by mentioning Solaris. I've found Solaris to be a PITA with all new subsequent releases. Surely we're not aiming towards their goals where we can only run on state of the art hardware? -- Jeroen Ruigrok van der Werven/Asmodai .oUo. asmodai@[wxs.nl|freebsd.org] Documentation nutter/C-rated Coder BSD: Technical excellence at its best D78D D0AD 244D 1D12 C9CA 7152 035C 1138 546A B867 Expansion of happiness is the purpose of life... To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Wed, 18 Apr 2001, Jeroen Ruigrok/Asmodai wrote: -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: Testing it 'on' in stable on production systems and observing the relative change in performance is a worthy experiment. Testing it 'on' in current is just an experiment. I have been running vfs.vmiodirenable=1 on two STABLE boxes for the last week or so. Still no problems. Been doing massive cvsups and all that. This is not in combination with softupdates. That's next on the agenda. So, how much slower was it? ;-) Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
-On [20010418 14:38], Bruce Evans ([EMAIL PROTECTED]) wrote: [vfs.vmiodirenable] So, how much slower was it? ;-) Not noticeable for me at least. -- Jeroen Ruigrok van der Werven/Asmodai --=-- asmodai@[wxs.nl|freebsd.org] Documentation nutter/C-rated Coder BSD: Technical excellence at its best http://www.freebsd.org/doc/en_US.ISO_8859-1/books/developers-handbook/ Cogito, ergo sum... To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On 18 Apr 2001, at 22:16, Bruce Evans wrote: On Wed, 18 Apr 2001, Jeroen Ruigrok/Asmodai wrote: -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: Testing it 'on' in stable on production systems and observing the relative change in performance is a worthy experiment. Testing it 'on' in current is just an experiment. I have been running vfs.vmiodirenable=1 on two STABLE boxes for the last week or so. Still no problems. Been doing massive cvsups and all that. This is not in combination with softupdates. That's next on the agenda. So, how much slower was it? ;-) $ uname -a FreeBSD cvsup.nz.freebsd.org 4.2-STABLE FreeBSD 4.2-STABLE #0: Thu Mar 8 01:24:24 NZDT 2001 [EMAIL PROTECTED]:/usr/src/sys/compile/ZEKE i386 The box I mentioned in my previous message is running softupdates. -- Dan Langille pgpkey - finger [EMAIL PROTECTED] | http://unixathome.org/finger.php To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Tue, 17 Apr 2001, Doug Barton wrote: OK... this brings up the question of what other cool optimizations are there that may have been disabled in the past for reasons that are no longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file with commented out examples of configurations for various systems. For example, # For more modern systems that have a reasonable amount of RAM #vfs.vmiodirenable=1 # Low memory systems # Systems that need lots of randomness # Low resource systems that need less randomness # Super high performance TCP options for various situations I smell a book. I was going to write it but I just don't have the time or knowledge. :) Doug White| FreeBSD: The Power to Serve [EMAIL PROTECTED] | www.FreeBSD.org To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Matt Dillon wrote: It is not implying that at all. There is no black and white here. This is a case where spending a huge amount of time and complexity to get the efficiency down to the Nth degree is nothing but a waste of time. What matters is what the user sees, what performance the application gets, and how many bugs you introduce when optimizing something that might not need optimizing. Sooo. sounds like making "on" the default in -current is a worthy experiment? Doug -- Perhaps the greatest damage the American system of education has done to its children is to teach them that their opinions are relevant simply because they are their opinions. Do YOU Yahoo!? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
* Doug Barton [EMAIL PROTECTED] [010417 01:08] wrote: Matt Dillon wrote: It is not implying that at all. There is no black and white here. This is a case where spending a huge amount of time and complexity to get the efficiency down to the Nth degree is nothing but a waste of time. What matters is what the user sees, what performance the application gets, and how many bugs you introduce when optimizing something that might not need optimizing. Sooo. sounds like making "on" the default in -current is a worthy experiment? More like I'm damn tired of having to remeber to turn it on all my machines in order to get decent performance when doing large amount of directory ops. I'm figuring the only time when it may be a problem is on machines with a small amount of memory. Since memory is cheap, I plan on turning it on within the next couple of days unless a stability issue comes up. I'll leave it to those people with low memory to remember to turn it off. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] Daemon News Magazine in your snail-mail! http://magazine.daemonnews.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
* Matt Dillon [EMAIL PROTECTED] [010415 23:16] wrote: For example, all this work on a preemptive kernel is just insane. Our entire kernel is built on the concept of not being preemptable except by interrupts. We virtually guarentee years of instability and bugs leaking out of the woodwork by trying to make it preemptable, and the performance gain we get for that pain is going to be zilch. Nada. Nothing. Pre-emption is mearly a side effect of a mutex'd kernel. The actual gains are in terms of parallel execution internally. Meaning if we happen to copyin() a 4 meg buffer we can allow more than one process to be completing some sort of work inside the kernel other than spinning on the giant lock. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Alfred Perlstein wrote: I'm figuring the only time when it may be a problem is on machines with a small amount of memory. Since memory is cheap, I plan on turning it on within the next couple of days unless a stability issue comes up. I'll leave it to those people with low memory to remember to turn it off. OK... this brings up the question of what other cool optimizations are there that may have been disabled in the past for reasons that are no longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file with commented out examples of configurations for various systems. For example, # For more modern systems that have a reasonable amount of RAM #vfs.vmiodirenable=1 # Low memory systems # Systems that need lots of randomness # Low resource systems that need less randomness # Super high performance TCP options for various situations etc. I'm sure y'all can come up with more. It might also be desirable to put these in etc/defautls/rc.conf, but I think something of this nature might be better suited in a freer format. Doug -- Perhaps the greatest damage the American system of education has done to its children is to teach them that their opinions are relevant simply because they are their opinions. Do YOU Yahoo!? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
: :There's actually very little code that non-premptable once we get the :kernel mutexed. The least complex way to accomplish this is to only :preempt kernel processes that hold no mutex (low level) locks. : :-- :-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] I wish it were that simple. Remember when we went to shared file descriptor arrays for processes? Half a dozen serious bugs were introduced into the kernel from that change which weren't fixed for over a year and half. And that was *WITH* a Giant lock left intact. A change to a single subsystem introduced 6+ serious bugs. Now you guys are introducing preemptive kernel scheduling and expect it will be a walk in the park? I expect it will introduce so many bugs that the system will be destabilized for over a year, because that sort of thing effects every single subsystem rather then just one or two. It is not going to be as simple as throwing mutexes around every little piece of code that might access shared state somewhere. How many mutex calls are we going to wind up making to implement the simplest system call now? How much overhead are we going to introduce to support a feature that isn't going to add one iota of performance to the system? An incredible number of very cool optimizations can be made when you have the ability to assume that the cpu you are running under will not get ripped out from under you. Things like per-cpu caching - not only of mbufs or memory, but of other things as well. Simple access to per-cpu globals now must suddenly be done with compare-exchange or otherwise be required to use atomic instructions. None of the current per-cpu variable handling code assumes that. It's just not a good idea. Do you want people to adopt -current when it becomes 5.0? It isn't going to happen if preemptive kernel scheduling is in there. I am going to wind up having to support 4.x for the next two years before 5.x becomes reliable enough to use in a production environment. Or, worse, -current is going to move down a path of no-return (becoming so unreliable that it will not be adopted by anyone) and we are going to wind up having to scrap the whole damn thing! I was somewhat skeptical about BSDI's interrupt-as-thread methodology, but at least they didn't bite of more then they could chew when they implemented it. Trying to extend that model to the Nth degree is creating a huge mess -- and for what? Just to support interrupts as threads? It isn't worth it. I would much rather see us changing the interrupt model to be less invasive, thus allowing interrupts to run unhindered as interrupts rather then threads, rather then see this preemptive kernel scheduling stuff go in. I love SMP, but I love stability more. BSDI focused on the things that actually effected performance. -current is not focused at all and is going off the deep end.. trying to do things that would be interesting work for 5.2 or 5.3 but just plain should not be in 5.0. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
You need to settle dude, pre-emption isn't a goal, it's mearly a _possible_ side effect. We're not aiming for pre-emption, we're aiming for more concurrancy. * Matt Dillon [EMAIL PROTECTED] [010417 13:51] wrote: : :There's actually very little code that non-premptable once we get the :kernel mutexed. The least complex way to accomplish this is to only :preempt kernel processes that hold no mutex (low level) locks. : :-- :-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] I wish it were that simple. Remember when we went to shared file descriptor arrays for processes? Half a dozen serious bugs were introduced into the kernel from that change which weren't fixed for over a year and half. And that was *WITH* a Giant lock left intact. A change to a single subsystem introduced 6+ serious bugs. Now you guys are introducing preemptive kernel scheduling and expect it will be a walk in the park? I expect it will introduce so many bugs that the system will be destabilized for over a year, because that sort of thing effects every single subsystem rather then just one or two. It is not going to be as simple as throwing mutexes around every little piece of code that might access shared state somewhere. How many mutex calls are we going to wind up making to implement the simplest system call now? How much overhead are we going to introduce to support a feature that isn't going to add one iota of performance to the system? An incredible number of very cool optimizations can be made when you have the ability to assume that the cpu you are running under will not get ripped out from under you. Things like per-cpu caching - not only of mbufs or memory, but of other things as well. Simple access to per-cpu globals now must suddenly be done with compare-exchange or otherwise be required to use atomic instructions. None of the current per-cpu variable handling code assumes that. It's just not a good idea. Do you want people to adopt -current when it becomes 5.0? It isn't going to happen if preemptive kernel scheduling is in there. I am going to wind up having to support 4.x for the next two years before 5.x becomes reliable enough to use in a production environment. Or, worse, -current is going to move down a path of no-return (becoming so unreliable that it will not be adopted by anyone) and we are going to wind up having to scrap the whole damn thing! I was somewhat skeptical about BSDI's interrupt-as-thread methodology, but at least they didn't bite of more then they could chew when they implemented it. Trying to extend that model to the Nth degree is creating a huge mess -- and for what? Just to support interrupts as threads? It isn't worth it. I would much rather see us changing the interrupt model to be less invasive, thus allowing interrupts to run unhindered as interrupts rather then threads, rather then see this preemptive kernel scheduling stuff go in. I love SMP, but I love stability more. BSDI focused on the things that actually effected performance. -current is not focused at all and is going off the deep end.. trying to do things that would be interesting work for 5.2 or 5.3 but just plain should not be in 5.0. -Matt -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
: :You need to settle dude, pre-emption isn't a goal, it's mearly a :_possible_ side effect. : :We're not aiming for pre-emption, we're aiming for more concurrancy. A goal of having more concurrency is laudable, but I think you are ignoring the costs of doing task switches verses the likely spin time for a mutex. The whole point of using fine-grained Mutexes is to not have significant performance-effecting collisions in the first place, so why bother to try to task switch if you wind up spining in one? The goal for Giant is to get rid of it, so why bother to implement preemption for Giant? The goal of taking an interrupt is to be able to take several interrupts in parallel on different cpu's, and they can't block anyway, so why try to turn interrupts into real threads? It just doesn't make sense, Alfred. You guys are trying to optimize things that don't need optimizing and my fear is that you will introduce so many bugs into the kernel that it will take us years to get it back to 4.x's level of reliability. The goals I see bandied about in the lists are at cross-purposes with each other. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Tue, 17 Apr 2001, Matt Dillon wrote: :You need to settle dude, pre-emption isn't a goal, it's mearly a :_possible_ side effect. : :We're not aiming for pre-emption, we're aiming for more concurrancy. A goal of having more concurrency is laudable, but I think you are ignoring the costs of doing task switches verses the likely spin time for a mutex. The whole point of using fine-grained Mutexes is to not have significant performance-effecting collisions in the first place, so why bother to try to task switch if you wind up spining in one? Exactly. It makes absolutely no sense to optimise mutexes for the case of having contention. If you have contention, you need to fix the contention, not introduce all kinds of singing-dancing cool things you can do when contention happens. Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
* Matt Dillon [EMAIL PROTECTED] [010417 14:07] wrote: : :You need to settle dude, pre-emption isn't a goal, it's mearly a :_possible_ side effect. : :We're not aiming for pre-emption, we're aiming for more concurrancy. A goal of having more concurrency is laudable, but I think you are ignoring the costs of doing task switches verses the likely spin time for a mutex. The whole point of using fine-grained Mutexes is to not have significant performance-effecting collisions in the first place, so why bother to try to task switch if you wind up spining in one? The goal for Giant is to get rid of it, so why bother to implement preemption for Giant? The goal of taking an interrupt is to be able to take several interrupts in parallel on different cpu's, and they can't block anyway, so why try to turn interrupts into real threads? It just doesn't make sense, Alfred. You guys are trying to optimize things that don't need optimizing and my fear is that you will introduce so many bugs into the kernel that it will take us years to get it back to 4.x's level of reliability. The goals I see bandied about in the lists are at cross-purposes with each other. You seem to be more concerned with the implementation rather than the goal. The goal is to have a kernel that's able to have more concurrancy, things like pre-emption and task switching on mutex collisions can be examined and possibly changed later. Once the mutexes are in place the underlying implementation can change pretty easily from task switching always to only task switching when the mutex is owned by the same CPU that I'm running on. (to avoid spinlock deadlock) I agree that task switching _might_ be a performance hit, however that can be fixed by only task switching when in interrupt context. A lot of things remain to be seen, but only if people like you sit down and start working with the SMP team to achieve the main goal, which is making the kernel MP safe for concurrant execution by locking down the appropriate structures and code paths. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
:Once the mutexes are in place the underlying implementation can :change pretty easily from task switching always to only task :switching when the mutex is owned by the same CPU that I'm running :on. (to avoid spinlock deadlock) That makes *NO* *SENSE* Alfred! So the first step is to introduce every single possible feature under the sun, requiring massive reworking of the code, even if it means the system becomes massively unstable, inefficient, and starts crashing and burning, and then only *later* we remove the features that don't pan out? That is the development model you are using for -current? :I agree that task switching _might_ be a performance hit, however :that can be fixed by only task switching when in interrupt context. WILL be a performance hit. WILL introduce major bugs. IS unnecessary, DOESN'T make any sense whatsoever, is at CROSS PURPOSES with goals already stated (not having any serious contention in the first place), REQUIRES massive changes to the code with not a chance in hell of producing an equivalent performance improvement for the trouble. There is no 'remains to be seen' here Alfred. This is setting up -current for a fall, one that could be entirely avoided *now* if you people would sit down and start thinking about what you are doing. Remember when Dyson introduced ten thousand cool things all at once and didn't debug them? Remember how long it took DG to the system back into some semblence of shape after that? Remember that, even after all that hard work, there were still literally hundreds of hard to find bugs that DG couldn't find that it took Alan, DG, and I a year's worth of work beyond all the work DG had already done to cleanup? Is that what you want to happen to current? Because that is where current is headed right now. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Date: Tue, 17 Apr 2001 14:52:06 -0700 From: Alfred Perlstein [EMAIL PROTECTED] Disclaimer: I am not a kernel hacker. The goal is to have a kernel that's able to have more concurrancy, Right... things like pre-emption and task switching on mutex collisions can be examined and possibly changed later. I think that Matt is saying, why bother? Task switches are expensive. With a heavily loaded system, the amount of time spent arbitrating mutexes would exceed spin time. With a lightly loaded system, do we really care that much about spinning? Once the mutexes are in place the underlying implementation can change pretty easily from task switching always to only task switching when the mutex is owned by the same CPU that I'm running I'm not sure that I follow... if the task switch is on the same CPU, then why do we need any new structures? Without a global memory cache*, I think that keeping processes on the same CPU is the first goal, anyway, and increased concurrency second. * Does Intel have any sort of cache coherency mechanism, where a processor can tell other CPUs, "hey, this line is dirty" and the others retrieve the new info as needed? The Alpha? on. (to avoid spinlock deadlock) I agree that task switching _might_ be a performance hit, however that can be fixed by only task switching when in interrupt context. Ughh. AFAIK, traditional wisdom holds that interrupt loops should be as short as possible. Assuming that one performs quick operations, DMA transfers, etc., preemptive task switching *would* be expensive. A lot of things remain to be seen, but only if people like you sit down and start working with the SMP team to achieve the main goal, which is making the kernel MP safe for concurrant execution by locking down the appropriate structures and code paths. Let's look at userland code. I don't use async I/O because I don't want to screw with interrupts and callbacks. Polling of large structures? Ughh. Kernel queues are the solution, and what totally sold me on FreeBSD. How about basing things on *cooperative* multitasking? All else being equal, cooperative is faster because of lower overhead. What makes co-op break down is greedy programs that monopolize the CPU... but there should not be any code like that in the kernel. My instinct (whatever it's worth; remember my disclaimer) is that co-op switching using something like tsleep() and wakeup_one() or similar would be more efficient than trying to screw with mutexes. However, perhaps that could be improved upon: Use something a la kqueue for portions of the kernel to say "I'm waiting for condition XYZ". When we wakeup_one(), we specify "for condition XYZ". We could then avoid waking processes that would only go back to sleep again. I hope that I'm not too far out of my league, here... :-) Eddy --- Brotsman Dreger, Inc. EverQuick Internet / EternalCommerce Division Phone: (316) 794-8922 --- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
* Matt Dillon [EMAIL PROTECTED] [010417 15:00] wrote: :Once the mutexes are in place the underlying implementation can :change pretty easily from task switching always to only task :switching when the mutex is owned by the same CPU that I'm running :on. (to avoid spinlock deadlock) That makes *NO* *SENSE* Alfred! So the first step is to introduce every single possible feature under the sun, requiring massive reworking of the code, even if it means the system becomes massively unstable, inefficient, and starts crashing and burning, and then only *later* we remove the features that don't pan out? That is the development model you are using for -current? No I am not, what I'm trying to do is _use_ the APIs given to me to make the kernel more concurrent. I see no reason to abandon the external API, they are perfectly fine. You may disagree with the underlying implementation of the API and I think you may be right. (although afaik we're basing it on both Solaris and BSD/os's implementation so... well I'm not going to bother defending it.) I just wish you'd sit down and try to work adapting the VM to these APIs You're not going to get an arguement out of me about the underlying implementation of the mutexes as I really don't care right now. What I am going to argue is that argueing about the underlying implementation is a waste of time that could be better spent using the API to gain more concurrancy. :I agree that task switching _might_ be a performance hit, however :that can be fixed by only task switching when in interrupt context. WILL be a performance hit. WILL introduce major bugs. IS unnecessary, DOESN'T make any sense whatsoever, is at CROSS PURPOSES with goals already stated (not having any serious contention in the first place), REQUIRES massive changes to the code with not a chance in hell of producing an equivalent performance improvement for the trouble. There is no 'remains to be seen' here Alfred. This is setting up -current for a fall, one that could be entirely avoided *now* if you people would sit down and start thinking about what you are doing. No, sitting down and spending many months hashing out the underlying mechnisms of what should be a pretty opaque API is a waste of time and will stagnate developement at much too great a cost. In fact, we sorta spent about 3 years doing it already haven't we? :) Remember when Dyson introduced ten thousand cool things all at once and didn't debug them? Remember how long it took DG to the system back into some semblence of shape after that? Remember that, even after all that hard work, there were still literally hundreds of hard to find bugs that DG couldn't find that it took Alan, DG, and I a year's worth of work beyond all the work DG had already done to cleanup? Is that what you want to happen to current? Because that is where current is headed right now. Actually back then I wasn't that much of a kernel hacker, don't you remeber me standing there at SF BAFUG while you, Julian and Terry were discussing cache-coherency issues? I sort of had to take a break from listening to the lot of you because my brain went into overload, I was 19 or 20 at the time. :) -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Date: Tue, 17 Apr 2001 22:18:34 + (GMT) From: E.B. Dreger [EMAIL PROTECTED] My instinct (whatever it's worth; remember my disclaimer) is that co-op switching using something like tsleep() and wakeup_one() or similar would be more efficient than trying to screw with mutexes. Oops. I meant asleep(). Eddy --- Brotsman Dreger, Inc. EverQuick Internet / EternalCommerce Division Phone: (316) 794-8922 --- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Sun, 15 Apr 2001, Justin T. Gibbs wrote: There's no downside, really. It just seems inelegant to have a system that, on paper, is so inefficient. Can't we do better? Sure. Don't discard buffer contents when recycling a B_MALLOC'ed buffer, but manage it using a secondary buffer cache that doesn't have as much overhead as the primary one (in particular, don't reserve BKVASIZE bytes of kernel virtual address space for each secondary buffer). This would be even more inelegant, and more complicated, but not so inefficient. Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
I don't consider it inefficient. Sure, if you look at this one aspect of the caching taken out of context it may appear to be inefficient, but if you look at the whole enchilada the memory issue is nothing more then a minor footnote - not worth the effort of worrying about. This is like saying that there is nothing to be gained by making better use of available cache memory. I don't care that the cache is dynamic and caches the right things when the cache is effectively made so small that it doesn't cache my working set. Even with VMIO turned on for directories, Linux kicks our ass in caching meta-data unless you have a lot of memory. That sucks. -- Justin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
: :I don't consider it inefficient. Sure, if you look at this one aspect :of the caching taken out of context it may appear to be inefficient, :but if you look at the whole enchilada the memory issue is nothing :more then a minor footnote - not worth the effort of worrying about. : :This is like saying that there is nothing to be gained by making better :use of available cache memory. I don't care that the cache is dynamic :and caches the right things when the cache is effectively made so small :that it doesn't cache my working set. Even with VMIO turned on for :directories, Linux kicks our ass in caching meta-data unless you have :a lot of memory. That sucks. : :-- :Justin It is not implying that at all. There is no black and white here. This is a case where spending a huge amount of time and complexity to get the efficiency down to the Nth degree is nothing but a waste of time. What matters is what the user sees, what performance the application gets, and how many bugs you introduce when optimizing something that might not need optimizing. And in regards to Linux... you can't possibly be implying that performance differences between linux and FreeBSD are due to whether we spend 4K or 512 bytes caching a directory block. The issue has nothing to do with small-directory memory efficiency. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
"Justin T. Gibbs" wrote: I notice that this option is off by default. Can you give a general idea of when it should be enabled, when it should be disabled, and what bad things might result with it on? It consumes a full page per-directory even though the majority of directories in a stock system are a small fraction of that size. It is my understanding that with the new directory layout strategies, this will be improved somewhat. ie: a single page is much more likely to cache up to 8 directories. Cheers, -Peter -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
:It is my understanding that with the new directory layout strategies, this :will be improved somewhat. ie: a single page is much more likely to cache :up to 8 directories. : :Cheers, :-Peter :-- :Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] :"All of this is for nothing if we don't go to the stars" - JMS/B5 Only in the drive's own cache. Our VM Page cache is object-oriented, meaning that even though directories may be packed in the filesystem, they will not be packed in the buffer cache / VM Page cache. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
There's no downside, really. It just seems inelegant to have a system that, on paper, is so inefficient. Can't we do better? -- Justin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Matt Dillon wrote: If directories are spread all over the disk, caching is non-optimal. But if they are relatively close to each other then both our VM cache (if vfs.vmiodirenable is set to 1) and the hard drive's internal cache become extremely effective. I notice that this option is off by default. Can you give a general idea of when it should be enabled, when it should be disabled, and what bad things might result with it on? Thanks, Doug -- Perhaps the greatest damage the American system of education has done to its children is to teach them that their opinions are relevant simply because they are their opinions. Do YOU Yahoo!? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
I notice that this option is off by default. Can you give a general idea of when it should be enabled, when it should be disabled, and what bad things might result with it on? It consumes a full page per-directory even though the majority of directories in a stock system are a small fraction of that size. -- Justin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
[.] The second improvement, contributed by [EMAIL PROTECTED], is a new directory allocation policy (codenamed "dirpref"). Coupled with soft updates, the new dirpref code offers up to a 60x speed increase in gluk's tests, documented here:" http://groups.google.com/groups?q=dirprefnum=100hl=enlr=safe=offrnum=2seld=905073910ic=1 I do like the dirpref stuff, but I can't comment much on it except that it looks like a good change that should be fairly easy to bring into FreeBSD. I'm not 100% convinced about the algorithm to avoid clusters filling up with directory-only entries (it looks like a worst-case would fill a cluster with 50% directories and 50% files leaving a bad layout when the directories are populated further), but then the non-dirpref scheme has some far worse worst-case scenarios ;-) Just to follow up on myself... it seems the dirpref stuff was committed to FreeBSD this morning :-] -- Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org http://www.Awfulhak.org brian@[uk.]OpenBSD.org Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
: I'm not 100% convinced about the algorithm to avoid clusters filling : up with directory-only entries (it looks like a worst-case would fill : a cluster with 50% directories and 50% files leaving a bad layout when : the directories are populated further), but then the non-dirpref : scheme has some far worse worst-case scenarios ;-) : :Just to follow up on myself... it seems the dirpref stuff was :committed to FreeBSD this morning :-] : :-- :Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org Yup, Kirk committed it. I really like the changes -- in the old days disk caches were tiny and directories were not well cached on top of that. It made sense to try to keep directories close to their files. But today the proximity of a directory to its files is not really that important. It is far more important for directories to have reasonable proximity to each other not only to improve directory scans and lookups, but also to improve caching. Especially for small directories. Consider that small directories are typically contained in a single fragment (1K). If directories are spread all over the disk, caching is non-optimal. But if they are relatively close to each other then both our VM cache (if vfs.vmiodirenable is set to 1) and the hard drive's internal cache become extremely effective. With the added effectiveness of the caches, seeking should wind up being significantly reduced even for things like 'tar'. Large directories also benefit, I think. From looking at the code, I don't think fragmentation will be an issue. Or, to be more specific, I don't think the fact that files may not wind up in the same cylinder group as their directory entry is an issue. Either you have a huge number of directories being accessed and need the locality of reference within the directory space even if it costs some additional seeking to access underlying files, or you don't and the active directories all wind up being cached, removing any additional seeking from the equation entirely. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
On Tue, 10 Apr 2001, Matt Dillon wrote: : I'm not 100% convinced about the algorithm to avoid clusters filling : up with directory-only entries (it looks like a worst-case would fill : a cluster with 50% directories and 50% files leaving a bad layout when : the directories are populated further), but then the non-dirpref : scheme has some far worse worst-case scenarios ;-) : :Just to follow up on myself... it seems the dirpref stuff was :committed to FreeBSD this morning :-] : :-- :Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org Yup, Kirk committed it. I really like the changes -- in the old days disk caches were tiny and directories were not well cached on top of that. It made sense to try to keep directories close to their files. But today the proximity of a directory to its files is not really that important. It is far more important for directories to have reasonable proximity to each other not only to improve directory scans and lookups, but also to improve caching. Especially for small directories. Consider that small directories are typically contained in a single fragment (1K). If directories are spread all over the disk, caching is non-optimal. But if they are relatively close to each other then both our VM cache (if vfs.vmiodirenable is set to 1) and the hard drive's internal cache become extremely effective. With the added Why VMIO dir works better if directories are placed close to each other? I think it only makes the cache data of an individual directory stay in the memory longer. Is there a way to measure the effectiveness of the disk drive's cache? -Zhihui effectiveness of the caches, seeking should wind up being significantly reduced even for things like 'tar'. Large directories also benefit, I think. From looking at the code, I don't think fragmentation will be an issue. Or, to be more specific, I don't think the fact that files may not wind up in the same cylinder group as their directory entry is an issue. Either you have a huge number of directories being accessed and need the locality of reference within the directory space even if it costs some additional seeking to access underlying files, or you don't and the active directories all wind up being cached, removing any additional seeking from the equation entirely. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Why VMIO dir works better if directories are placed close to each other? I think it only makes the cache data of an individual directory stay in the memory longer. Is there a way to measure the effectiveness of the disk drive's cache? The real performance gain is seen when doing stuff with large directory hierarchies such as /usr/ports or (I think) a squid cache. The close proximity of the directories means they can be read/written far more quickly than before (where they were specifically placed in different clusters). -Zhihui -- Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org http://www.Awfulhak.org brian@[uk.]OpenBSD.org Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
:Why VMIO dir works better if directories are placed close to each other? I :think it only makes the cache data of an individual directory stay in the :memory longer. Is there a way to measure the effectiveness of the disk :drive's cache? : :-Zhihui I wasn't being clear enough. There are two different things going on. VMIO works better because it is capable of caching many, many more directories using the VM page cache (all of physical memory) rather then buffer malloc space (which is limited to a few megabytes). The buffer cache in general will also operate better with the improved locality of reference for the inodes underlying the files in the directories. The disk drive's cache works better because it will cache things immediately that the VM system caches across a longer span of time. If you are scanning a large number of small directories you can wind up with a situation where each small directory is being held in a file fragment (1K on disk). Several small directories may wind up together on-disk, either contiguous or very close by. The VM page cache cannot cache an entire filesystem block (potentially holdling 8 small directories) in a single operation oweing to the virtual, file-oriented nature of the cache. However, the disk drive's cache CAN, and will. The result is that you are suddenly able to read a huge number of small directories with 1/10 the number of hard seeks on the disk that you would otherwise have needed. Prior to this change, all those small directories were not near each other on-disk. Not only was the VM cache not able to cache the surrounding blocks, but even if the disk drive did the data would wind up being thrown away anyway because the surrounding blocks were not where the other directories resided. Disk drives are more limited by whatever seeking they have to do then by the amount of data they are transfering. Anything that reduces seeking has a huge effect on performance, and this is what is accomplished by the patch. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Yup, Kirk committed it. I really like the changes -- in the old days disk caches were tiny and directories were not well cached on top of that. It made sense to try to keep directories close to their files. So I'm all excited now at the progress that ufs/ffs are making recently. Yay Kirk. But this leaves one nagging question in my mind. What sort of intervention does one need to do to enable this newfangled wunderfs code? - alex To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FW: Filesystem gets a huge performance boost
Another important change is that it is no longer necessary to run tunefs in single user mode to activate soft updates. All that is needed is to add the "softdep" mount option to the partitions you want soft updates enabled on in /etc/fstab." [.] I especially like not having to run tunefs :-) [.] Having the softdep option in fstab(5) doesn't gel well with the recent background-fsck work being introduced by Kirk - although it works from what I can tell. In both OpenBSD and NetBSD, a filesystem mounted with the ``softdep'' option will update the super-block flags with the FS_DOSOFTDEP bit, so it's easy for fsck(8) to tell how an unclean filesystem was last mounted. In fact, OpenBSD has ``if 0''d code that allows unclean filesystem mounts if they have that FS_DOSOFTDEP bit set (NetBSD doesn't seem to have this). The problem I think is where a ``mount -u'' is done to downgrade a filesystem from soft-udpates to no soft-updates. Both OpenBSD and NetBSD have comments to the effect /* * Flush soft dependencies if disabling it via an update * mount. This may leave some items to be processed, * so don't do this yet XXX. */ and both ignore the problem (leaving soft-updates set). I don't think there's a satisfactory way of doing this - in much the same way as downgrading a read-write filesystem to read-only doesn't quite work. If certain operations are in effect (like a background fsck in the first instance or a reference is held to a file with a zero link count in the second), all hell can break loose. Having said all that, I quite like the softdep option in OpenBSD NetBSD, despite it only being a half-option :-) The second improvement, contributed by [EMAIL PROTECTED], is a new directory allocation policy (codenamed "dirpref"). Coupled with soft updates, the new dirpref code offers up to a 60x speed increase in gluk's tests, documented here:" http://groups.google.com/groups?q=dirprefnum=100hl=enlr=safe=offrnum=2seld=905073910ic=1 I do like the dirpref stuff, but I can't comment much on it except that it looks like a good change that should be fairly easy to bring into FreeBSD. I'm not 100% convinced about the algorithm to avoid clusters filling up with directory-only entries (it looks like a worst-case would fill a cluster with 50% directories and 50% files leaving a bad layout when the directories are populated further), but then the non-dirpref scheme has some far worse worst-case scenarios ;-) -- Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org http://www.Awfulhak.org brian@[uk.]OpenBSD.org Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
