Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Doug Barton wrote: Alfred Perlstein wrote: I'm figuring the only time when it may be a problem is on machines with a small amount of memory. Since memory is cheap, I plan on turning it on within the next couple of days unless a stability issue comes up. I'll leave it to those

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Alfred Perlstein wrote: * Maxim Sobolev [EMAIL PROTECTED] [010419 05:48] wrote: Doug Barton wrote: Alfred Perlstein wrote: I'm figuring the only time when it may be a problem is on machines with a small amount of memory. Since memory is cheap, I plan on turning it on

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Alfred Perlstein
* Maxim Sobolev [EMAIL PROTECTED] [010419 06:20] wrote: OOPS, I see. See updated patch. Looks ok. Index: Makefile === RCS file: /home/ncvs/src/etc/Makefile,v retrieving revision 1.248 diff -d -u -r1.248 Makefile ---

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Alex Kapranoff
On Thu, Apr 19, 2001 at 03:46:39PM +0300, Maxim Sobolev wrote: What do you think about attached patch? -Maxim Index: Makefile === RCS file: /home/ncvs/src/etc/Makefile,v retrieving revision 1.248 diff -d -u -r1.248

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Doug Barton
Maxim Sobolev wrote: What do you think about attached patch? Definitely the right idea, however I'm waiting on input from a couple people on some additional suggestions, so if you'd hold off I'd appreciate it. -- "One thing they don't tell you about doing experimental physics is

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Doug Barton wrote: Maxim Sobolev wrote: What do you think about attached patch? Definitely the right idea, however I'm waiting on input from a couple people on some additional suggestions, so if you'd hold off I'd appreciate it. Unfortunately I've already cvs ci it. :(

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
: :What do you think about attached patch? : :-Maxim mmm.. I think it would just confuse the issue and prevent us from being able to change the kernel default trivially. 99.5% of the FreeBSD boxes out there are just going to want it to be on by default. We could provide a

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Matt Dillon wrote: : :What do you think about attached patch? : :-Maxim mmm.. I think it would just confuse the issue and prevent us from being able to change the kernel default trivially. 99.5% of the FreeBSD boxes out there are just going to want it to be on by default.

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
:But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim If it's commented out, it's fine. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
:But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim Let me explain a little more. If it's commented out, it's fine. But if you are actually setting a value in there you will override whatever is

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Andrey A. Chernov
On Thu, Apr 19, 2001 at 10:39:58 -0700, Matt Dillon wrote: :But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim Let me explain a little more. If it's commented out, it's fine. But if you are

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
: set that default in stone and prevent us from being able to change : it with a new kernel rev. This being a *kernel* specific feature, : we need to have control over the default in the kernel itself. : :What about simple check in the kernel: if total memory is above 64Mb, then

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Andrey A. Chernov
On Thu, Apr 19, 2001 at 10:47:20 -0700, Matt Dillon wrote: : set that default in stone and prevent us from being able to change : it with a new kernel rev. This being a *kernel* specific feature, : we need to have control over the default in the kernel itself. : :What about

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Bruce Evans
On Thu, 19 Apr 2001, Andrey A. Chernov wrote: On Thu, Apr 19, 2001 at 10:39:58 -0700, Matt Dillon wrote: Let me explain a little more. If it's commented out, it's fine. But if you are actually setting a value in there you will override whatever is set in the kernel. When

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Jeroen Ruigrok/Asmodai
-On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: Testing it 'on' in stable on production systems and observing the relative change in performance is a worthy experiment. Testing it 'on' in current is just an experiment. I have been running vfs.vmiodirenable=1 on two

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Jeroen Ruigrok/Asmodai
-On [20010418 01:00], Alfred Perlstein ([EMAIL PROTECTED]) wrote: (although afaik we're basing it on both Solaris and BSD/os's implementation so... well I'm not going to bother defending it.) You just scared the shit out of me by mentioning Solaris. I've found Solaris to be a PITA with all

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Bruce Evans
On Wed, 18 Apr 2001, Jeroen Ruigrok/Asmodai wrote: -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: Testing it 'on' in stable on production systems and observing the relative change in performance is a worthy experiment. Testing it 'on' in current is just an

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Jeroen Ruigrok/Asmodai
-On [20010418 14:38], Bruce Evans ([EMAIL PROTECTED]) wrote: [vfs.vmiodirenable] So, how much slower was it? ;-) Not noticeable for me at least. -- Jeroen Ruigrok van der Werven/Asmodai --=-- asmodai@[wxs.nl|freebsd.org] Documentation nutter/C-rated Coder BSD: Technical excellence at its

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Dan Langille
On 18 Apr 2001, at 22:16, Bruce Evans wrote: On Wed, 18 Apr 2001, Jeroen Ruigrok/Asmodai wrote: -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: Testing it 'on' in stable on production systems and observing the relative change in performance is a worthy experiment.

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Doug White
On Tue, 17 Apr 2001, Doug Barton wrote: OK... this brings up the question of what other cool optimizations are there that may have been disabled in the past for reasons that are no longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file with commented out examples

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Doug Barton
Matt Dillon wrote: It is not implying that at all. There is no black and white here. This is a case where spending a huge amount of time and complexity to get the efficiency down to the Nth degree is nothing but a waste of time. What matters is what the user sees, what

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Doug Barton [EMAIL PROTECTED] [010417 01:08] wrote: Matt Dillon wrote: It is not implying that at all. There is no black and white here. This is a case where spending a huge amount of time and complexity to get the efficiency down to the Nth degree is nothing but a waste

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Matt Dillon [EMAIL PROTECTED] [010415 23:16] wrote: For example, all this work on a preemptive kernel is just insane. Our entire kernel is built on the concept of not being preemptable except by interrupts. We virtually guarentee years of

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Doug Barton
Alfred Perlstein wrote: I'm figuring the only time when it may be a problem is on machines with a small amount of memory. Since memory is cheap, I plan on turning it on within the next couple of days unless a stability issue comes up. I'll leave it to those people with low memory to

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
: :There's actually very little code that non-premptable once we get the :kernel mutexed. The least complex way to accomplish this is to only :preempt kernel processes that hold no mutex (low level) locks. : :-- :-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] I wish it were that

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
You need to settle dude, pre-emption isn't a goal, it's mearly a _possible_ side effect. We're not aiming for pre-emption, we're aiming for more concurrancy. * Matt Dillon [EMAIL PROTECTED] [010417 13:51] wrote: : :There's actually very little code that non-premptable once we get the

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
: :You need to settle dude, pre-emption isn't a goal, it's mearly a :_possible_ side effect. : :We're not aiming for pre-emption, we're aiming for more concurrancy. A goal of having more concurrency is laudable, but I think you are ignoring the costs of doing task switches verses the

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Rik van Riel
On Tue, 17 Apr 2001, Matt Dillon wrote: :You need to settle dude, pre-emption isn't a goal, it's mearly a :_possible_ side effect. : :We're not aiming for pre-emption, we're aiming for more concurrancy. A goal of having more concurrency is laudable, but I think you are ignoring

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Matt Dillon [EMAIL PROTECTED] [010417 14:07] wrote: : :You need to settle dude, pre-emption isn't a goal, it's mearly a :_possible_ side effect. : :We're not aiming for pre-emption, we're aiming for more concurrancy. A goal of having more concurrency is laudable, but I think you

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
:Once the mutexes are in place the underlying implementation can :change pretty easily from task switching always to only task :switching when the mutex is owned by the same CPU that I'm running :on. (to avoid spinlock deadlock) That makes *NO* *SENSE* Alfred! So the first step is to

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread E.B. Dreger
Date: Tue, 17 Apr 2001 14:52:06 -0700 From: Alfred Perlstein [EMAIL PROTECTED] Disclaimer: I am not a kernel hacker. The goal is to have a kernel that's able to have more concurrancy, Right... things like pre-emption and task switching on mutex collisions can be examined and possibly

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Matt Dillon [EMAIL PROTECTED] [010417 15:00] wrote: :Once the mutexes are in place the underlying implementation can :change pretty easily from task switching always to only task :switching when the mutex is owned by the same CPU that I'm running :on. (to avoid spinlock deadlock)

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread E.B. Dreger
Date: Tue, 17 Apr 2001 22:18:34 + (GMT) From: E.B. Dreger [EMAIL PROTECTED] My instinct (whatever it's worth; remember my disclaimer) is that co-op switching using something like tsleep() and wakeup_one() or similar would be more efficient than trying to screw with mutexes. Oops. I

Re: FW: Filesystem gets a huge performance boost

2001-04-16 Thread Bruce Evans
On Sun, 15 Apr 2001, Justin T. Gibbs wrote: There's no downside, really. It just seems inelegant to have a system that, on paper, is so inefficient. Can't we do better? Sure. Don't discard buffer contents when recycling a B_MALLOC'ed buffer, but manage it using a secondary buffer

Re: FW: Filesystem gets a huge performance boost

2001-04-16 Thread Justin T. Gibbs
I don't consider it inefficient. Sure, if you look at this one aspect of the caching taken out of context it may appear to be inefficient, but if you look at the whole enchilada the memory issue is nothing more then a minor footnote - not worth the effort of worrying about.

Re: FW: Filesystem gets a huge performance boost

2001-04-16 Thread Matt Dillon
: :I don't consider it inefficient. Sure, if you look at this one aspect :of the caching taken out of context it may appear to be inefficient, :but if you look at the whole enchilada the memory issue is nothing :more then a minor footnote - not worth the effort of worrying

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Peter Wemm
"Justin T. Gibbs" wrote: I notice that this option is off by default. Can you give a general idea of when it should be enabled, when it should be disabled, and what bad things might result with it on? It consumes a full page per-directory even though the majority of directories in a

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Matt Dillon
:It is my understanding that with the new directory layout strategies, this :will be improved somewhat. ie: a single page is much more likely to cache :up to 8 directories. : :Cheers, :-Peter :-- :Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] :"All of this is for nothing

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Justin T. Gibbs
There's no downside, really. It just seems inelegant to have a system that, on paper, is so inefficient. Can't we do better? -- Justin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message

Re: FW: Filesystem gets a huge performance boost

2001-04-14 Thread Doug Barton
Matt Dillon wrote: If directories are spread all over the disk, caching is non-optimal. But if they are relatively close to each other then both our VM cache (if vfs.vmiodirenable is set to 1) and the hard drive's internal cache become extremely effective. I notice

Re: FW: Filesystem gets a huge performance boost

2001-04-14 Thread Justin T. Gibbs
I notice that this option is off by default. Can you give a general idea of when it should be enabled, when it should be disabled, and what bad things might result with it on? It consumes a full page per-directory even though the majority of directories in a stock system are a small

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Brian Somers
[.] The second improvement, contributed by [EMAIL PROTECTED], is a new directory allocation policy (codenamed "dirpref"). Coupled with soft updates, the new dirpref code offers up to a 60x speed increase in gluk's tests, documented here:"

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Matt Dillon
: I'm not 100% convinced about the algorithm to avoid clusters filling : up with directory-only entries (it looks like a worst-case would fill : a cluster with 50% directories and 50% files leaving a bad layout when : the directories are populated further), but then the non-dirpref : scheme

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Zhihui Zhang
On Tue, 10 Apr 2001, Matt Dillon wrote: : I'm not 100% convinced about the algorithm to avoid clusters filling : up with directory-only entries (it looks like a worst-case would fill : a cluster with 50% directories and 50% files leaving a bad layout when : the directories are populated

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Brian Somers
Why VMIO dir works better if directories are placed close to each other? I think it only makes the cache data of an individual directory stay in the memory longer. Is there a way to measure the effectiveness of the disk drive's cache? The real performance gain is seen when doing stuff with

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Matt Dillon
:Why VMIO dir works better if directories are placed close to each other? I :think it only makes the cache data of an individual directory stay in the :memory longer. Is there a way to measure the effectiveness of the disk :drive's cache? : :-Zhihui I wasn't being clear enough. There are

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Alex Zepeda
Yup, Kirk committed it. I really like the changes -- in the old days disk caches were tiny and directories were not well cached on top of that. It made sense to try to keep directories close to their files. So I'm all excited now at the progress that ufs/ffs are making recently.

Re: FW: Filesystem gets a huge performance boost

2001-04-09 Thread Brian Somers
Another important change is that it is no longer necessary to run tunefs in single user mode to activate soft updates. All that is needed is to add the "softdep" mount option to the partitions you want soft updates enabled on in /etc/fstab." [.] I especially like not having to run tunefs