Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Bruce Evans
On Thu, 19 Apr 2001, Andrey A. Chernov wrote: > On Thu, Apr 19, 2001 at 10:39:58 -0700, Matt Dillon wrote: > > Let me explain a little more. If it's commented out, it's fine. But > > if you are actually setting a value in there you will override whatever > > is set in the kernel. W

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Andrey A. Chernov
On Thu, Apr 19, 2001 at 10:47:20 -0700, Matt Dillon wrote: > :> set that default in stone and prevent us from being able to change > :> it with a new kernel rev. This being a *kernel* specific feature, > :> we need to have control over the default in the kernel itself. > : > :What abo

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
:> set that default in stone and prevent us from being able to change :> it with a new kernel rev. This being a *kernel* specific feature, :> we need to have control over the default in the kernel itself. : :What about simple check in the kernel: if total memory is above 64Mb, then :e

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Andrey A. Chernov
On Thu, Apr 19, 2001 at 10:39:58 -0700, Matt Dillon wrote: > :But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's > :wrong with putting some useful payload into it? > : > :-Maxim > > Let me explain a little more. If it's commented out, it's fine. But > if you a

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
:But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim Let me explain a little more. If it's commented out, it's fine. But if you are actually setting a value in there you will override whatever is se

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
:But we already have sysctl.conf and appropriate rc.sysctl, haven't we? What's :wrong with putting some useful payload into it? : :-Maxim If it's commented out, it's fine. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebs

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Matt Dillon wrote: > : > :What do you think about attached patch? > : > :-Maxim > > mmm.. I think it would just confuse the issue and prevent us from > being able to change the kernel default trivially. 99.5% of the > FreeBSD boxes out there are just going to want it to be on by defa

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Matt Dillon
: :What do you think about attached patch? : :-Maxim mmm.. I think it would just confuse the issue and prevent us from being able to change the kernel default trivially. 99.5% of the FreeBSD boxes out there are just going to want it to be on by default. We could provide a comment

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Doug Barton wrote: > Maxim Sobolev wrote: > > > > > What do you think about attached patch? > > Definitely the right idea, however I'm waiting on input from a couple > people on some additional suggestions, so if you'd hold off I'd appreciate > it. Unfortunately I've already cvs ci it. :

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Doug Barton
Maxim Sobolev wrote: > > > What do you think about attached patch? Definitely the right idea, however I'm waiting on input from a couple people on some additional suggestions, so if you'd hold off I'd appreciate it. -- "One thing they don't tell you about doing experimental physics is

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Alex Kapranoff
On Thu, Apr 19, 2001 at 03:46:39PM +0300, Maxim Sobolev wrote: > > What do you think about attached patch? > > -Maxim > > Index: Makefile > === > RCS file: /home/ncvs/src/etc/Makefile,v > retrieving revision 1.248 > diff -d -u -r1

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Alfred Perlstein
* Maxim Sobolev <[EMAIL PROTECTED]> [010419 06:20] wrote: > > OOPS, I see. See updated patch. Looks ok. > Index: Makefile > === > RCS file: /home/ncvs/src/etc/Makefile,v > retrieving revision 1.248 > diff -d -u -r1.248 Makefile > -

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Alfred Perlstein wrote: > * Maxim Sobolev <[EMAIL PROTECTED]> [010419 05:48] wrote: > > Doug Barton wrote: > > > > > Alfred Perlstein wrote: > > > > > > > I'm figuring the only time when it may be a problem is on machines > > > > with a small amount of memory. Since memory is cheap, I plan on >

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Alfred Perlstein
* Maxim Sobolev <[EMAIL PROTECTED]> [010419 05:48] wrote: > Doug Barton wrote: > > > Alfred Perlstein wrote: > > > > > I'm figuring the only time when it may be a problem is on machines > > > with a small amount of memory. Since memory is cheap, I plan on > > > turning it on within the next coup

Re: FW: Filesystem gets a huge performance boost

2001-04-19 Thread Maxim Sobolev
Doug Barton wrote: > Alfred Perlstein wrote: > > > I'm figuring the only time when it may be a problem is on machines > > with a small amount of memory. Since memory is cheap, I plan on > > turning it on within the next couple of days unless a stability > > issue comes up. > > > > I'll leave it

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Doug White
On Tue, 17 Apr 2001, Doug Barton wrote: > OK... this brings up the question of what other cool optimizations are > there that may have been disabled in the past for reasons that are no > longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file > with commented out example

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Jesper Skriver
On Wed, Apr 18, 2001 at 10:33:32AM +0200, Jeroen Ruigrok/Asmodai wrote: > -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: > >Testing it 'on' in stable on production systems and observing the > >relative change in performance is a worthy experiment. Testing it > >'on' in c

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Dan Langille
On 18 Apr 2001, at 22:16, Bruce Evans wrote: > On Wed, 18 Apr 2001, Jeroen Ruigrok/Asmodai wrote: > > > -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: > > >Testing it 'on' in stable on production systems and observing the > > >relative change in performance is a worthy expe

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Dan Langille
On 18 Apr 2001, at 10:33, Jeroen Ruigrok/Asmodai wrote: > -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: > >Testing it 'on' in stable on production systems and observing the > >relative change in performance is a worthy experiment. Testing it > >'on' in current is just

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Jeroen Ruigrok/Asmodai
-On [20010418 14:38], Bruce Evans ([EMAIL PROTECTED]) wrote: [vfs.vmiodirenable] >So, how much slower was it? ;-) Not noticeable for me at least. -- Jeroen Ruigrok van der Werven/Asmodai --=-- asmodai@[wxs.nl|freebsd.org] Documentation nutter/C-rated Coder BSD: Technical excellence at its bes

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Bruce Evans
On Wed, 18 Apr 2001, Jeroen Ruigrok/Asmodai wrote: > -On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: > >Testing it 'on' in stable on production systems and observing the > >relative change in performance is a worthy experiment. Testing it > >'on' in current is just an ex

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Jeroen Ruigrok/Asmodai
-On [20010418 01:00], Alfred Perlstein ([EMAIL PROTECTED]) wrote: > (although afaik we're basing it on both Solaris and BSD/os's > implementation so... well I'm not going to bother defending it.) You just scared the shit out of me by mentioning Solaris. I've found Solaris to be a PITA with all

Re: FW: Filesystem gets a huge performance boost

2001-04-18 Thread Jeroen Ruigrok/Asmodai
-On [20010417 20:47], Matt Dillon ([EMAIL PROTECTED]) wrote: >Testing it 'on' in stable on production systems and observing the >relative change in performance is a worthy experiment. Testing it >'on' in current is just an experiment. I have been running vfs.vmiodirenable=1 on two ST

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Bosko Milekic
On Tue, Apr 17, 2001 at 10:18:34PM +, E.B. Dreger wrote: > > Once the mutexes are in place the underlying implementation can > > change pretty easily from task switching always to only task > > switching when the mutex is owned by the same CPU that I'm running > > I'm not sure that I follow.

SMP architecture (Re: FW: Filesystem gets a huge performance boost)

2001-04-17 Thread E.B. Dreger
(cross-posting to SMP and renaming in an effort to move the thread) > Date: Tue, 17 Apr 2001 16:04:18 -0700 > From: Alfred Perlstein <[EMAIL PROTECTED]> (Repeat disclaimer: I am not a kernel hacker.) > seriously, it would be _trivial_ to: > > 1) make interrupts the only thing that could swit

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread E.B. Dreger
> Date: Tue, 17 Apr 2001 22:18:34 + (GMT) > From: E.B. Dreger <[EMAIL PROTECTED]> > > My instinct (whatever it's worth; remember my disclaimer) is that co-op > switching using something like tsleep() and wakeup_one() or similar would > be more efficient than trying to screw with mutexes. Oop

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Matt Dillon <[EMAIL PROTECTED]> [010417 15:00] wrote: > > :Once the mutexes are in place the underlying implementation can > :change pretty easily from task switching always to only task > :switching when the mutex is owned by the same CPU that I'm running > :on. (to avoid spinlock deadlock) >

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread E.B. Dreger
> Date: Tue, 17 Apr 2001 15:00:29 -0700 (PDT) > From: Matt Dillon <[EMAIL PROTECTED]> > > WILL be a performance hit. WILL introduce major bugs. IS unnecessary, > DOESN'T make any sense whatsoever, is at CROSS PURPOSES with goals > already stated (not having any serious contention in the first p

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread E.B. Dreger
> Date: Tue, 17 Apr 2001 14:52:06 -0700 > From: Alfred Perlstein <[EMAIL PROTECTED]> Disclaimer: I am not a kernel hacker. > The goal is to have a kernel that's able to have more concurrancy, Right... > things like pre-emption and task switching on mutex collisions can > be examined and possib

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
:Once the mutexes are in place the underlying implementation can :change pretty easily from task switching always to only task :switching when the mutex is owned by the same CPU that I'm running :on. (to avoid spinlock deadlock) That makes *NO* *SENSE* Alfred! So the first step is to intro

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Matt Dillon <[EMAIL PROTECTED]> [010417 14:07] wrote: > > : > :You need to settle dude, pre-emption isn't a goal, it's mearly a > :_possible_ side effect. > : > :We're not aiming for pre-emption, we're aiming for more concurrancy. > > A goal of having more concurrency is laudable, but I t

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Rik van Riel
On Tue, 17 Apr 2001, Matt Dillon wrote: > :You need to settle dude, pre-emption isn't a goal, it's mearly a > :_possible_ side effect. > : > :We're not aiming for pre-emption, we're aiming for more concurrancy. > > A goal of having more concurrency is laudable, but I think you are > ig

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
: :You need to settle dude, pre-emption isn't a goal, it's mearly a :_possible_ side effect. : :We're not aiming for pre-emption, we're aiming for more concurrancy. A goal of having more concurrency is laudable, but I think you are ignoring the costs of doing task switches verses the l

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
You need to settle dude, pre-emption isn't a goal, it's mearly a _possible_ side effect. We're not aiming for pre-emption, we're aiming for more concurrancy. * Matt Dillon <[EMAIL PROTECTED]> [010417 13:51] wrote: > : > :There's actually very little code that non-premptable once we get the > :k

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
: :There's actually very little code that non-premptable once we get the :kernel mutexed. The least complex way to accomplish this is to only :preempt kernel processes that hold no mutex (low level) locks. : :-- :-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] I wish it were that

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Matt Dillon <[EMAIL PROTECTED]> [010417 10:22] wrote: > > :* Matt Dillon <[EMAIL PROTECTED]> [010415 23:16] wrote: > :> > :> For example, all this work on a preemptive > :> kernel is just insane. Our entire kernel is built on the concept of > :> not being

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
: :Matt Dillon wrote: : :> It is not implying that at all. There is no black and white here. :> This is a case where spending a huge amount of time and complexity :> to get the efficiency down to the Nth degree is nothing but a waste :> of time. What matters is what the user see

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon
:* Matt Dillon <[EMAIL PROTECTED]> [010415 23:16] wrote: :> :> For example, all this work on a preemptive :> kernel is just insane. Our entire kernel is built on the concept of :> not being preemptable except by interrupts. We virtually guarentee :> ye

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Doug Barton
Alfred Perlstein wrote: > I'm figuring the only time when it may be a problem is on machines > with a small amount of memory. Since memory is cheap, I plan on > turning it on within the next couple of days unless a stability > issue comes up. > > I'll leave it to those people with low memory t

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Matt Dillon <[EMAIL PROTECTED]> [010415 23:16] wrote: > > For example, all this work on a preemptive > kernel is just insane. Our entire kernel is built on the concept of > not being preemptable except by interrupts. We virtually guarentee > years of

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein
* Doug Barton <[EMAIL PROTECTED]> [010417 01:08] wrote: > Matt Dillon wrote: > > > It is not implying that at all. There is no black and white here. > > This is a case where spending a huge amount of time and complexity > > to get the efficiency down to the Nth degree is nothing but

Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Doug Barton
Matt Dillon wrote: > It is not implying that at all. There is no black and white here. > This is a case where spending a huge amount of time and complexity > to get the efficiency down to the Nth degree is nothing but a waste > of time. What matters is what the user sees, what p

Re: FW: Filesystem gets a huge performance boost

2001-04-16 Thread Matt Dillon
:> It just seems inelegant to have a system that, on paper, is :> so inefficient. Can't we do better? : :Sure. Don't discard buffer contents when recycling a B_MALLOC'ed buffer, :but manage it using a secondary buffer cache that doesn't have as much :overhead as the primary one (in particular,

Re: FW: Filesystem gets a huge performance boost

2001-04-16 Thread Matt Dillon
: :>I don't consider it inefficient. Sure, if you look at this one aspect :>of the caching taken out of context it may appear to be inefficient, :>but if you look at the whole enchilada the memory issue is nothing :>more then a minor footnote - not worth the effort of worrying a

Re: FW: Filesystem gets a huge performance boost

2001-04-16 Thread Justin T. Gibbs
>I don't consider it inefficient. Sure, if you look at this one aspect >of the caching taken out of context it may appear to be inefficient, >but if you look at the whole enchilada the memory issue is nothing >more then a minor footnote - not worth the effort of worrying about.

Re: FW: Filesystem gets a huge performance boost

2001-04-16 Thread Bruce Evans
On Sun, 15 Apr 2001, Justin T. Gibbs wrote: > >There's no downside, really. > > It just seems inelegant to have a system that, on paper, is > so inefficient. Can't we do better? Sure. Don't discard buffer contents when recycling a B_MALLOC'ed buffer, but manage it using a secondary buffer

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Matt Dillon
: :>There's no downside, really. : :It just seems inelegant to have a system that, on paper, is :so inefficient. Can't we do better? : :-- :Justin I don't consider it inefficient. Sure, if you look at this one aspect of the caching taken out of context it may appear to be inefficie

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Justin T. Gibbs
>There's no downside, really. It just seems inelegant to have a system that, on paper, is so inefficient. Can't we do better? -- Justin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Matt Dillon
:It is my understanding that with the new directory layout strategies, this :will be improved somewhat. ie: a single page is much more likely to cache :up to 8 directories. : :Cheers, :-Peter :-- :Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] :"All of this is for nothing if

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Peter Wemm
"Justin T. Gibbs" wrote: > > I notice that this option is off by default. Can you give a general > >idea of when it should be enabled, when it should be disabled, and what bad > >things might result with it on? > > It consumes a full page per-directory even though the majority of > directori

Re: FW: Filesystem gets a huge performance boost

2001-04-15 Thread Matt Dillon
: : I notice that this option is off by default. Can you give a general idea :of when it should be enabled, when it should be disabled, and what bad :things might result with it on? : :Thanks, : :Doug There's no downside, really. The directory cache is so tiny without it that you

Re: FW: Filesystem gets a huge performance boost

2001-04-14 Thread Justin T. Gibbs
> I notice that this option is off by default. Can you give a general >idea of when it should be enabled, when it should be disabled, and what bad >things might result with it on? It consumes a full page per-directory even though the majority of directories in a stock system are a small fr

Re: FW: Filesystem gets a huge performance boost

2001-04-14 Thread Doug Barton
Matt Dillon wrote: > If directories are spread all over the disk, caching > is non-optimal. But if they are relatively close to each other then > both our VM cache (if vfs.vmiodirenable is set to 1) and the hard > drive's internal cache become extremely effective. I notice

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Alex Zepeda
> Yup, Kirk committed it. I really like the changes -- in the old days > disk caches were tiny and directories were not well cached on top of that. > It made sense to try to keep directories close to their files. So I'm all excited now at the progress that ufs/ffs are making recently

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Matt Dillon
:Why VMIO dir works better if directories are placed close to each other? I :think it only makes the cache data of an individual directory stay in the :memory longer. Is there a way to measure the effectiveness of the disk :drive's cache? : :-Zhihui I wasn't being clear enough. There are tw

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Brian Somers
> Why VMIO dir works better if directories are placed close to each other? I > think it only makes the cache data of an individual directory stay in the > memory longer. Is there a way to measure the effectiveness of the disk > drive's cache? The real performance gain is seen when doing stuff wi

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Zhihui Zhang
On Tue, 10 Apr 2001, Matt Dillon wrote: > :> I'm not 100% convinced about the algorithm to avoid clusters filling > :> up with directory-only entries (it looks like a worst-case would fill > :> a cluster with 50% directories and 50% files leaving a bad layout when > :> the directories are po

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Matt Dillon
:> I'm not 100% convinced about the algorithm to avoid clusters filling :> up with directory-only entries (it looks like a worst-case would fill :> a cluster with 50% directories and 50% files leaving a bad layout when :> the directories are populated further), but then the non-dirpref :> sche

Re: FW: Filesystem gets a huge performance boost

2001-04-10 Thread Brian Somers
[.] > > The second improvement, contributed by > > [EMAIL PROTECTED], is a new directory allocation policy (codenamed > > "dirpref"). Coupled with soft updates, the new dirpref code offers up > > to a 60x speed increase in gluk's tests, documented here:" > > > > >ht

Re: FW: Filesystem gets a huge performance boost

2001-04-09 Thread Brian Somers
> Another important change is that it is no longer necessary to run > tunefs in single user mode to activate soft updates. All that is > needed is to add the "softdep" mount option to the partitions you > want soft updates enabled on in /etc/fstab." [.] > I especially like not having to run tu