Re: NFS HEADS UP (was Re: cvs commit: src/sys/nfs nfsm_subs.h xdr_subs.h)

1999-08-21 Thread John S. Dyson

Matthew Dillon said:
 : global references across subroutine calls!  I'll send Luoqi another email.
 : 
 : In the case of the NFS stuff, the changes have been pretty well tested
 : so I think we are in the clear.
 :
 :On a somewhat similar note, what do you think about converting a lot
 :of the NFS macros to functions, yes i know it will be difficult, but
 :there is so much forced inlining it just seems like it would reduce
 :the codesize signifigantly and play nicer with the CPU cache.
 :
 :It would also make the code a lot more readable.
 :
 :Worthwhile exercise?
 :
 :-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] 
 
 Well, the issue with converting many of the macros to inline functions
 is with the embedded goto's and references to variables defined outside 
 the macros.  Converting them to functions would basically require 
 rewriting a huge chunk of NFS code.  
 
My working kernel runs with a few strategic NFS macros being converted
to functions, and the size improvement is about 50K or so (maybe more!!!)

John


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: NFS HEADS UP (was Re: cvs commit: src/sys/nfs nfsm_subs.h xdr_subs.h)

1999-08-21 Thread John S. Dyson

Matthew Dillon said:
 : Well, the issue with converting many of the macros to inline functions
 : is with the embedded goto's and references to variables defined outside 
 : the macros.  Converting them to functions would basically require 
 : rewriting a huge chunk of NFS code.  
 : 
 :My working kernel runs with a few strategic NFS macros being converted
 :to functions, and the size improvement is about 50K or so (maybe more!!!)
 :
 :John
 
 If you want to port some of it in that part of the source tree will be
 available in a month or two, depending on how quickly the other stuff in
 my queue gets committed.  I've got two patch sets currently under test
 related to other NFS issues and a third one in the wings.

The changes are "semi-trivial", and hope that a new kernel hacker can
take a crack at it.  Part of my comment however true, was meant as a
tease to get more kernel people involved (helping the cause.)  Anything
that any of us does can be done by others, and fostering a talent search
is a good thing (IMO.)

I am willing to provide the info, but hope that a naiscent kernel hacker
comes forward...

John


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: SMP users (important)

1999-04-06 Thread John S. Dyson
 
  per-processor registers that one could use (but loading a
  general register with that per processor register would be
  needed for access.)  Also, since the PPC has lots of registers,
  one could? permanently reserve one of the general registers (r13?).
 
 I really don't like the idea of breaking the normal language
 conventions for any processor.  For the i386, the use of segment
 registers is not relevant to C language conventions, but reserving a
 general register on any architecture should, IMHO, be avoided if at
 all possible.

I believe that SYSV might do it.

 
 I believe that having meaningful values context-switched in segment
 registers can add context switch overhead.  It shouldn't be
 significant, though.
 
It is probably *much* cheaper than the VM approaches.

John



To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: SMP users (important)

1999-04-03 Thread John S. Dyson
Alan Cox said:

 I've committed the basic infrastructure to improve TLB management
 on SMPs.  Translation: this will lead to the elimination of a LOT
 of interprocessor interrupts to invalidate TLB entries.  I'll be
 turning on the new mechanisms slowly so we can carefully debug
 each step and (hopefully) avoid any problems.
 
(To the rest of the team, Alan and Luoqi already know my opinion.)

I just wanted to chime in and say that the new patches are based
on a really good concept, and is much cleaner than the previous
method.  Also, many RISC architectures can utilize this
method due to the availability of lots of general registers.
(One could go so far as to have the compiler reserve the
 register.)  Non-threaded user mode apps could optionally
 use the reserved register, but for threaded user mode apps,
 that reserved register could also be used as a per-thread
 base pointer.

I believe that NT does the above (%fs for X86, and general
register for Alpha.)  On PPC, there are several local,
per-processor registers that one could use (but loading a
general register with that per processor register would be
needed for access.)  Also, since the PPC has lots of registers,
one could? permanently reserve one of the general registers (r13?).

All in all, this change has the potential for better context
switching time (and less memory/better performance for multi-threaded
processes.)  This is a serious, non-trivial movement in the *right*
direction!!! :-).  SMP users should be pleased with this movement (I
certainly am!!!)

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: UPDATE4: ATA/ATAPI driver new version available.

1999-03-29 Thread John S. Dyson
Soren Schmidt said:
 
 DMA support has been added to the ATA disk driver.
 This only works on Intel PIIX3/4, Acer Aladdin and Promise controllers. 
 The promise support works without the BIOS on the board,
 and timing modes are set to support up to UDMA speed. This
 solves the problems with having more than one promise controller
 in the same system.

Excellent!!!

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: rfork()

1999-03-20 Thread John S. Dyson
Michael E. Mercer said:
 Hello,
 
 This was posted to freebsd-questions with no reply.
 I tried this and the child process created a core file.
 I also tried the other options and they seem to work.
 Just RFPROC and RFMEM DON'T!
 
rfork(RFMEM) doesn't easily work from C.  You need to
create an assembly stub.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: rfork()

1999-03-20 Thread John S. Dyson
 On Sat, 20 Mar 1999, John S. Dyson wrote:
 
  Michael E. Mercer said:
   Hello,
   
   This was posted to freebsd-questions with no reply.
   I tried this and the child process created a core file.
   I also tried the other options and they seem to work.
   Just RFPROC and RFMEM DON'T!
   
  rfork(RFMEM) doesn't easily work from C.  You need to
  create an assembly stub.
  
  -- 
  John  | Never try to teach a pig to sing,
  dy...@iquest.net  | it makes one look stupid
  jdy...@nc.com | and it irritates the pig.
  
 
 I've seen about 6 people ask about this because the manual lies about
 what is done.  I asked a while back about it, and John was kind enough
 to dig up some code that used rfork to properly split the stack should
 I try to dig it up?
 
I suggest trying to find the example.  I might have it sitting around
here also.

John


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: VM86 assembly code problem

1999-03-16 Thread John S. Dyson
Matthew Dillon said:
 :There're a couple of places in swtch.s where code looks like this,
 :
 :#ifdef VM86
 :btrl%esi, _private_tss
 :je  3f
 : ...
 :3:
 :#endif
 :
 :The conditional jump statement doesn't seem right, according to manual,
 :btrl instruction modifies CF flag but not Z, so the jump should be jae/jb
 :instead of je/jne. Could anyone confirm this?
 :
 :-lq
 
 btrl only effects the Carry.  The VM86 code looks wrong to me, though
 there is an outside chance that it is doing a conditional jump based
 on something that occured prior to the btrl.
 
Even though you are correct in practice, the Intel Architecture Software
developer's manual #2 says that the ZF is undefined, not that it is
unchanged.  In fact, the above code sequence is incorrect by the book.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Killed Myself

1999-03-08 Thread John S. Dyson
Eivind Eklund said:
 On Tue, Nov 17, 1998 at 09:49:31PM -0500, HighWind Software Information wrote:
  
  After installing the recent libc_r and libc, I'm getting:
  
  ld.so failed: Undefined symbol SYS_kldsym in 
  make:/usr/lib/aout/libc.so.3.1 
  
  I also get it sometimes when I link against libc_r.
  
  SYS_kldsym is always the thing I don't seem to have a definition for.
  
  This just started happening. UGH!
 
 If you do not know how FreeBSD works to a detailed enough level to NOT
 HAVE TO ASK THIS, then you should MAKE WORLD.  You should NOT try to
 do incremental recompiles.  That is reserved for those people that
 know exactly what they are doing.
 
And those who know exactly what they are doing still get zapped.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Killed Myself

1999-03-08 Thread John S. Dyson
Eivind Eklund said:
 On Mon, Mar 08, 1999 at 05:59:05PM -0500, John S. Dyson wrote:
  Eivind Eklund said:
   If you do not know how FreeBSD works to a detailed enough level to NOT
   HAVE TO ASK THIS, then you should MAKE WORLD.  You should NOT try to
   do incremental recompiles.  That is reserved for those people that
   know exactly what they are doing.
  
  And those who know exactly what they are doing still get zapped.
 
 Of course - but we know not to ask :-)
 
I agree...  This is a deal where people need to know the limits of their
experience.  Of course, it is good when people try to push themselves.
When people delve deeper and deeper into the system, it seems that there
are more and more ways of burning oneself.  Anything like new system calls,
changing kernel data structures and (even worse) on disk formats is more
or less tricky for anyone.  With experience, ones is surprised less and less,
but still mistakes do happen :-(.

Is this clearly noted somewhere?  I know that it is common knowledge, but
as new people use the system (and get adventureous) some of 'em will get
burnt with mistakes.

Back in the late '70's, I started investigating C as a programming language.  It
took a week to get a hello world to compile and link.  I was an island, 
without any
support from anyone.  Such experiences do teach one to be somewhat tolerant.

My mistake:  I came from a DEC programming world, and I wrote the program
like this:

Main()
{
Printf(Hello world\n);
}

My brain wasn't trained to be case sensitive (of course, it is now), but that
was a most irritating experience.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: lockmgr panic with mmap()

1999-03-03 Thread John S. Dyson
Brian Feldman said:
  
  The lock manager isn't bright enough to detect that the process
  already holds a read lock when it attempts to get the write lock.
  Thus, you get the thrd_sleep instead of a panic.
  
  In short, same bug, different symptoms.
  
 Ahh, makes sense.
 
 Quick question: how in the heck did you learn this whole VM system?
 
Alan has had a history helping on the VM code for quite a while, and
has contributed some really good ideas.  Frankly, he appears to
understand the code as well as anyone does.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: gcc

1999-03-01 Thread John S. Dyson
John Polstra said:
 In article 19990228152909.e2...@relay.nuxi.com,
 David O'Brien obr...@nuxi.com wrote:
   I keep on hearing about how we're losing because we don't have the 3
   month old latest feature
  
  With EGCS the issue isn't having the latest 3 mo. feature, but we have a
  totally BROKEN C++ compiler.
 
 Yes.  We desperately need to upgrade our compiler, and egcs looks
 like it's definitely the way to go.
 
  What I have won't build kernels yet.  I could hack our system Makefiles
  to use different compile options, but I don't like that approach.  So
  I'm working on adding our compiler flags and such.
 
 Hang on.  Others have reported success building kernels with egcs.
 But even if there are problems, don't jump to the conclusion that
 they are caused by compiler bugs.  Linux also had problems building
 working kernels at first, but the problems turned out to be caused by
 incorrect clobber specs in asm statements.  Such errors have also
 been found in the FreeBSD kernel.  Some of them have been fixed, but
 maybe not all of them.  In any case, the problems should be fixed at
 their source, not kludged around in the compiler.
 
I can generally build a kernel with EGCS, if I change how the .text and
.data are laid out for initialized data.  It seems that the initialization
code makes assumptions about the order or layout of the initialization
data.  Once the stuff is made to act more like the version of GCC that
FreeBSD uses, the kernel will most often build and work.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: gcc

1999-03-01 Thread John S. Dyson
Jordan K. Hubbard said:
  I can generally build a kernel with EGCS, if I change how the .text and
  .data are laid out for initialized data.  It seems that the initialization
  code makes assumptions about the order or layout of the initialization
  data.  Once the stuff is made to act more like the version of GCC that
  FreeBSD uses, the kernel will most often build and work.
 
 It really does appear to be a simple matter of first making egcs take over
 the system compiler:
 
Okay, I guess I have been missing something :-).

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: gcc

1999-03-01 Thread John S. Dyson
Chuck Robey said:
 On Mon, 1 Mar 1999, John S. Dyson wrote:
 
  I can generally build a kernel with EGCS, if I change how the .text and
  .data are laid out for initialized data.  It seems that the initialization
  code makes assumptions about the order or layout of the initialization
  data.  Once the stuff is made to act more like the version of GCC that
  FreeBSD uses, the kernel will most often build and work.
 
 I'd like to experiment ... Next time you go to the trouble of making a
 kernel with egcs, could you make a few notes on those changes, and post
 them?
 
Okay, but JKH isn't having troubles...  Hmmm...  I do work with the
latest CVS checkout on egcs, so there could be a minor difference
there.  I might play with it tonight, and I do have a generally working
copy of egcs-current right now...  If it isn't tonight, will be tomorrow.


-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: gcc

1999-03-01 Thread John S. Dyson
John Polstra said:
 John S. Dyson wrote:
  Jordan K. Hubbard said:
   I can generally build a kernel with EGCS, if I change how the .text and
   .data are laid out for initialized data.  It seems that the 
   initialization
   code makes assumptions about the order or layout of the initialization
   data.  Once the stuff is made to act more like the version of GCC that
   FreeBSD uses, the kernel will most often build and work.
  
  It really does appear to be a simple matter of first making egcs take 
  over
  the system compiler:
  
  Okay, I guess I have been missing something :-).
 
 Keep in mind that a whole bunch of stuff connected with kernel
 initialization has been changed lately.  It may be simply that you're
 dealing with an older vintage kernel.
 
Ahhaaa...  That must be it -- I have a parallel track VM system project
going.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Promise IDE board docs

1999-02-24 Thread John S. Dyson
Martin Cracauer said:
 In 199902230725.caa02...@y.dyson.net, John S. Dyson wrote: 
  Søren Schmidt said:
   
   It should work, but the promise support in the old system is, well,
   hacky at best. I'm not sure if Promise supports more than one card
   at a time, but from looking at the chip specs, it should work just
   fine, and if the hardware works, at least the new driver will support
   it.
   
  I run with two (2) boards, but it appears that certain (all?) versions
  of the bios require that you remove the chip from all but one board.
 
 I did run such a setup as well, but the disks on the first controller
 with BIOS ran much faster than those on the BIOSless controller.
 
The order of the boards is critical, and which bios is installed.  Of course,
YMMV.  Each drive on each controller in my case runs at full speed :-).

On the bootup sequence, I get one BIOS display of eight drive slots, as if
I am using one board, with 8 drive positions (even though I really have
two boards.)  Of course, the PCI bus probes the controllers as two controllers,
but the BIOS kind of acts like one logical controller.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Filesystem deadlock

1999-02-23 Thread John S. Dyson
Luoqi Chen said:
  
 Do you still have that piece of code? Does it handle the case involves more
 than one process? For example, process 1 mmaps file B and reads file A into
 the mmapped region, while process 2 mmaps file A and reads file B, this could
 also result in a deadlock.
 
It used to be part of the tree, but I seem to remember that it was removed
(by those who understand the code :-)) soon after I left.  I will look for
it, and see if it would help with the problem(s).

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Promise IDE board docs

1999-02-23 Thread John S. Dyson
Søren Schmidt said:
 It seems John S. Dyson wrote:
  Søren Schmidt said:
   
   It should work, but the promise support in the old system is, well,
   hacky at best. I'm not sure if Promise supports more than one card
   at a time, but from looking at the chip specs, it should work just
   fine, and if the hardware works, at least the new driver will support
   it.
   
  I run with two (2) boards, but it appears that certain (all?) versions
  of the bios require that you remove the chip from all but one board.
 
 Bingo, looking at the HW specs I was pretty sure it would, but not
 being able to try it, I wouldn't bet on it.
 The BIOS on those boards are a different story, I run mine without
 the BIOS too, but then one has to setup the tranferrates etc by
 hand.
 
Now, that is INTERESTING.  With the bios that I have, all of the ports
on both boards get configured properly, and run UDMA.  Next time I
crack my box open, I'll let you know the bios version and board
(and bios) positions.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Filesystem deadlock

1999-02-22 Thread John S. Dyson
Luoqi Chen said:
  
 This seems to be the good old vnode deadlock during vm_fault() that has been
 reported a couple of times, and there's still no satisfactory solution to it:
 fgrep does something like this: (don't ask me why)
 
   addr = mmap(0, len, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, offset);
   read(fd, addr, count);
 
 the read() syscall first locks the vnode, read the data from disk, then copy
 the data to buffer at addr, now if addr is not in core, there'll be a page
 fault and the fault handler vm_fault will try to lock the vnode pager backing
 the page at addr, which is already locked, deadlock. This deadlock then
 propagates all the way back to the root vnode and the whole system would
 freeze.

I believe that I had a pseudo-fix to that, and it might have been removed.
(In non-multithreaded kernels, when having to do things like the above,
 allowing recursive locks under certain circumstances can solve the problem.
 The key is to avoid the case where it covers up real bugs.)

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Promise IDE board docs

1999-02-22 Thread John S. Dyson
Søren Schmidt said:
 
 It should work, but the promise support in the old system is, well,
 hacky at best. I'm not sure if Promise supports more than one card
 at a time, but from looking at the chip specs, it should work just
 fine, and if the hardware works, at least the new driver will support
 it.
 
I run with two (2) boards, but it appears that certain (all?) versions
of the bios require that you remove the chip from all but one board.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: SMP and SO5.0

1999-02-17 Thread John S. Dyson
Julian Elischer said:
 
 
 On Tue, 16 Feb 1999, Luoqi Chen wrote:
  You may try my patch at http://www.freebsd.org/~luoqi, which would allow
  linux threads to run on SMP.
 
 I've gone through these patches and I can see that they are really needed
 for SMP where address spaces are shared.
 
I agree -- a per-cpu page directory per multithreaded process is the way that
I had implemented.  Excellent!!!

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.


To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Problems in VM structure ?

1999-02-16 Thread John S. Dyson
Matthew Dillon said:
 :maxusers 256
 
 Try reducing maxusers to 128.  Another person reported similar behavior
 to me and after a bunch of work he tried going back to a basic 
 distribution -- and everything started working again.
 
 It turned out that a maxusers value of 256 and 512 were causing his 
 machine
 to go poof, but a maxusers value of 128 worked fine.
 
 I haven't tracked the problem down yet.  Please try reducing your maxusers
 to 128 and email the results to current.
 
Likely because data structures are getting too big.  The kernel is limited
to (I forget) how big in VA space.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message



Re: Significant page coloring improvement

1999-02-07 Thread John S. Dyson
Matthew Dillon said:

 Ah, interesting.  I understand the second bit.  The first bit seems
 somewhat odd, though - the automatic page coloring adjustment made
 by _vm_object_allocate() doesn't work well enough for kmem_object?
 

The problem with it was that there appeared to be a clash.  The color
allocation in _vm_object_allocate is ad-hoc, and tuned for a general
case, essentially randomizing the coloring (but also statistically
coloring processes approximately correctly.)  My original code (and
I forget if the current code does this) attempts to color the objects
(or pages in the objects) so that there isn't much overlap in normal
processes.

There is still an opportunity to improve the color allocations.  I also
have some mods that remove the L1 coloring (since it is overkill, and
just complicates the code.)  L2 coloring (or L3 as appropriate on machines
like Alphas) is all that is needed!!!  I did the L1 coloring for an exercise,
and forgot reality when I committed the code :-).  (Okay, that isn't quite
true, I did think that L1 coloring would have been useful -- but after
alot of thought and paper research, have decided that L1 page coloring for a
small L1 cache is kind-of useless.)

If you want to see the L1 mods and perhaps remove the L1 coloring, you are
welcome and it would be a good thing to remove it.  The L1 mods are pretty
much straight-forward, and might be a compromise between removing the coloring
all together and keeping all of the complexity.

I do suggest that the base color allocation (and proper management of the
coloring) would be a good day or so project to clean-up.  Again, right
now, the coloring looks okay, and the kernel page coloring choices were just
a degenerate case.  The low level coloring code is good -- so improving
the upper level mgmt is fertile ground.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message


Re: Significant page coloring improvement

1999-02-07 Thread John S. Dyson
Matthew Dillon said:
 
 : next_index += PQ_L2_SIZE/4;
 : if (next_index  PQ_L2_MASK)
 : next_index = (next_index + 1)  PQ_L2_MASK;
 
 Oops, make that:
 
   next_index += PQ_L2_SIZE/4;
   if (next_index  PQ_L2_MASK)
   next_index = (next_index + PQ_PRIME1)  PQ_L2_MASK;
 
 Or even just:
 
   next_index = (next_index + PQ_PRIME1)  PQ_L2_MASK;
 
 Both seem to work pretty well w/ lmbench, though nothing really sticks
 its nose out.
 
The reason why you might want to incr by PQ_L2_SIZE/N (or some other
large number) is that it will likely decrease conflicts for larger objects.

Note that the color should be chosen with more context (virtual address or
memory region type like shared lib) than I originally implemented.   There
are better approaches that take into consideration dynamic conflicts.  Such
dynamic conflicts might be more complex to determine, and a good static
choice adds only minimal overhead, yet provides some improvement.  I suggest
that until a major project can be undertaken, the static stuff is the right
thing.  It is easy to screw things up (as you can tell by my original
choice for coloring being suboptimal, but not necessarily destructive.)

I am agnostic regarding the coloring scheme, but I am glad that removing L1
coloring might be acceptable...  If anything, it will decrease instruction
cache footprint, and not cause a significant (hopefully any) decrease in
performance.

I will try to package up the patches (it is an issue of merging them in from
my codebase.)  They are essentially a result of hand optimizing the case of
setting PQ_L1_SIZE to 1.  Give me a day or so to put it together.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message


Re: Significant page coloring improvement

1999-02-07 Thread John S. Dyson
Matthew Dillon said:

 Ah, interesting.  I understand the second bit.  The first bit seems
 somewhat odd, though - the automatic page coloring adjustment made
 by _vm_object_allocate() doesn't work well enough for kmem_object?
 
There appears to be a clash.  I haven't really carefully evaluate it, but
did see a small improvement.  (I hate tweaks though!!!)  I guess until a
more scientific approach can be established, better tweaks are better than
worse tweaks :-).

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message


Significant page coloring improvement

1999-02-06 Thread John S. Dyson
When reviewing the VM code regarding another issue (another significant
VM contributor had found an interesting anomoly), I noticed that the
coloring wasn't as complete as it should be.

Attached is a patch that appears to make a reasonable improvement in
performance, when using both my slightly more advanced VM kernel, and
also the stuff in -current.  I seem to see a fork() only performance
improvement of about 10% on a 2 processor SMP PPro, using lmbench.  On
vfork (which isn't completely implemented on a PPro, but is still faster
than fork), the improvement appears to be about 5%.

Of course, any page coloring improvement is dependent on alot of factors,
but the missing object coloring handling is a problem...

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.
Index: vm/vm_object.c
===
RCS file: /local/home/ncvs/src/sys/vm/vm_object.c,v
retrieving revision 1.144
diff -r1.144 vm_object.c
215a216
   kmem_object-pg_color = (kernel_object-pg_color + PQ_L2_SIZE/4)  
 PQ_L2_MASK;
945a947
   result-pg_color = (source-pg_color + OFF_TO_IDX(*offset))  
 PQ_L2_MASK;


Re: aio_read panics SMP kernel

1999-02-04 Thread John S. Dyson
Brian Dean said:
 
 I'm using a dual 350MHz Dell Precision 410 with 4.0-19990130-SNAP (SMP
 enabled) to prototype a program that uses asynchronous read and write
 (aio_read() and aio_write()), and found that the following simple and
 not very useful program (it's for demonstration purposes only!) causes
 the system to do one of three things:
 
The VM code needs to be upgraded to support proper threading and shared
address space operations under SMP.  That is a major missing feature, and
probably something that should be higher priority than it currently is.

I have experimental work to support it (and alot of other interesting things),
but it will probably be quite a while before those who are currently working
on the code will be able to add the capability.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message


Re: more about yield() versus sched_yield()

1999-02-04 Thread John S. Dyson
Richard Seaman, Jr. said:
 
 As I indicated to you several weeks ago, I think it is possible to have
 a priority inversion problem in spinlocks that spin on sched_yield.
 The yield call, as implemented, partly addresses the issue.  However,
 as you commented to me, it does so poorly.
 
 If the consensus is to drop or modify SYS_yield, my vote would be to
 give consideration to changing the call to more correctly address the
 priority inversion issue.  Possibly a yield (pid_t pid) where the 
 pid passed in is the pid of the thread (process) holding a lock.  The
 system could then check the credentials of the pid to see if there
 is permission (eg. its a kernel thread of the same process) and if
 so, raise the priority of pid to that of the calling process for one
 time slice so pid has a chance to execute and free the lock.  
 
I didn't add the system call to the library (yet), because wasn't sure if
it was working the way that I wanted.  I do suggest that there be only one
(yield) type system call, and if special behavior is desired, then give an
argument describing the special behavior.

I suggest that the yield that is kept should be the one where there is a
current maintainer for it :-).

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message


Re: swap_page_getswapspace failed (don't do stupid things with /dev/mem)

1999-02-04 Thread John S. Dyson
Matthew Dillon said:
 :Matt,
 :
 :Does datasize limit the number of backed pages, or the amount of address
 :space used by a process?  I.e., can I grow myself a large chunk of address
 :space using mmap to the same region of a file, and then read into that
 :large chunk (presumably larger than the cache size if I want to be nasty)?
 :If datasize only affects backed pages, then we can still do nasty large
 :copies; if it affects address space, then nasty large copies are limited
 :to the size of the writable address space (if using readv) or the size of
 :the largest contiguous writable space (if using read).
 :
 :  Robert N Watson 
 :
 :rob...@fledge.watson.org  http://www.watson.org/~robert/
 :PGP key fingerprint: 03 01 DD 8E 15 67 48 73  25 6D 10 FC EC 68 C1 1C
 
 The 'datasize' limit does not effect mmap(), only brk/sbrk.  So,
 in fact, I believe you can bypass the datasize limit by allocating
 anonymous memory using mmap().
 
 This is probably a bug.  We should either limit the mmap()able space to 
 about the same size as the data segment limit, or keep track of the
 amount of anonymous mapped memory and count that in the datasize
 limit.
 
FYI, the information that provides the virtual space used by a map (process)
is in the vm_map data structure (so you don't need to reinvent it.)  It is
now used by PS also.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message


Re: Memory usage weirdness

1999-01-27 Thread John S. Dyson
Dan Root said:
Content-Description: Mail message

 
 Is this normal, or should I look for some process that's thrashing through
 vast amounts of pages in short periods of time?

It is normal and expected.  A little secret about FreeBSD's VM is that
it works on a page demand type timeclock and not an arbitrary real time
timeclock.  This helps the system (partially) to be self-tuning.

-- 
John  | Never try to teach a pig to sing,
dy...@iquest.net  | it makes one look stupid
jdy...@nc.com | and it irritates the pig.

To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message