Re: FIO* doc added to tty.4 (review)

2001-04-17 Thread John W. De Boskey

Hi,

   I wondered about this. However, after looking around what
I found was that the FIO* requests seemed to be documented
in driver pages (and only a handful at that), and not
documented with ioctl.

   Since I don't want to change the other existing driver
man pages, what if I add generic documentation to the
ioctl along with a comment saying to look at the individual
driver man pages for information specific to the driver.

ie: FIONREAD will succeed from a backgrounded process, but the
data read itself with hang (tty driver).


Thanks,
John

ps: Is cross-referencing #define values frowned upon? It would
be nice if "man -k FIONREAD" (for instance) would return
useful results.

- Garrett Wollman's Original Message -
 On Sun, 15 Apr 2001 22:26:37 -0700, "John W. De Boskey" [EMAIL PROTECTED] said:
 
 I've been doing some tty related work and found the FIO*
  requests don't seem to be documented.
 
 I've added some reasonable doc to tty.4 and put up the
  the diff and html forms:
 
 As the initial letter suggests, they are generic file ioctls and not
 specific to the teletype driver.  They should be described in the
 ioctl(2) manual page.
 
 -GAWollman
 
 
 To Unsubscribe: send mail to [EMAIL PROTECTED]
 with "unsubscribe freebsd-current" in the body of the message

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Doug Barton

Matt Dillon wrote:

 It is not implying that at all.  There is no black and white here.
 This is a case where spending a huge amount of time and complexity
 to get the efficiency down to the Nth degree is nothing but a waste
 of time.  What matters is what the user sees, what performance
 the application gets, and how many bugs you introduce when optimizing
 something that might not need optimizing.

Sooo. sounds like making "on" the default in -current is a worthy
experiment?

Doug
-- 
Perhaps the greatest damage the American system of education has done
to its children is to teach them that their opinions are relevant
simply because they are their opinions.

Do YOU Yahoo!?

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein

* Doug Barton [EMAIL PROTECTED] [010417 01:08] wrote:
 Matt Dillon wrote:
 
  It is not implying that at all.  There is no black and white here.
  This is a case where spending a huge amount of time and complexity
  to get the efficiency down to the Nth degree is nothing but a waste
  of time.  What matters is what the user sees, what performance
  the application gets, and how many bugs you introduce when optimizing
  something that might not need optimizing.
 
   Sooo. sounds like making "on" the default in -current is a worthy
 experiment?

More like I'm damn tired of having to remeber to turn it on all my
machines in order to get decent performance when doing large amount
of directory ops.

I'm figuring the only time when it may be a problem is on machines
with a small amount of memory.  Since memory is cheap, I plan on
turning it on within the next couple of days unless a stability
issue comes up.

I'll leave it to those people with low memory to remember to turn
it off.

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Daemon News Magazine in your snail-mail! http://magazine.daemonnews.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein

* Matt Dillon [EMAIL PROTECTED] [010415 23:16] wrote:
 
   For example, all this work on a preemptive
 kernel is just insane.  Our entire kernel is built on the concept of
 not being preemptable except by interrupts.  We virtually guarentee
 years of instability and bugs leaking out of the woodwork by trying to
 make it preemptable, and the performance gain we get for that pain
 is going to be zilch.  Nada.  Nothing.

Pre-emption is mearly a side effect of a mutex'd kernel.

The actual gains are in terms of parallel execution internally.
Meaning if we happen to copyin() a 4 meg buffer we can allow more
than one process to be completing some sort of work inside the
kernel other than spinning on the giant lock.

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Instead of asking why a piece of software is using "1970s technology,"
start asking why software is ignoring 30 years of accumulated wisdom.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Doug Barton

Alfred Perlstein wrote:
 
 I'm figuring the only time when it may be a problem is on machines
 with a small amount of memory.  Since memory is cheap, I plan on
 turning it on within the next couple of days unless a stability
 issue comes up.
 
 I'll leave it to those people with low memory to remember to turn
 it off.

OK... this brings up the question of what other cool optimizations are
there that may have been disabled in the past for reasons that are no
longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file
with commented out examples of configurations for various systems. For
example,

# For more modern systems that have a reasonable amount of RAM
#vfs.vmiodirenable=1

# Low memory systems

# Systems that need lots of randomness

# Low resource systems that need less randomness

# Super high performance TCP options for various situations

 etc. I'm sure y'all can come up with more.


It might also be desirable to put these in etc/defautls/rc.conf, but I
think something of this nature might be better suited in a freer format.

Doug
-- 
Perhaps the greatest damage the American system of education has done
to its children is to teach them that their opinions are relevant
simply because they are their opinions.

Do YOU Yahoo!?

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



No Subject

2001-04-17 Thread Storms of Perfection

subscribe freebsd-current


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: sysctl optimisations (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Brian Somers

   OK... this brings up the question of what other cool optimizations are
 there that may have been disabled in the past for reasons that are no
 longer pertinent? It might be worthwhile to create an /etc/sysctl.conf file
 with commented out examples of configurations for various systems. For
 example,
 
 # For more modern systems that have a reasonable amount of RAM
 #vfs.vmiodirenable=1
 
 # Low memory systems
 
 # Systems that need lots of randomness
 
 # Low resource systems that need less randomness
 
 # Super high performance TCP options for various situations
 
  etc. I'm sure y'all can come up with more.
 
 
 It might also be desirable to put these in etc/defautls/rc.conf, but I
 think something of this nature might be better suited in a freer format.

I would have thought a bunch of comments in /etc/sysctl.conf would 
be sufficient.

 Doug
 -- 
 Perhaps the greatest damage the American system of education has done
 to its children is to teach them that their opinions are relevant
 simply because they are their opinions.
 
   Do YOU Yahoo!?

-- 
Brian [EMAIL PROTECTED]brian@[uk.]FreeBSD.org
  http://www.Awfulhak.org   brian@[uk.]OpenBSD.org
Don't _EVER_ lose your sense of humour !



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ISO image available?

2001-04-17 Thread Clive Lin

Sorry for replying this topic, a little bit OT this time :-)

On Thu, Apr 12, 2001 at 09:20:34AM -0700, Lars Eggert wrote:
 Michael Johnson wrote:
  Theres not a iso for -CURRENT .. It changes too much.
 
 Too bad.
  
  you can make your own iso though. ports/sysutils/mkisofs
 
 Yes, I've done that before for -stable, but it involves a make world :-)

Relax, all you need is something like mirror/ncftp/lftp which could do
recursive grabbing and, well, /usr/share/examples/worm/makecdfs.sh :-)

sh /usr/share/examples/worm/makecdfs.sh would show a simple usage
message. That's all. If you want a bootable iso (of course you want
it), add a -b flag, which is only documented in the script itself :

 Grabbing an ISO from somewhere and quickly doing a CD install to test some
 bugfixes would be much faster. (I'm not that interested in actively
 tracking -current; I just want to be able to quickly run it whenever
 someone asks for feedback on a change that'd affect our setup.)
 
 Isn't someone out there doing a nightly scripted make world? How about
 doing a make release after?
 
 Lars

I did make release before if I simply want a bootable and up-to-date
iso image, but it costs too much time. : IMHO, unless you want a
customized, locally hacked cvs repository become a iso, you don't have
to expend time on making release.

Last time (2 days ago) I packed a releng_4 iso to install FreeBSD on an
old box, which has no workable NIC, no usable floppy, only a 12X
CDROM. The time consumes for make/burn the iso is around 10 minutes :-)

-- 
pub  1024D/F8D2B472 2000-08-05 Clive Lin [EMAIL PROTECTED]
 Key fingerprint = 7F9D 57A8 55C7 AA18 49B5  3820 570B 27F6 F8D2 B472

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



** HEADS UP ** Groff 1.17 (including -mdocNG) imported

2001-04-17 Thread Ruslan Ermilov

Hi!

It is my great pleasure to announce the availability of just released
Groff 1.17.  Please refer to the src/contrib/groff/NEWS for details on
what's new in this release.

This release includes the complete re-implementation of the BSD mdoc(7)
macro package, earlier referred to as -mdocNG.  The new mdoc(7) manpage
completely documents the package, thus replacing mdoc.samples(7).  For
now, mdoc.samples(7) is installed as MLINK to mdoc(7), but I will likely
replace it with a short manpage referring to the mdoc(7).  For a complete
set of new features, please refer to the new mdoc(7).  Note that this
manpage could only be displayed with the new groff tools and new -mdoc
package.

FreeBSD-specific -mdoc configuration file is called mdoc.local.  This
version also provides a simple way to localize -mdoc.  For now, only
KOI8-R support is implemented.  See gnu/usr.bin/groff/tmac/koi8-r for
details.

Below is the brief info on -mdoc from the NEWS file:

: o The mdoc package has been completely rewritten, using the full power of
:   GNU troff to remove limitations of Unix troff (which is no longer
:   supported).  Most important changes are:
: 
:   . No argument limit.
:   . Almost all macros are parsed and callable (if it makes sense).
:   . `.Lb': prints library names
:   . `.Nm punctuation' now works as expected; `.Nm "" punctuation' has
: been withdrawn.
:   . Updated `.St' command
:   . `.Fx': prints FreeBSD
:   . `.Ox': prints OpenBSD
:   . `.Bsx': prints BSD/OS
:   . `.Brq', `.Bro', `.Brc': brace enclosure macros
:   . `.Bd -centered': center lines
:   . `.Bl -xwidth string': interpret string and use the resulting width
:   . support for double-sided printing (-rD1 command line switch)
:   . support for 11pt and 12pt document sizes (-rS11, -rS12 command line
: switches)
: 
:   `groff_mdoc.man' replaces `groff_mdoc.samples.man'; it now completely
:   documents the mdoc package.
: 
:   Great care has been taken to assure backwards compatibility.  If you
:   encounter any abnormal results, please report them to [EMAIL PROTECTED]



Cheers,
-- 
Ruslan Ermilov  Oracle Developer/DBA,
[EMAIL PROTECTED]   Sunbay Software AG,
[EMAIL PROTECTED]  FreeBSD committer,
+380.652.512.251Simferopol, Ukraine

http://www.FreeBSD.org  The Power To Serve
http://www.oracle.com   Enabling The Information Age

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ** HEADS UP ** Groff 1.17 (including -mdocNG) imported

2001-04-17 Thread Andrey A. Chernov

On Tue, Apr 17, 2001 at 16:33:18 +0300, Ruslan Ermilov wrote:

 FreeBSD-specific -mdoc configuration file is called mdoc.local.  This
 version also provides a simple way to localize -mdoc.  For now, only
 KOI8-R support is implemented.  See gnu/usr.bin/groff/tmac/koi8-r for
 details.

Is this message means that import is finished? :-)
ISO-8859-1 support was implemented too, at least in old version.
BTW, where Russian hyphentation now gone?

-- 
Andrey A. Chernov
http://ache.pp.ru/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ** HEADS UP ** Groff 1.17 (including -mdocNG) imported

2001-04-17 Thread Robert Watson

On Tue, 17 Apr 2001, Ruslan Ermilov wrote:

 It is my great pleasure to announce the availability of just released
 Groff 1.17.  Please refer to the src/contrib/groff/NEWS for details on
 what's new in this release. 

This is great news -- especially the argument limit removal :-).  Thanks
for all your work on this; all these commits over the last half a year
look like they were a serious time investment, and are much appreciated.

Thanks,

Robert N M Watson FreeBSD Core Team, TrustedBSD Project
[EMAIL PROTECTED]  NAI Labs, Safeport Network Services




To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: kernel panic in -current, ithread or newcard related ?

2001-04-17 Thread John Baldwin


On 17-Apr-01 Jesper Skriver wrote:
 On Mon, Apr 16, 2001 at 08:10:37PM -0700, John Baldwin wrote:
 
 On 15-Apr-01 Jesper Skriver wrote:
  About every other time I boot my IBM ThinkPad 600E I get this panic
  (hand typed, as I don't have a second machine here to be able to use a
  serial console).
  
  Fatal trap 12: page fault while in kernel mode
  Fault virtual address = 0x28
 
 It's a null pointer dereference.  If you've compiled a debug kernel then do
 'gdb -k /usr/obj/usr/src/sys/TAM2/kernel.debug' and then do
 'l *csa_readio+0x17' to find the offending line.  It's usually pretty easy
 to
 figure out then.
 
 I's not obvious to me, newbee in kernel debugging, how is the below
 (from the trace) related ?
 
 /Jesper
 
 (kgdb) l *csa_readio+0x17
 0xc0159cd3 is in csa_readio (machine/bus_at386.h:205).
 200 }
 201
 202 static __inline u_int32_t
 203 bus_space_read_4(bus_space_tag_t tag, bus_space_handle_t handle,
 204  bus_size_t offset)
 205 {
 206 #if defined(_I386_BUS_PIO_H_)
 207 #if defined(_I386_BUS_MEMIO_H_)
 208 if (tag == I386_BUS_SPACE_IO)
 209 #endif

Hmm, well, looking in dev/sound/pci/csa.c at the csa_readio() function,
bus_space_read_4() is called once:

if (offset  BA0_AC97_RESET)
return bus_space_read_4(rman_get_bustag(resp-io), rman_get_bush
andle(resp-io), offset)  0x;
else {
if (csa_readcodec(resp, offset, ul))
ul = 0;
return (ul);
}

My guess is that resp is NULL here.  At this point, you may want to poke
Cameron Grant [EMAIL PROTECTED] with a bug report as he is Mr. Sound and he
probably knows what has gone wrong at this point.

-- 

John Baldwin [EMAIL PROTECTED] -- http://www.FreeBSD.org/~jhb/
PGP Key: http://www.baldwin.cx/~john/pgpkey.asc
"Power Users Use the Power to Serve!"  -  http://www.FreeBSD.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



mount_smbfs missing, SAMBA package installed, where is it?

2001-04-17 Thread estair

I'm running 5.0-Current-4.12.2001, X-Kern-Developer with all binaries,
have just installed the samba package (pkg_add -r samba) but the smb
client services, specifically mount_smbfs, aren't installed.  

I cannot find reference anywhere to which package the mount_smbfs binary
belongs to.  Can anyone point me in the right direction?

Thanks,

Eli Stair



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Snapshot Log - current broke

2001-04-17 Thread David Wolfskill

Date: Tue, 17 Apr 2001 12:34:39 -0700 (PDT)
From: John Baldwin [EMAIL PROTECTED]


=== usr.sbin/pcvt/vttest
cc -O -pipe  -traditional -DUSEMYSTTY   -I/usr/obj/usr/src/i386/usr/include -c
/usr/src/usr.sbin/pcvt/vttest/main.c
cc -O -pipe  -traditional -DUSEMYSTTY   -I/usr/obj/usr/src/i386/usr/include -c
/usr/src/usr.sbin/pcvt/vttest/esc.c
gzip -cn /usr/src/usr.sbin/pcvt/vttest/vttest.1  vttest.1.gz
In file included from /usr/src/usr.sbin/pcvt/vttest/header.h:26,
 from /usr/src/usr.sbin/pcvt/vttest/esc.c:1:
/usr/obj/usr/src/i386/usr/include/stdio.h:302: syntax error before `char'
In file included from /usr/src/usr.sbin/pcvt/vttest/header.h:26,
 from /usr/src/usr.sbin/pcvt/vttest/main.c:20:
/usr/obj/usr/src/i386/usr/include/stdio.h:302: syntax error before `char'
*** Error code 1
*** Error code 1
2 errors
...

Whacking src/include/stdio.h thus:

Index: include/stdio.h
===
RCS file: /cvs/freebsd/src/include/stdio.h,v
retrieving revision 1.32
diff -u -u -r1.32 stdio.h
--- include/stdio.h 2001/04/17 07:59:521.32
+++ include/stdio.h 2001/04/17 17:54:57
@@ -299,7 +299,7 @@
 __BEGIN_DECLS
 int asprintf __P((char **, const char *, ...)) __printflike(2, 3);
 char   *ctermid_r __P((char *));
-const char *fmtcheck __P((const char *, const char *))
+__const char *fmtcheck __P((const char *, const char *))
__attribute__((__format_arg__(2)));
 char   *fgetLn __P((FILE *, size_t *));
 int fpurge __P((FILE *));


has allowed the "make buildworld" to succeed for me.  Building the
kernel as I type; above was hand-transcribed (machine is not networked
at the moment).  Use at your own risk; void where prohibited, etc

Cheers,
david
-- 
David H. Wolfskill  [EMAIL PROTECTED]
As a computing professional, I believe it would be unethical for me to
advise, recommend, or support the use (save possibly for personal
amusement) of any product that is or depends on any Microsoft product.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



BAD BUG: Second try

2001-04-17 Thread Terry Lambert

Oops.

NOTE: I don't follow this lists for weeks at a time, so please
include me directly in any responses.  Thanks.

Matt Dillon was looking at this, but I haven't heard from him
for a while on it.

Here is a patch to make it panic, instead of really, really
trashing memory (ignore the version, I'm using a vendor import
locally); the patch is to "crfree() and should be obvious:

===
diff -c -r1.2 kern_prot.c
*** kern/kern_prot.c2001/03/21 02:33:03 1.2
--- kern/kern_prot.c2001/04/17 02:22:48
***
*** 1001,1006 
--- 1001,1009 
  int s;
  
  s = splhigh();
+   if ( cr-cr_ref == 0) {
+   panic("Freeing already free credential!\n");
+   }
if (--cr-cr_ref == 0) {
/*
 * Some callers of crget(), such as nfs_statfs(),
===


Unfortunately, There's also a nameidata structure (it's the
only data structure that's exactly 72 bytes long, which I was
able to determine by printing sizeof() information for all
kernel structures, and gre'ping for "72") getting freed and
then either continued to be used, or being used as a result
of an unchecked allocation failure (I'm still looking for
that one).

Basically, the second causes invariants to whine about data
modified on the freelist to my console, while the first one
results in an eventual panic dues to spammed memory (for the
obvious reason that you can't free the same thing twice).

The problems only become obvious when you open and then close
around 30,000 TCP connections; sometimes it takes a couple of
tries before it panics your machine.

I have some programs that demonstrate the bug, if anyone is
interested in repeating it on their machines locally (you will
need appropriate open file limits and bump up to 40,000 or so
net.inet.ip.portrange.last, e.g.:

sysctl -w net.inet.ip.portrange.last=45000

Which means your box will need about a gig of memory.


Terry Lambert
[EMAIL PROTECTED]
---
Any opinions in this posting are my own and not those of my present
or previous employers.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon

:
:There's actually very little code that non-premptable once we get the
:kernel mutexed.  The least complex way to accomplish this is to only
:preempt kernel processes that hold no mutex (low level) locks.
:
:-- 
:-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]

I wish it were that simple.   Remember when we went to shared file
descriptor arrays for processes?  Half a dozen serious bugs were 
introduced into the kernel from that change which weren't fixed for
over a year and half.  And that was *WITH* a Giant lock left intact.
A change to a single subsystem introduced 6+ serious bugs.  

Now you guys are introducing preemptive kernel scheduling and expect
it will be a walk in the park?  I expect it will introduce so many bugs
that the system will be destabilized for over a year, because that sort
of thing effects every single subsystem rather then just one or two.

It is not going to be as simple as throwing mutexes around every little
piece of code that might access shared state somewhere.  How many
mutex calls are we going to wind up making to implement the simplest 
system call now?  How much overhead are we going to introduce to support
a feature that isn't going to add one iota of performance to the system?

An incredible number of very cool optimizations can be made when you
have the ability to assume that the cpu you are running under will not
get ripped out from under you.  Things like per-cpu caching - not only
of mbufs or memory, but of other things as well.  Simple access to 
per-cpu globals now must suddenly be done with compare-exchange or
otherwise be required to use atomic instructions.   None of the current 
per-cpu variable handling code assumes that.

It's just not a good idea.  Do you want people to adopt -current when it
becomes 5.0?  It isn't going to happen if preemptive kernel scheduling
is in there.  I am going to wind up having to support 4.x for the next
two years before 5.x becomes reliable enough to use in a production 
environment.  Or, worse, -current is going to move down a path of 
no-return (becoming so unreliable that it will not be adopted by anyone)
and we are going to wind up having to scrap the whole damn thing!

I was somewhat skeptical about BSDI's interrupt-as-thread methodology, 
but at least they didn't bite of more then they could chew when they
implemented it.  Trying to extend that model to the Nth degree is
creating a huge mess -- and for what?  Just to support interrupts as
threads?  It isn't worth it.

I would much rather see us changing the interrupt model to be less
invasive, thus allowing interrupts to run unhindered as interrupts 
rather then threads, rather then see this preemptive kernel scheduling
stuff go in.  I love SMP, but I love stability more.  BSDI focused on
the things that actually effected performance.  -current is not focused
at all and is going off the deep end.. trying to do things that would
be interesting work for 5.2 or 5.3 but just plain should not be in 5.0.

-Matt


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein

You need to settle dude, pre-emption isn't a goal, it's mearly a 
_possible_ side effect.

We're not aiming for pre-emption, we're aiming for more concurrancy.

* Matt Dillon [EMAIL PROTECTED] [010417 13:51] wrote:
 :
 :There's actually very little code that non-premptable once we get the
 :kernel mutexed.  The least complex way to accomplish this is to only
 :preempt kernel processes that hold no mutex (low level) locks.
 :
 :-- 
 :-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
 
 I wish it were that simple.   Remember when we went to shared file
 descriptor arrays for processes?  Half a dozen serious bugs were 
 introduced into the kernel from that change which weren't fixed for
 over a year and half.  And that was *WITH* a Giant lock left intact.
 A change to a single subsystem introduced 6+ serious bugs.  
 
 Now you guys are introducing preemptive kernel scheduling and expect
 it will be a walk in the park?  I expect it will introduce so many bugs
 that the system will be destabilized for over a year, because that sort
 of thing effects every single subsystem rather then just one or two.
 
 It is not going to be as simple as throwing mutexes around every little
 piece of code that might access shared state somewhere.  How many
 mutex calls are we going to wind up making to implement the simplest 
 system call now?  How much overhead are we going to introduce to support
 a feature that isn't going to add one iota of performance to the system?
 
 An incredible number of very cool optimizations can be made when you
 have the ability to assume that the cpu you are running under will not
 get ripped out from under you.  Things like per-cpu caching - not only
 of mbufs or memory, but of other things as well.  Simple access to 
 per-cpu globals now must suddenly be done with compare-exchange or
 otherwise be required to use atomic instructions.   None of the current 
 per-cpu variable handling code assumes that.
 
 It's just not a good idea.  Do you want people to adopt -current when it
 becomes 5.0?  It isn't going to happen if preemptive kernel scheduling
 is in there.  I am going to wind up having to support 4.x for the next
 two years before 5.x becomes reliable enough to use in a production 
 environment.  Or, worse, -current is going to move down a path of 
 no-return (becoming so unreliable that it will not be adopted by anyone)
 and we are going to wind up having to scrap the whole damn thing!
 
 I was somewhat skeptical about BSDI's interrupt-as-thread methodology, 
 but at least they didn't bite of more then they could chew when they
 implemented it.  Trying to extend that model to the Nth degree is
 creating a huge mess -- and for what?  Just to support interrupts as
 threads?  It isn't worth it.
 
 I would much rather see us changing the interrupt model to be less
 invasive, thus allowing interrupts to run unhindered as interrupts 
 rather then threads, rather then see this preemptive kernel scheduling
 stuff go in.  I love SMP, but I love stability more.  BSDI focused on
 the things that actually effected performance.  -current is not focused
 at all and is going off the deep end.. trying to do things that would
 be interesting work for 5.2 or 5.3 but just plain should not be in 5.0.
 
   -Matt

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Represent yourself, show up at BABUG http://www.babug.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon


:
:You need to settle dude, pre-emption isn't a goal, it's mearly a 
:_possible_ side effect.
:
:We're not aiming for pre-emption, we're aiming for more concurrancy.

A goal of having more concurrency is laudable, but I think you are 
ignoring the costs of doing task switches verses the likely spin time
for a mutex.  The whole point of using fine-grained Mutexes is to not
have significant performance-effecting collisions in the first place,
so why bother to try to task switch if you wind up spining in one? 
The goal for Giant is to get rid of it, so why bother to implement 
preemption for Giant?  The goal of taking an interrupt is to be able to
take several interrupts in parallel on different cpu's, and they can't
block anyway, so why try to turn interrupts into real threads?  It just
doesn't make sense, Alfred.  You guys are trying to optimize things that
don't need optimizing and my fear is that you will introduce so many
bugs into the kernel that it will take us years to get it back to 4.x's
level of reliability.  The goals I see bandied about in the lists are
at cross-purposes with each other.

-Matt


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FIO* doc added to tty.4 (review)

2001-04-17 Thread Garrett Wollman

On Wed, 18 Apr 2001 06:40:38 +1000 (EST), Bruce Evans [EMAIL PROTECTED] said:

 Does POSIX now specify select() and/or poll() precisely enough to
 show that the current behaviour is wrong?

In addition to more explicit requirements for sockets, draft 6 has the
following to say about select() and pselect():

A descriptor shall be considered ready for reading when a call
to an input function with O_NONBLOCK clear would not block,
whether or not the function would transfer data
successfully. (The function might return data, an end-of-file
indication, or an error other than one indicating that it is
blocked, and in each of these cases the descriptor shall be
considered ready for reading.)

The socket semantic requirements come from 1003.1g-2000; this
paragraph looks to have come from XSH4.2 (SUSv1).

-GAWollman


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Rik van Riel

On Tue, 17 Apr 2001, Matt Dillon wrote:

 :You need to settle dude, pre-emption isn't a goal, it's mearly a 
 :_possible_ side effect.
 :
 :We're not aiming for pre-emption, we're aiming for more concurrancy.
 
 A goal of having more concurrency is laudable, but I think you are 
 ignoring the costs of doing task switches verses the likely spin time
 for a mutex.  The whole point of using fine-grained Mutexes is to not
 have significant performance-effecting collisions in the first place,
 so why bother to try to task switch if you wind up spining in one? 

Exactly.

It makes absolutely no sense to optimise mutexes for the case
of having contention.  If you have contention, you need to fix
the contention, not introduce all kinds of singing-dancing cool
things you can do when contention happens.

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein

* Matt Dillon [EMAIL PROTECTED] [010417 14:07] wrote:
 
 :
 :You need to settle dude, pre-emption isn't a goal, it's mearly a 
 :_possible_ side effect.
 :
 :We're not aiming for pre-emption, we're aiming for more concurrancy.
 
 A goal of having more concurrency is laudable, but I think you are 
 ignoring the costs of doing task switches verses the likely spin time
 for a mutex.  The whole point of using fine-grained Mutexes is to not
 have significant performance-effecting collisions in the first place,
 so why bother to try to task switch if you wind up spining in one? 
 The goal for Giant is to get rid of it, so why bother to implement 
 preemption for Giant?  The goal of taking an interrupt is to be able to
 take several interrupts in parallel on different cpu's, and they can't
 block anyway, so why try to turn interrupts into real threads?  It just
 doesn't make sense, Alfred.  You guys are trying to optimize things that
 don't need optimizing and my fear is that you will introduce so many
 bugs into the kernel that it will take us years to get it back to 4.x's
 level of reliability.  The goals I see bandied about in the lists are
 at cross-purposes with each other.

You seem to be more concerned with the implementation rather than
the goal.

The goal is to have a kernel that's able to have more concurrancy,
things like pre-emption and task switching on mutex collisions can
be examined and possibly changed later.

Once the mutexes are in place the underlying implementation can
change pretty easily from task switching always to only task
switching when the mutex is owned by the same CPU that I'm running
on. (to avoid spinlock deadlock)

I agree that task switching _might_ be a performance hit, however
that can be fixed by only task switching when in interrupt context.

A lot of things remain to be seen, but only if people like you sit
down and start working with the SMP team to achieve the main goal,
which is making the kernel MP safe for concurrant execution by
locking down the appropriate structures and code paths.

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Instead of asking why a piece of software is using "1970s technology,"
start asking why software is ignoring 30 years of accumulated wisdom.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: BAD BUG: Second try

2001-04-17 Thread Matt Dillon

:Oops.
:
:NOTE: I don't follow this lists for weeks at a time, so please
:include me directly in any responses.  Thanks.
:
:Matt Dillon was looking at this, but I haven't heard from him
:for a while on it.
:
:Here is a patch to make it panic, instead of really, really
:trashing memory (ignore the version, I'm using a vendor import
:locally); the patch is to "crfree() and should be obvious:

Sorry, ran out of time on the weekend.  It's still on my TODO list.

-Matt

:===
:diff -c -r1.2 kern_prot.c
:*** kern/kern_prot.c   2001/03/21 02:33:03 1.2
:--- kern/kern_prot.c   2001/04/17 02:22:48
:***
:*** 1001,1006 
:--- 1001,1009 
:  int s;
:  
:  s = splhigh();
:+  if ( cr-cr_ref == 0) {
:+  panic("Freeing already free credential!\n");
:+  }
:   if (--cr-cr_ref == 0) {
:   /*
:* Some callers of crget(), such as nfs_statfs(),
:===
:
:
:Unfortunately, There's also a nameidata structure (it's the
:only data structure that's exactly 72 bytes long, which I was
:able to determine by printing sizeof() information for all
:kernel structures, and gre'ping for "72") getting freed and
:then either continued to be used, or being used as a result
:of an unchecked allocation failure (I'm still looking for
:that one).
:
:Basically, the second causes invariants to whine about data
:modified on the freelist to my console, while the first one
:results in an eventual panic dues to spammed memory (for the
:obvious reason that you can't free the same thing twice).
:
:The problems only become obvious when you open and then close
:around 30,000 TCP connections; sometimes it takes a couple of
:tries before it panics your machine.
:
:I have some programs that demonstrate the bug, if anyone is
:interested in repeating it on their machines locally (you will
:need appropriate open file limits and bump up to 40,000 or so
:net.inet.ip.portrange.last, e.g.:
:
:   sysctl -w net.inet.ip.portrange.last=45000
:
:Which means your box will need about a gig of memory.
:
:
:   Terry Lambert
:   [EMAIL PROTECTED]
:---
:Any opinions in this posting are my own and not those of my present
:or previous employers.
:
:To Unsubscribe: send mail to [EMAIL PROTECTED]
:with "unsubscribe freebsd-current" in the body of the message
:


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Matt Dillon


:Once the mutexes are in place the underlying implementation can
:change pretty easily from task switching always to only task
:switching when the mutex is owned by the same CPU that I'm running
:on. (to avoid spinlock deadlock)

That makes *NO* *SENSE* Alfred!   So the first step is to introduce
every single possible feature under the sun, requiring massive reworking
of the code, even if it means the system becomes massively unstable,
inefficient, and starts crashing and burning, and then only *later* we
remove the features that don't pan out?  That is the development model
you are using for -current?

:I agree that task switching _might_ be a performance hit, however
:that can be fixed by only task switching when in interrupt context.

WILL be a performance hit.  WILL introduce major bugs.  IS unnecessary,
DOESN'T make any sense whatsoever, is at CROSS PURPOSES with goals
already stated (not having any serious contention in the first place),
REQUIRES massive changes to the code with not a chance in hell of 
producing an equivalent performance improvement for the trouble.

There is no 'remains to be seen' here Alfred.  This is setting up
-current for a fall, one that could be entirely avoided *now*
if you people would sit down and start thinking about what you are
doing.

Remember when Dyson introduced ten thousand cool things all at once
and didn't debug them?  Remember how long it took DG to the system
back into some semblence of shape after that?  Remember that, even after
all that hard work, there were still literally hundreds of hard to find
bugs that DG couldn't find that it took Alan, DG, and I a year's worth
of work beyond all the work DG had already done to cleanup?  Is that
what you want to happen to current?  Because that is where current is
headed right now.

-Matt


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: miibus/fxp intel etherexpress broken

2001-04-17 Thread Darren Henderson


Thanks, the patch appears to work fine. 

Sorry for the delay in testing it, long weekend here.

-Darren

On Fri, 13 Apr 2001, Andrew Gallatin wrote:

 
 Darren Henderson writes:
   
   I had a functional install of -current which was created back on Jan 11th of
   this year. 
   
   I cvsup'd yesterday (4/12. I had also tried this on 4/4) fixxed what I
   needed too from UPDATING and RELNOTES, built and installed world and a new
   kernel.
   
   fxp now fails.
   
   ata1: at 0x170 irq 15 on atapci0
 ...
   fxp0: Intel Pro 10/100B/100+ Ethernet port 0x5400-0x541f mem 
0x6000-0x600f,0x6010-0x60100fff irq 15 at device 11.0 on pci0
   fxp0: could not map interrupt
   device_probe_and_attach: fxp0 attach returned 6
   isa0: unexpected small tag 14
 
 
 This has nothing to do with fxp.  The problem is that irq15 is shared
 between ata1 and fxp0  the ata driver doesn't want to share
 interrupts because some older boards will somehow barf when 
 sharing irqs between ata and other devices.
 
 Try this patch (I haven't tested it myself)
 
 Index: dev/ata/ata-pci.c
 ===
 RCS file: /home/ncvs/src/sys/dev/ata/ata-pci.c,v
 retrieving revision 1.3
 diff -u -r1.3 ata-pci.c
 --- dev/ata/ata-pci.c 2001/03/19 13:31:58 1.3
 +++ dev/ata/ata-pci.c 2001/04/13 18:32:21
 @@ -508,7 +508,7 @@
  
   return BUS_ALLOC_RESOURCE(device_get_parent(dev), child,
 SYS_RES_IRQ, rid,
 -   irq, irq, 1, flags  ~RF_SHAREABLE);
 +   irq, irq, 1, flags);
  #endif
   }
   else {
 
 
 Drew
 --
 Andrew Gallatin, Sr Systems Programmerhttp://www.cs.duke.edu/~gallatin
 Duke University   Email: [EMAIL PROTECTED]
 Department of Computer SciencePhone: (919) 660-6590
 





Darren Henderson  [EMAIL PROTECTED]
[EMAIL PROTECTED]


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread E.B. Dreger

 Date: Tue, 17 Apr 2001 14:52:06 -0700
 From: Alfred Perlstein [EMAIL PROTECTED]

Disclaimer: I am not a kernel hacker.

 The goal is to have a kernel that's able to have more concurrancy,

Right...

 things like pre-emption and task switching on mutex collisions can
 be examined and possibly changed later.

I think that Matt is saying, why bother?  Task switches are expensive.
With a heavily loaded system, the amount of time spent arbitrating mutexes
would exceed spin time.  With a lightly loaded system, do we really care
that much about spinning?

 Once the mutexes are in place the underlying implementation can
 change pretty easily from task switching always to only task
 switching when the mutex is owned by the same CPU that I'm running

I'm not sure that I follow... if the task switch is on the same CPU, then
why do we need any new structures?  Without a global memory cache*, I
think that keeping processes on the same CPU is the first goal, anyway,
and increased concurrency second.

* Does Intel have any sort of cache coherency mechanism, where a processor
can tell other CPUs, "hey, this line is dirty" and the others retrieve the
new info as needed?  The Alpha?

 on. (to avoid spinlock deadlock)
 
 I agree that task switching _might_ be a performance hit, however
 that can be fixed by only task switching when in interrupt context.

Ughh.  AFAIK, traditional wisdom holds that interrupt loops should be as
short as possible.  Assuming that one performs quick operations, DMA
transfers, etc., preemptive task switching *would* be expensive.

 A lot of things remain to be seen, but only if people like you sit
 down and start working with the SMP team to achieve the main goal,
 which is making the kernel MP safe for concurrant execution by
 locking down the appropriate structures and code paths.

Let's look at userland code.  I don't use async I/O because I don't want
to screw with interrupts and callbacks.  Polling of large structures?
Ughh.  Kernel queues are the solution, and what totally sold me on
FreeBSD.

How about basing things on *cooperative* multitasking?  All else being
equal, cooperative is faster because of lower overhead.  What makes co-op
break down is greedy programs that monopolize the CPU... but there should
not be any code like that in the kernel.

My instinct (whatever it's worth; remember my disclaimer) is that co-op
switching using something like tsleep() and wakeup_one() or similar would
be more efficient than trying to screw with mutexes.

However, perhaps that could be improved upon:  Use something a la kqueue
for portions of the kernel to say "I'm waiting for condition XYZ".  When
we wakeup_one(), we specify "for condition XYZ".  We could then avoid
waking processes that would only go back to sleep again.

I hope that I'm not too far out of my league, here... :-)


Eddy

---

Brotsman  Dreger, Inc.
EverQuick Internet / EternalCommerce Division

Phone: (316) 794-8922

---


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread Alfred Perlstein

* Matt Dillon [EMAIL PROTECTED] [010417 15:00] wrote:
 
 :Once the mutexes are in place the underlying implementation can
 :change pretty easily from task switching always to only task
 :switching when the mutex is owned by the same CPU that I'm running
 :on. (to avoid spinlock deadlock)
 
 That makes *NO* *SENSE* Alfred!   So the first step is to introduce
 every single possible feature under the sun, requiring massive reworking
 of the code, even if it means the system becomes massively unstable,
 inefficient, and starts crashing and burning, and then only *later* we
 remove the features that don't pan out?  That is the development model
 you are using for -current?

No I am not, what I'm trying to do is _use_ the APIs given to me to
make the kernel more concurrent.

I see no reason to abandon the external API, they are perfectly fine.

You may disagree with the underlying implementation of the
API and I think you may be right.

  (although afaik we're basing it on both Solaris and BSD/os's
  implementation so... well I'm not going to bother defending it.)

I just wish you'd sit down and try to work adapting the VM to these
APIs

You're not going to get an arguement out of me about the underlying
implementation of the mutexes as I really don't care right now.

What I am going to argue is that argueing about the underlying
implementation is a waste of time that could be better spent
using the API to gain more concurrancy.

 :I agree that task switching _might_ be a performance hit, however
 :that can be fixed by only task switching when in interrupt context.
 
 WILL be a performance hit.  WILL introduce major bugs.  IS unnecessary,
 DOESN'T make any sense whatsoever, is at CROSS PURPOSES with goals
 already stated (not having any serious contention in the first place),
 REQUIRES massive changes to the code with not a chance in hell of 
 producing an equivalent performance improvement for the trouble.
 
 There is no 'remains to be seen' here Alfred.  This is setting up
 -current for a fall, one that could be entirely avoided *now*
 if you people would sit down and start thinking about what you are
 doing.

No, sitting down and spending many months hashing out the underlying
mechnisms of what should be a pretty opaque API is a waste of time and
will stagnate developement at much too great a cost.

In fact, we sorta spent about 3 years doing it already haven't we? :)

 
 Remember when Dyson introduced ten thousand cool things all at once
 and didn't debug them?  Remember how long it took DG to the system
 back into some semblence of shape after that?  Remember that, even after
 all that hard work, there were still literally hundreds of hard to find
 bugs that DG couldn't find that it took Alan, DG, and I a year's worth
 of work beyond all the work DG had already done to cleanup?  Is that
 what you want to happen to current?  Because that is where current is
 headed right now.

Actually back then I wasn't that much of a kernel hacker, don't
you remeber me standing there at SF BAFUG while you, Julian and
Terry were discussing cache-coherency issues?  I sort of had to
take a break from listening to the lot of you because my brain went
into overload, I was 19 or 20 at the time. :)

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Represent yourself, show up at BABUG http://www.babug.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Snapshot Log - current broke

2001-04-17 Thread Bruce Evans

On Tue, 17 Apr 2001, David Wolfskill wrote:

 Date: Tue, 17 Apr 2001 12:34:39 -0700 (PDT)
 From: John Baldwin [EMAIL PROTECTED]
 ...
 In file included from /usr/src/usr.sbin/pcvt/vttest/header.h:26,
  from /usr/src/usr.sbin/pcvt/vttest/main.c:20:
 /usr/obj/usr/src/i386/usr/include/stdio.h:302: syntax error before `char'
 *** Error code 1
 *** Error code 1
 2 errors
 ...
 
 Whacking src/include/stdio.h thus:
 
 Index: include/stdio.h
 ===
 RCS file: /cvs/freebsd/src/include/stdio.h,v
 retrieving revision 1.32
 diff -u -u -r1.32 stdio.h
 --- include/stdio.h 2001/04/17 07:59:521.32
 +++ include/stdio.h 2001/04/17 17:54:57
 @@ -299,7 +299,7 @@
  __BEGIN_DECLS
  int   asprintf __P((char **, const char *, ...)) __printflike(2, 3);
  char *ctermid_r __P((char *));
 -const char *fmtcheck __P((const char *, const char *))
 +__const char *fmtcheck __P((const char *, const char *))
   __attribute__((__format_arg__(2)));
  char *fgetLn __P((FILE *, size_t *));
  int   fpurge __P((FILE *));
 
 has allowed the "make buildworld" to succeed for me.  Building the
 kernel as I type; above was hand-transcribed (machine is not networked
 at the moment).  Use at your own risk; void where prohibited, etc

This is because vttest is compiled with a KRish compiler (gcc
-traditional) for some reason, and the hard `const' doesn't work for
KR compilers.  The `const's inside __P(()) don't cause problems
because they are soft -- __P(()) defines them away as part of defining
everything inside the inner parentheses away, although this is bogus
since prototypes work for `gcc -traditional'.

Bruce


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FW: Filesystem gets a huge performance boost

2001-04-17 Thread E.B. Dreger

 Date: Tue, 17 Apr 2001 22:18:34 + (GMT)
 From: E.B. Dreger [EMAIL PROTECTED]
 
 My instinct (whatever it's worth; remember my disclaimer) is that co-op
 switching using something like tsleep() and wakeup_one() or similar would
 be more efficient than trying to screw with mutexes.

Oops.  I meant asleep().


Eddy

---

Brotsman  Dreger, Inc.
EverQuick Internet / EternalCommerce Division

Phone: (316) 794-8922

---


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Swap partition panic on -current

2001-04-17 Thread Storms of Perfection

Does anyone know of a possible bug/panic with the swap partition? I tried 
to view a file with vi/pico on -current (10+M file) and it crashed the machine.



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



mounting netware fs

2001-04-17 Thread User

Hello everybody,

 first of all sorry if this is not the proper list for writing this. This is my first 
posting and I'm new to FreeBSD, so please forgive me if I am sending crap in this 
message. 

 I needed to install Netware filesystem support and noticed that if I want to mount 
the nwfs via /etc/fstab, I have to change the /etc/rc file so that the system won't 
try to mount the nwfs before there is networking support started. So that's why I had 
to change /etc/rc on line 258 from:

# Mount everything except nfs filesystems.
mount -a -t nonfs

to

# Mount everything except nfs filesystems.
mount -a -t nonfs,nwfs

In the first case at boot time the machine stops and wants me to enter in shell and 
fix the problem. I don't know whether I'm right, but I think that my modification can 
help.

Boris Georgiev

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



More on Bad Bug

2001-04-17 Thread Terry Lambert

NB: To keep me in the loop, keep me in the Cc: list, since I
rarely follow -current these days).

I've further localized the bug to the freeing of credentials
associated with socket buffers, and being unrelated to the
crhold()/crfree() calls in socreate() and sodealloc().

Specifically, the bug exhibits as INVARIANTS complaining about
a 72 byte malloc -- it seems that in avoiding system call
structures, I neglected struct rusage -- which is exactly 72
bytes in size, as well.

There seems to be some bad code in soo_close(), which looks like:

int
soo_close(fp, p)
struct file *fp;
struct proc *p;   
{
int error = 0;

fp-f_ops = badfileops;
if (fp-f_data)
error = soclose((struct socket *)fp-f_data);
fp-f_data = 0; 
return (error);
}

It seems to me this should be?

int
soo_close(fp, p)
struct file *fp;
struct proc *p;   
{
int error = 0;

if (fp-f_data)
error = soclose((struct socket *)fp-f_data);
if (!error) {
fp-f_data = 0; 
fp-f_ops = badfileops;
}
return (error);
}

But it's not clear that this is correct for the socket code.

The INVARIANTS code is actually part of the problem here, since the
reference count on the credential is one of the fields stomped by
the WIERD_ADDR "to be safer".

This doesn't evidence as a problem with a double free in the case
that INVARIANTS aren't used, in that the decremented count in the
crfree() (which decrements prior to examining the value to see if
it is exactly zero), would continue to be non-zero, and not decrement
back to zero (resulting in the double free).

Credentials are actually used _AMAZINGLY_ much; it seems that they
are a good candidate for some optimization to throw away references
that aren't really necessary (for example, it seems to me that a
socket can not exist without an fdp referencing it, and the fdp has
a reference count on the cred which the socket inherits from the fdp,
so the fdp's reference protects the sockets reference, and so the
socket's reference doesn't really need to be reference counted).

In any case, I'm leaving in the panic patch I sent earlier, and am
now rebuilding with my ucred reference count moved past the area
stomped by INVARIANTS.  This should permit the INVARIANTS to catch
double frees (which they can't, otherwise, because of the refcnt
decrement making it look like the block was used after being freed,
and tricking INVARIANTS in free() in kern_malloc.c.

I guess no one else is interested in this bug hunt, or no one else
is using 30,000 sockets on any of their machines?


Terry Lambert
[EMAIL PROTECTED]
---
Any opinions in this posting are my own and not those of my present
or previous employers.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Greg Lehey

On Tuesday, 17 April 2001 at  1:19:57 -0700, Alfred Perlstein wrote:
 * Matt Dillon [EMAIL PROTECTED] [010415 23:16] wrote:

   For example, all this work on a preemptive
 kernel is just insane.  Our entire kernel is built on the concept of
 not being preemptable except by interrupts.  We virtually guarentee
 years of instability and bugs leaking out of the woodwork by trying to
 make it preemptable, and the performance gain we get for that pain
 is going to be zilch.  Nada.  Nothing.

 Pre-emption is mearly a side effect of a mutex'd kernel.

 The actual gains are in terms of parallel execution internally.
 Meaning if we happen to copyin() a 4 meg buffer we can allow more
 than one process to be completing some sort of work inside the
 kernel other than spinning on the giant lock.

*sigh* Couldn't you have changed the subject line when discussing
something of this importance?

Greg
--
Finger [EMAIL PROTECTED] for PGP public key
See complete headers for address and phone numbers

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Alfred Perlstein

* Greg Lehey [EMAIL PROTECTED] [010417 17:02] wrote:
 On Tuesday, 17 April 2001 at  1:19:57 -0700, Alfred Perlstein wrote:
  * Matt Dillon [EMAIL PROTECTED] [010415 23:16] wrote:
 
For example, all this work on a preemptive
  kernel is just insane.  Our entire kernel is built on the concept of
  not being preemptable except by interrupts.  We virtually guarentee
  years of instability and bugs leaking out of the woodwork by trying to
  make it preemptable, and the performance gain we get for that pain
  is going to be zilch.  Nada.  Nothing.
 
  Pre-emption is mearly a side effect of a mutex'd kernel.
 
  The actual gains are in terms of parallel execution internally.
  Meaning if we happen to copyin() a 4 meg buffer we can allow more
  than one process to be completing some sort of work inside the
  kernel other than spinning on the giant lock.
 
 *sigh* Couldn't you have changed the subject line when discussing
 something of this importance?

I wasn't discussing, I was explaining.

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Represent yourself, show up at BABUG http://www.babug.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Bosko Milekic


On Tue, Apr 17, 2001 at 05:47:23PM -0700, Matt Dillon wrote:
 Proposed:
 
   mainline kernel {
   get_spin_mutex(somemutex);
   .
   .
   masked interrupt occurs, interrupt structure contains array
   of mutexes the interrupt will need.  Check said mutexes, one
   found to be held by current cpu.  Set interrupt-pending bit
   in mutex and iret immediately.

You cannot be pre-empted by an interrupt if you are holding a spin
 mutex, AFAIK, even under present implementation.

   .
   .
   release_spin_mutex(somemutex)
   (bit found to be set in mutex, triggers interrupt reissuing code)
   }
 
 And there you have it.  The mutex/array test is takes very little time
 being a read-only test that requires no bus locking, and the collision
 case is cheap also because the current cpu already owns the mutex, allowing
 us to set the interrupt-pending bit in that mutex without any bus
 locking.  The check during the release of the mutex is two instructions,
 no bus locking required.  The whole thing can be implemented without any
 additional bus locking and virtually no contention.
 
 The case could be further optimized by requiring that interrupts only
 use a single mutex, period.  This would allow the mainline interrupt
 routine to obtain the mutex on entry to the interrupt and allow the 
 reissuing code to reissue the interrupt without freeing the mutex that
 caused the reissue, so the mutex is held throughout and then freed by
 the interrupt itself.

 Holy shit.  I think that's it!  I don't think it can get much better then
 that.  It solves all of BDE's issues, solves the interrupt-as-thread
 issue (by not using threads for interrupts at all), and removes a huge
 amount of unnecessary complexity from the system.  We could even get rid
 of the idle processes if we wanted to.

What happens if we get an interrupt, we're thinking about servicing
it, about to check whether we're already holding a mutex that may potentially
be used inside the mainline int routine, and another CPU becomes idle?
In this particular case, let's say that we decide that we have to set
ipending and iret immediately, because we're already holding a potential
lock when we got interrupted. Isn't the result that we have a second
CPU idling while we just set ipending? (I could be missing something, really).
Also, some mainline interrupt code may need to acquire a really
large number of locks, but only in some cases. Let's say we have to first
check if we have a free cached buffer sitting somewhere, and if not, malloc()
a new one. Well, the malloc() will eventually trigger a chain of mutex
lock operations, but only in the case where we lack the cached buffer to
allocate it. There is no practical way of telling up front whether or not
we'll have to malloc(), so I'm wondering how efficiently we would be able
to predict in cases such as these?

   -Matt

Cheers,
-- 
 Bosko Milekic
 [EMAIL PROTECTED]


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Alfred Perlstein

* Matt Dillon [EMAIL PROTECTED] [010417 17:47] wrote:
...
 
 Interrupts by definition know precisely what they are going to do, so by
 definition they know precisely which mutexes (if any) they may need
 to get.  This means that, in fact, it is possible to implement a check
 to determine if any of the mutexes an interrupt might want to get are
 already being held by the SAME cpu or not, and if they are to do the
 equivalent of what our delayed-interrupt stuff does in the stable's
 spl/splx code, but instead do the check when a mutex is released.
 
 The result is:  No need for an idle process to support interrupt
 contexts, no need to implement interrupts as threads, and no need
 to implement fancy scheduling tricks or Giant handling.
 
...
 And there you have it.  The mutex/array test is takes very little time
 being a read-only test that requires no bus locking, and the collision
 case is cheap also because the current cpu already owns the mutex, allowing
 us to set the interrupt-pending bit in that mutex without any bus
 locking.  The check during the release of the mutex is two instructions,
 no bus locking required.  The whole thing can be implemented without any
 additional bus locking and virtually no contention.
 
 The case could be further optimized by requiring that interrupts only
 use a single mutex, period.  This would allow the mainline interrupt
 routine to obtain the mutex on entry to the interrupt and allow the 
 reissuing code to reissue the interrupt without freeing the mutex that
 caused the reissue, so the mutex is held throughout and then freed by
 the interrupt itself.
 
 Holy shit.  I think that's it!  I don't think it can get much better then
 that.  It solves all of BDE's issues, solves the interrupt-as-thread
 issue (by not using threads for interrupts at all), and removes a huge
 amount of unnecessary complexity from the system.  We could even get rid
 of the idle processes if we wanted to.

We can switch to this mechism at a later date.

There's issues here though:

  Mutex creation can be expensive as it seems like each interrupt
  needs to register what sort of mutex it's interested in, when a
  mutex is created the list must be scanned and each interrupt
  updated.

  Interrupts do not know "exactly" which mutexes they will need, they
  know about a subset of the mutexes they may need, this scheme causes
  several problems:
1) interrupts are again fan-in, meaning if you block an interrupt
class on one cpu you block them on all cpus
2) when we may have things like per-socket mutexes we are blocking
interrupts that may not need twiddling by the interrupt handler,
yet we need to block the interrupt anyway because it _may_ want
the same mutex that we have.

  Windriver has a full time developer working on the existing
  implementation, as far as I know we can only count on you for
  weekends and spare time.

  I'm starting to feel that I'm wasting time trying to get you to
  see the bigger picture; the fact that niether system means diddly
  unless we get to work on locking the rest of the kernel.

With that said, I'd really like to see the better of the two schemes
implemented when the dust settles.  The problem is that right now
neither scheme is buying us much other than overhead without
signifigant parts of the kernel being converted over to a mutexed
system.

Your proposal is valueable and might be something that we switch
to, however for the time being it's far more important to work on
locking down subsystems than working on the locking subsystem.

In fact if you proposed a new macro wrapper for mtx_* that would
make it easier at a later date to implement _your_ version of the
locking subsystem I would back it just to get you interested in
participating in locking down the other subsystems.

-Alfred

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Matt Dillon

:
:   You cannot be pre-empted by an interrupt if you are holding a spin
: mutex, AFAIK, even under present implementation.

Since spin mutexes are going to be held all over the place, this 
type of restriction would seem to be detrimental.  If you can do all
the checking up-front it is also unnecessary.

:   What happens if we get an interrupt, we're thinking about servicing
:it, about to check whether we're already holding a mutex that may potentially
:be used inside the mainline int routine, and another CPU becomes idle?
:In this particular case, let's say that we decide that we have to set
:ipending and iret immediately, because we're already holding a potential

You could, but you are talking about a very small window of opportunity
assuming that we are running similar code to what we have now in regards
to assigning the interrupts to a cpu.  In 4.x we assign interrupts to
whichever cpu is holding Giant.  With mutexes we would want to assign
interrupts to whichever cpu is idle and if no cpu is idle we round-robin
the interrupt assignments (e.g. cpu takes interrupt, assigns all 
interrupts to the next cpu if no cpus are idle).  With that in place,
the best course is almost certainly going to be to do nothing ... that is,
take the interrupt even though it might not be optimal.

If once every X thousand interrupts we happen to hit a case where a cpu
remains idle when it doesn't have to be, who gives a flying f**k if
that one interrupt is non-optimal?  I'm not kidding... that sort of
case is not a problem that needs to be solved.

:   Also, some mainline interrupt code may need to acquire a really
:large number of locks, but only in some cases. Let's say we have to first
:check if we have a free cached buffer sitting somewhere, and if not, malloc()

None of our current interrupt code needs to aquire a huge number of
locks, and if some piece of interrupt code is so complex that it does,
it should be relegated to a software interrupt (e.g. like the TCP stack).
Lets not create problems where they don't exist.  If one of our subsystems
happened to be require more complexity - for example, the I/O completion
handling (biodone() code), it's a solveable problem.

Simplification is what is needed here.  Creating a complex solution to
a complex problem only results in a mess.  Simplifying the problem so
that it covers most of your codebase and then focusing on the one or two
cases it doesn't cover would seem to be a better way of dealing with
the issue.

:lock operations, but only in the case where we lack the cached buffer to
:allocate it. There is no practical way of telling up front whether or not
:we'll have to malloc(), so I'm wondering how efficiently we would be able
:to predict in cases such as these?
:
:Cheers,
:-- 
: Bosko Milekic

It could very well be that for an interrupt we might need to list 
two mutexes --- one for the memory subsystem and one for the interrupt's
subsystem.  It would be nice if we could get away with one, but having
to list two or even three mutexes would not be much of a burden on
the interrupt code.

-Matt


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread E.B. Dreger

 Date: Tue, 17 Apr 2001 21:20:45 -0400
 From: Bosko Milekic [EMAIL PROTECTED]
 
 What happens if we get an interrupt, we're thinking about servicing
 it, about to check whether we're already holding a mutex that may
 potentially be used inside the mainline int routine, and another CPU
 becomes idle? In this particular case, let's say that we decide that we
 have to set ipending and iret immediately, because we're already holding
 a potential lock when we got interrupted. Isn't the result that we have
 a second CPU idling while we just set ipending? (I could be missing
 something, really).

(Thinking hard... this is fun stuff...)

 Also, some mainline interrupt code may need to acquire a really large
 number of locks, but only in some cases. Let's say we have to first
 check if we have a free cached buffer sitting somewhere, and if not,
 malloc() a new one. Well, the malloc() will eventually trigger a chain
 of mutex lock operations, but only in the case where we lack the cached
 buffer to allocate it. There is no practical way of telling up front
 whether or not we'll have to malloc(), so I'm wondering how efficiently
 we would be able to predict in cases such as these?

In this case, why not have a memory allocator similar to Hoard?

Let's say that I have a four-way system with 256 MB.  First CPU gets first
64 MB, second gets the next 64 MB, and so on.  Now we needn't lock before
malloc(), because each CPU knows ahead of time what is "off limits".

When one reaches a high water mark, it steals half the available space
from the CPU with the least memory utilization.  This _would_ require a
lock, but should only happen in rare instances.

I know that memory could become fragmented over time, but as long as we
don't screw up caching (which shouldn't be a problem considering that
pages are much larger than cache lines), who cares?


Eddy

---

Brotsman  Dreger, Inc.
EverQuick Internet / EternalCommerce Division

Phone: (316) 794-8922

---


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: More on Bad Bug

2001-04-17 Thread Alfred Perlstein

* Terry Lambert [EMAIL PROTECTED] [010417 16:28] wrote:
 
 I guess no one else is interested in this bug hunt, or no one else
 is using 30,000 sockets on any of their machines?

I've committed a variation of your invariants check to -current,
-stable is frozen and I'd prefer to leave it as is until after
the release date.

As far as tracking down the problem I don't have the resources in
terms of hardware, code and time for that right now, however it
looks like you've found a possible bug.

Please keep us in the loop on this.

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
http://www.egr.unlv.edu/~slumos/on-netbsd.html

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Alfred Perlstein

* E.B. Dreger [EMAIL PROTECTED] [010417 18:36] wrote:
 
 In this case, why not have a memory allocator similar to Hoard?

It doesn't work, but it's close:

http://people.freebsd.org/~alfred/memcache/

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Represent yourself, show up at BABUG http://www.babug.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Alfred Perlstein

* E.B. Dreger [EMAIL PROTECTED] [010417 18:48] wrote:
  Date: Tue, 17 Apr 2001 18:28:40 -0700
  From: Alfred Perlstein [EMAIL PROTECTED]
  
  1) interrupts are again fan-in, meaning if you block an interrupt
  class on one cpu you block them on all cpus
 
 When would this be a bad case?  i.e., if an interrupt [class] must be
 blocked, would we not it blocked across the board?

It'd be nice if you had something like 16 nic cards working
independantly of each other to not be in the same collision domain
if they don't have to.

-- 
-Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]]
Daemon News Magazine in your snail-mail! http://magazine.daemonnews.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a huge performance boost)

2001-04-17 Thread Matt Dillon


:  Mutex creation can be expensive as it seems like each interrupt
:  needs to register what sort of mutex it's interested in, when a
:  mutex is created the list must be scanned and each interrupt
:  updated.

The list is based in the interrupt structure.  The cost is, what,
four or five instructions in a loop for which the vast majority will
only have to iterate once.  All the operations are read-only unless
you get a hit.  Very cheap.  It would be nice if the list could be
fixed to one or two items... same number of instructions, but no loop,
fewer memory accesses, and cheaper to execute.

The only interrupts we care about in regards to the efficiency of this 
design are:  network interrupts and I/O interrupts, yes?  Network
interrupts can get away with one or two mutexes (memory and queue, or
perhaps even just memory).  I/O interrupts are a stickier issue but
up until softupdates the only thing biodone() did was release a lock
already held, so it wouldn't be an issue.  I think softupdates relegates
nearly all of its work to a software interrupt or process so softupdates
would not represent a problem either.  I'd have to review it.  I made one
change to the VM system in 4.x which was to free swap indirectly from
biodone which I would have to rip out, but that would pretty much be it.

:  Interrupts do not know "exactly" which mutexes they will need, they
:  know about a subset of the mutexes they may need, this scheme causes
:  several problems:
:1) interrupts are again fan-in, meaning if you block an interrupt
:class on one cpu you block them on all cpus

They don't have to be.  If you have four NICs each one can be its own
interrupt, each with its own mutex.  Thus all four can be taken in
parallel.  I was under the impression that BSDI had achieved that
in their scheme.

If you have one NIC then obviously you can't take multiple interrupts
for that one NIC on different cpu's.  No great loss, you generally don't
want to do that anyway.

:2) when we may have things like per-socket mutexes we are blocking
:interrupts that may not need twiddling by the interrupt handler,
:yet we need to block the interrupt anyway because it _may_ want
:the same mutex that we have.

Network interrupts do not mess around with sockets.  The packets are
passed to a software interrupt level which is certainly a more heavyweight
entity.  I can be argued very easily that the TCP stack should operate
as a thread -- actually, one thread for each cpu, so if you have a lot
of TCP activity you can activate several threads and process TCP
operations in parallel.  (IRIX did this to good effect).

Nobody should ever do complex processing in an interrupt, period.  If
you need to do complex processing, you do it in a software interrupt
(in -stable), or a thread in the new design.

:  Windriver has a full time developer working on the existing
:  implementation, as far as I know we can only count on you for
:  weekends and spare time.

Doesn't effect the discussion, really.  It's nice that people are dedicated
to the project.  I wish someone were in charge of it, like Linus is
in charge of Linux.  When my time frees up (A year from now?  Less?  More?
I don't know).. when my time frees up I am going to start working from
whatever platform is the most stable.  If 5.x isn't stable by that time
it's probably hopeless and I'll have to start work from the 4.x base.  If
5.x is stable then I'll be able to start from 5.x.

I know that sounds harsh, but it's a realistic view.  I truely do not
believe that SMPifying things needs to be this difficult, if only
people would focus on the things that matter and stop trying to throw
the kitchen sink into -current (especially without adequate testing).
That's my beef with current.  I find it ironic that I was shot down for
not following the BSDI mutex model in the name of compatibility when I
did that first push, but when other people started messing with the 
system compatibility went flying right out the window.  Very ironic.

:neither scheme is buying us much other than overhead without
:signifigant parts of the kernel being converted over to a mutexed
:system.
:
:Your proposal is valueable and might be something that we switch
:to, however for the time being it's far more important to work on
:locking down subsystems than working on the locking subsystem.
:
:In fact if you proposed a new macro wrapper for mtx_* that would
:make it easier at a later date to implement _your_ version of the
:locking subsystem I would back it just to get you interested in
:participating in locking down the other subsystems.
:
:-Alfred

I wasn't really proposing a new macro wrapper, it was just pseudo
code.  If I were doing mutexes from scratch I would scrap all the
fancy features and just have spin 

Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread E.B. Dreger

 Date: Tue, 17 Apr 2001 19:06:08 -0700 (PDT)
 From: Matt Dillon [EMAIL PROTECTED]
 
 They don't have to be.  If you have four NICs each one can be its own
 interrupt, each with its own mutex.  Thus all four can be taken in
 parallel.  I was under the impression that BSDI had achieved that
 in their scheme.

IIRC, didn't the NT driver for some NIC (Intel?) switch to polling,
anyway, under heavy load?  The reasoning being that you _know_ that you're
going to get something... why bother an IRQ hit?

That said, IRQ distribution sounds like a good thing for the general case.

 If you have one NIC then obviously you can't take multiple interrupts
 for that one NIC on different cpu's.  No great loss, you generally don't
 want to do that anyway.

Actually, I should think that one would _want_ to serialize traffic for a
given NIC.  (I'm ignoring when one trunks NICs... speaking of which,
anyone have info on 802.3ad? ;-)  Otherwise, one ends up with a race that
[potentially] screws up packet sequence.


Eddy

---

Brotsman  Dreger, Inc.
EverQuick Internet / EternalCommerce Division

Phone: (316) 794-8922

---


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread Matt Dillon


:IIRC, didn't the NT driver for some NIC (Intel?) switch to polling,
:anyway, under heavy load?  The reasoning being that you _know_ that you're
:going to get something... why bother an IRQ hit?
:
:That said, IRQ distribution sounds like a good thing for the general case.

Under a full load polling would work just as well as an interrupt.
With NT for the network tests they hardwired each NIC to a particular
CPU.  I don't know if they did any polling or not.

: If you have one NIC then obviously you can't take multiple interrupts
: for that one NIC on different cpu's.  No great loss, you generally don't
: want to do that anyway.
:
:Actually, I should think that one would _want_ to serialize traffic for a
:given NIC.  (I'm ignoring when one trunks NICs... speaking of which,
:anyone have info on 802.3ad? ;-)  Otherwise, one ends up with a race that
:[potentially] screws up packet sequence.
:
:Eddy

Yes.  Also NICs usually have circular buffers for packets so, really,
only one cpu can be processing a particular NIC's packets at any given
moment.

-Matt

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread E.B. Dreger

 Date: Tue, 17 Apr 2001 19:34:56 -0700 (PDT)
 From: Matt Dillon [EMAIL PROTECTED]
 
 Yes.  Also NICs usually have circular buffers for packets so, really,
 only one cpu can be processing a particular NIC's packets at any given
 moment.

We could always have a mutex for each NIC's ring buffer...

*ducking and running*

Sorry... couldn't resist. :-)


Eddy

---

Brotsman  Dreger, Inc.
EverQuick Internet / EternalCommerce Division

Phone: (316) 794-8922

---


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread janb

 IIRC, didn't the NT driver for some NIC (Intel?) switch to polling,
 anyway, under heavy load?  The reasoning being that you _know_ that you're
 going to get something... why bother an IRQ hit?
THis is very interesting. How does this affect performance?

JAn


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Bad Bug update

2001-04-17 Thread Terry Lambert

Just an update...

] In any case, I'm leaving in the panic patch I sent earlier, and am
] now rebuilding with my ucred reference count moved past the area
 
Moving the reference count of of the range of the stop clears 
up the invariant corruption complaint.
 
It is now abundantly clear that a credential is being freed while
there is still a reference to it.

So either somone is not doing a crhold(), or crfree() is being
called one too many times, when you have a boatload of fd's that
are sockets.


Terry Lambert
[EMAIL PROTECTED]
---
Any opinions in this posting are my own and not those of my present
or previous employers.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread Rik van Riel

On Tue, 17 Apr 2001, Matt Dillon wrote:

 Under a full load polling would work just as well as an interrupt.
 With NT for the network tests they hardwired each NIC to a particular
 CPU.  I don't know if they did any polling or not.

Not true. Interrupts work worse than polling because the interrupt
top halves can keep the CPU busy, whereas with polling you only
peek at the card when you have time.

This means pure interrupts can possibly DoS a CPU (think about a
gigabit ping flood) while polling leaves the box alive and still
allows it to process as much as it can (while not wasting CPU on
taking in packets it cannot process higher up the stack).

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread E.B. Dreger

 Date: Wed, 18 Apr 2001 00:04:12 -0300 (BRST)
 From: Rik van Riel [EMAIL PROTECTED]
 
 Not true. Interrupts work worse than polling because the interrupt
 top halves can keep the CPU busy, whereas with polling you only

Top halves and _task switching_.  Again, in a well-written handler with a
tight loop, task switching becomes expensive.

 peek at the card when you have time.

Think aio_ versus kernel queues. :-)

 This means pure interrupts can possibly DoS a CPU (think about a
 gigabit ping flood) while polling leaves the box alive and still
 allows it to process as much as it can (while not wasting CPU on
 taking in packets it cannot process higher up the stack).

I should hope that the card would be smart enough to combine consecutive
packets into a single DMA transfer, but I know what you mean.


Eddy

---

Brotsman  Dreger, Inc.
EverQuick Internet / EternalCommerce Division

Phone: (316) 794-8922

---


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread E.B. Dreger

Going back to basic principles:

For minimal CPU utilization, it would be nice skip task switching, period.
Run something to completion, then go on to the next task.  Poll without
ever using an interrupt.

The problem is that latency becomes totally unacceptable.

So now let's go to the other extreme:  Create a Transputer-like array with
hundreds of 65xx-complexity CPUs.  Each atomic task runs on its own
private CPU.

The problem is that the electronics become a pain, and are often idle.
When too many tasks are launched, we run out of CPU power.

The compromise is to switch tasks on whatever CPU power is available...
balancing switching overhead with latency.  *Let the latency be as high as
is acceptable to reduce overhead as low as is practical.*

Hence, my philosophy is that task switching and preemption are necessary
evils because hardware does not perfectly accomodate software.  If we
must, we must... otherwise, use co-op switching as the next best thing to
straight run-to-completion.


Eddy

---

Brotsman  Dreger, Inc.
EverQuick Internet / EternalCommerce Division

Phone: (316) 794-8922

---


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: hid_report_size() prototype wrong in both -stable and -current

2001-04-17 Thread Nick Sayer

 
 usb.h != libusb.h
 
 NetBSD has a stupid naming convention. Use libusb.h in FreeBSD.
 
 Ok. That's as may be, but then why are there prototypes for libusb
 calls in dev/usb.h?

Sorry, meant to say /usr/include/dev/usb/hid.h

 
 
 The prototypes are correct as is.
 
 They may be in libusb.h, but the prototype for that function in
sys/dev/usb.h is mysterious.

/usr/include/dev/usb/hid.h



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: hid_report_size() prototype wrong in both -stable and -current

2001-04-17 Thread n_hibma


usb.h != libusb.h

NetBSD has a stupid naming convention. Use libusb.h in FreeBSD.

The prototypes are correct as is. The changes to libusb made in CURRENT
have not been backported to STABLE (yet).

Nick

On Tue, 17 Apr 2001, Nick Sayer wrote:


 The definition of hid_report_size() differs between -stable and -current (as
 to its prototype), yet /usr/src/sys/dev/usb.h shows no difference between
 -stable and -current, and is wrong in either case.

 This is causing some heartburn for -current builders of the xmame port.

 To Unsubscribe: send mail to [EMAIL PROTECTED]
 with "unsubscribe freebsd-current" in the body of the message


-- 
The USB for FreeBSD project.
[EMAIL PROTECTED]
http://www.etla.net/~n_hibma/usb/usb.pl


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: hid_report_size() prototype wrong in both -stable and -current

2001-04-17 Thread Nick Sayer

 
 usb.h != libusb.h
 
 NetBSD has a stupid naming convention. Use libusb.h in FreeBSD.

Ok. That's as may be, but then why are there prototypes for libusb calls in
dev/usb.h?

 
 The prototypes are correct as is.

They may be in libusb.h, but the prototype for that function in
sys/dev/usb.h is mysterious.



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: mounting netware fs

2001-04-17 Thread Boris Popov

On Wed, 18 Apr 2001, User  wrote:

 mount -a -t nonfs
 
 to
 
 # Mount everything except nfs filesystems.
 mount -a -t nonfs,nwfs
 
 In the first case at boot time the machine stops and wants me to enter
 in shell and fix the problem. I don't know whether I'm right, but I
 think that my modification can help.

Well, nwfs and smbfs can be added to the list along with
nfs. However I would recommend to use 'noauto' modifier and the shell
script like one in the /usr/share/examples/nwclient

--
Boris Popov
http://www.butya.kz/~bp/


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: mount_smbfs missing, SAMBA package installed, where is it?

2001-04-17 Thread Boris Popov

On Tue, 17 Apr 2001 [EMAIL PROTECTED] wrote:

 I'm running 5.0-Current-4.12.2001, X-Kern-Developer with all binaries,
 have just installed the samba package (pkg_add -r samba) but the smb
 client services, specifically mount_smbfs, aren't installed.  

mount_smbfs doesn't belong to the Samba package. Take a look at
ports/net/smbfs.

--
Boris Popov
http://www.butya.kz/~bp/


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel preemption, yes or no? (was: Filesystem gets a hugeperformance boost)

2001-04-17 Thread Rik van Riel

On Wed, 18 Apr 2001, E.B. Dreger wrote:

 For minimal CPU utilization, it would be nice skip task switching,
 period. Run something to completion, then go on to the next task.  
 Poll without ever using an interrupt.

[snip]

 Hence, my philosophy is that task switching and preemption are
 necessary evils because hardware does not perfectly accomodate
 software.  If we must, we must... otherwise, use co-op switching as
 the next best thing to straight run-to-completion.

Except that for the [extremely performance critical] interrupt
handlers the "software" is under control of the folks who write
the OS.

You need preemption for userspace because it's possibly "hostile"
software, but things like the interrupt handlers and the kernel
code are under your control ... this means you can code it to be
as efficient as possible without impacting latency too much.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/
http://www.conectiva.com/   http://distro.conectiva.com.br/


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message