Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Jeroen Ruigrok/asmodai

Marc,

-On [20020421 00:30], Marc G. Fournier ([EMAIL PROTECTED]) wrote:

Over the past week, I've been trying to get information on how to fix a
server that panics with:

| panic: vm_map_entry_create: kernel resources exhausted
| mp_lock = 0101; cpuid = 1; lapic.id = 0100
| boot() called on cpu#1

Take a look at this:

http://www.freebsd.org/cgi/getmsg.cgi?fetch=245329+248644+/usr/local/www/db/text/2001/freebsd-hackers/20010624.freebsd-hackers

Hope this helps,

-- 
Jeroen Ruigrok van der Werven / asmodai / Kita no Mono
asmodai@[wxs.nl|xmach.org], finger [EMAIL PROTECTED]
http://www.softweyr.com/asmodai/ | http://www.[tendra|xmach].org/
How many cares one loses when one decides not to be something but to be
someone.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Odd problem with MTRR and ACPI

2002-04-22 Thread David Malone

I have an ASUS A7A266 motherboard with an Athlon XP processor which
seems prone to weirdness. The BIOS seems to set the MTRRs to some
undocumented values, which used to prevent X starting. I've now
fixed the MTRR code and X works fine.

Unfortunately, when X changes the MTRRs then ACPI stops working.
I tracked this down and found that the ACPI data just vanishes out
of memory when you change the MTRRs! (Illustration included below,
including hexdump of the bits of memory in question.)

Has anyone seen anything like this? Does anyone have any idea what
the old MTRR values mean? They are changed from 0x10(=???) to
0x01(=write-combine).

David.

MSR 26e, old=0x1010101010101010 new=0x0101010101010101
MSR 26f, old=0x1010101010101010 new=0x0101010101010101

gonzo 3 # acpidump | head -3
Found sig at f78c0
Checksum OK at f78c0
/*
RSD PTR: Checksum=144, OEMID=ASUS, RsdtAddress=0x17fec000
 */
gonzo 4# dd if=/dev/mem bs=1024 count=1024 | hd -s 0xf78c0
000f78c0  52 53 44 20 50 54 52 20  90 41 53 55 53 20 20 00  |RSD PTR .ASUS  .|
000f78d0  00 c0 fe 17 00 c0 fe 17  40 c0 fe 17 80 c0 fe 17  |@...|
000f78e0  00 c1 fe 17 00 f0 ff 17  00 00 00 00 00 00 00 00  ||
000f78f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
*
000f8000  90 90 90 90 90 90 90 90  e9 41 61 2e 8b c0 8b c0  |.Aa.|
 snipped binary data, which looks like it might be bios continues up to 1M
0010
gonzo 5# memcontrol set -b 983040 -l 65536 -o XFree86 write-combine
gonzo 6# acpidump | head -3
acpidump: Can't find ACPI information
gonzo 7# dd if=/dev/mem bs=1024 count=1024 | hd -s 0xf78c0
000f78c0  0c 00 04 00 00 40 f0 17  01 00 00 00 0b 80 00 00  |.@..|
*
000fc240  0c 00 04 00 00 40 f0 17  01 00 00 00 00 00 00 00  |.@..|
000fc250  0c 00 00 00 00 08 3e ca  01 00 00 00 00 00 00 00  |...|
*
0010

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: new expr(1) behaviour breaks libtool

2002-04-22 Thread Terry Lambert

John Hay wrote:
   I see the new new behaviour of expr(1) requires you to add '--' if your
   commandline arguments might start with a '-'. This does break things
   a little because our old expr(1) does not understand a '--' in the
   beginning and the new one don't work right without it. :-(((
 
  I'm almost positive this issue was discussed before.  Check the follow
  ups to the commit.
 
 The only one I could find was in -current, where Kris asked if w3m or
 expr is to blame and Garrett said w3m is to blame.

Garrett is right.  The expr is being called with non-POSIX
arguments by the w3c scripts.


 H. I can understand the requirement to eat '--', but to throw a
 tantrum just because the commandline started with a '-' is a little
 too much. BTW, was the response posted somewhere? I searched through
 -standards, -commit and -current but couldn't find it. Maybe I just
 didn't ask the right question to the search engine or maybe it was
 in another list.

The correct way to handle the continuing brokenness is to provide
patches that will get applied to libtool.

Also, you complaint about different versionsion of FreeBSD being
different is wrong: POSIX compliant scripts will call the command
with POSIX compliant arguments, regardless of the FreeBSD version.
Just because you can get away with it on older versions of FreeBSD
doesn't mean that the patches shouldn't be applied there, too: they
should, because it's important that the resulting scripts be
standards compliant, even if the particular version of FreeBSD you
are using has an implementation of expr that illegally extends
the standard.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: panic:bremfree with today's current and linux-netscape

2002-04-22 Thread Bruce Evans

On Sun, 21 Apr 2002, Maxim Konovalov wrote:

 As Adrian Penisoara already reported

 http://docs.freebsd.org/cgi/getmsg.cgi?fetch=19645+0+current/freebsd-current

 there is panic in -current. I believe it is related to the next
 commit:

 nectar  2002/04/18 17:45:29 PDT

   Modified files:
 sys/kern kern_descrip.c kern_exec.c
 sys/sys  filedesc.h
   Log:
   When exec'ing a set[ug]id program, make sure that the stdio file descriptors
   (0, 1, 2) are allocated by opening /dev/null for any which are not already
   open.

   Reviewed by:alfred, phk
   MFC after:  2 days

 Here is my workaround but I am not sure is it correct or not. Seems
 falloc() takes care about locking itself.

 Index: src/sys/kern/kern_descrip.c
 ===
 RCS file: /home/ncvs/src/sys/kern/kern_descrip.c,v
 retrieving revision 1.138
 diff -u -r1.138 kern_descrip.c
 --- src/sys/kern/kern_descrip.c   20 Apr 2002 12:02:52 -  1.138
 +++ src/sys/kern/kern_descrip.c   21 Apr 2002 17:04:58 -
 @@ -1528,9 +1528,7 @@
   if (fdp-fd_ofiles[i] != NULL)
   continue;
   if (devnull  0) {
 - FILEDESC_LOCK(fdp);
   error = falloc(td, fp, fd);
 - FILEDESC_UNLOCK(fdp);
   if (error != 0)
   break;
   NDINIT(nd, LOOKUP, FOLLOW, UIO_SYSSPACE, /dev/null,

 %%%

I use the same patch.  Locking here is essentially equivalent to calling
panic() here (except it gives a more confusing panic message :-).

I think opening file descriptors on exec is wrong anyway.  POSIX has
close-on-exec but not open-on-exec.

Bruce


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Terry Lambert

Marc G. Fournier wrote:
  No, there's no stats collected on this stuff, because it's a
  pretty obvious and straight-forward thing: you have to have a
  KVA space large enough that, once you subtract out 4K for each
  4M of physical memory and swap (max 4G total for both), you
  end up with memory left over for the kernel to use, and your
  limits are such that the you don't run out of PTEs before you
  run out of mbufs (or whatever you plan on allocating).
 
 ... and translated to english, this means? :)
 
 Okay, I'm going to assume that I'm allowed 4Gig of RAM + 4Gig of Swap, for
 a total of 8Gig ... so, if I subtract out 4K for each 4M, that is 8M for
 ... what?
 
 So, I've theoretically got 8184M of VM available for the kernel to use
 right now?  what are PTEs and how do I know how many I have right now?  as
 for mbufs, I've currently got:

No.

Each 4M of physical memory takes 4K of statically allocated KVA.
Each 4M of backing store takes 4K of statically allocated KVA.

The definition of backing store includes:

o   All dirty data pages in swap
o   All dirty code pages in swap
o   All clean data pages in files mapped into process or kernel
address space
o   All clean code pages for executables mapped into process or
kernel address space
o   Reserved mappings for copy-on-write pages that haven't yet
been written

A PTE is a page table entry.  It's the 32 bit value in the page
table for each address space (one for the kernel, one per process).
See the books I posted the titles of for more details, or read the
Intel processor PDF's from their developer web site.


 jupiter netstat -m
 173/1664/61440 mbufs in use (current/peak/max):
 77 mbufs allocated to data
 96 mbufs allocated to packet headers
 71/932/15360 mbuf clusters in use (current/peak/max)
 2280 Kbytes allocated to network (4% of mb_map in use)
 0 requests for memory denied
 0 requests for memory delayed
 0 calls to protocol drain routines
 
 So how do I find out where my PTEs are sitting at?

The mbufs are only important because most people allocate a
large number of mbufs up front for networking applications, or
for alrge numbers of users with network applications that will
need resources in order to be able to actually run.  There's
also protocol control blocks and other allocation that occur
up front, based on the maximum number of system open files
and sockets you intend to permit.

The user space stuff is generally a lot easier to calculate:
do a ps -gaxl, round each entry in the VSZ column up to
4M, divide by 4K, and that tells you how many 4K units you
have allocated for user space.  For kernel space, the answer
is that there are some allocated at boot time, (120M worth),
and then the kernel map is grown, as necessary, until it hits
the KVA space limit.  If you plan on using up every byte, then
divide your total KVA space by 4K to get the number of 4K pages
allocated there.

For the kernel stuff... you basically need to know where the
kernel puts how much memory, based on the tuning parameters
you use on it.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



FreeBSD 5.0 Developer Preview #1 Now Available / diskless booting

2002-04-22 Thread Danny Braniss

Haven't tried all the new stuff, but i've installed it on several different
hosts and so far have had no real problems, good work!!

Since im very interested in diskless/dataless, and i've been at it for some
time now, i made some changes to libstand/bootp.c and /etc/rc.diskless1
which i wouldn't mind receiving commnets/suggestions, and if possible
incorporated.

the stuff is in:
ftp://ftp.cs.huji.ac.il/users/danny/freebsd/diskless-boot/

danny




To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Terry Lambert

Marc G. Fournier wrote:
 On Sun, 21 Apr 2002, Terry Lambert wrote:
  No, there's no stats collected on this stuff, because it's a pretty
  obvious and straight-forward thing: you have to have a KVA space large
  enough that, once you subtract out 4K for each 4M of physical memory and
  swap (max 4G total for both), you end up with memory left over for the
  kernel to use, and your limits are such that the you don't run out of
  PTEs before you run out of mbufs (or whatever you plan on allocating).
 
 God, I'm glad its straightforwards :)
 
 Okay, first off, you say (max 4G total for both) ... do you max *total*
 between the two, or phy can be 4g *plus* swap can be 4g for a total of 8g?

You aren't going to be able to exceed 4G, no matter what you do,
because that's the limit of your address space.

If you want more, then you need to use a 64 bit processor (or use a
processor that supports bank selection, and hack up FreeBSD to do
bank swapping on 2G at a time, just like Linux has been hacked up,
and expect that it won't be very useful).

If you are swapping, you are demand paging.

The way demand paging works is that you reference a page that has
been swapped out, or for which physical memory backing store has
not been addigned.

When you make this reference, you get a page not present fault (a
trap 12).  The trap handler puts the faulting process to sleep,
and then starts the process of pulling the page in from backing
store (if it's not a create-on-reference), which, among other
things, locates a physical page to contain the copy of the data
pulled in from the backing store (or zero'ed out of physical memoy,
if it's an unbacked page, e.g. non-swappable, or swappable, but for
which swap has not yet been allocated, because it's the first use).

Only certain types of kernel memory are swappable -- mostly kernel
memory that's allocated on a per process basis.  Kernel swapping
really does you no good, if you have a fully populated physical
memory in the virtual address space, since there's only one kernel
virtual address space (SMP reserves a little bit of per processor
memory, but the amount is tiny: one page descriptor's worth: 4M);
after a certain point, your KVA is committed, and it's a mistake to
have it compete in the same LRU domain as processes.  You can't
really avoid that, for the most part, since there's a shared TLB
cache that you really don't have opportunity to manage, other than
by seperating 4M vs. 4K pages (and 2M, etc., for the Pentium Pro,
though variable page granularity is not supported in FreeBSD, since
it's not common to most hardware people actually have).


 For instance, right now, I have 3Gig of physical and ~3gig of swap
 allocated ...

Each process maintains its own virtual address space.  Almost all
of a process virtual address space is swappable.  So if you are
swapping, it's going to be process address space: UVA, not KVA.

If you increase the KVA, then you will decrease the UVA available to
user processes.  The total of the two can not exceed 4G.


With 4G of physical memory, then 3G of KVA is practically a
requirement, particularly if you intend to use the additional memory
for kernel data (you will have to, for PDE's: you have no choice).
For 3G, it's ~2.5G KVA minimally required.  Personally, I'd just
put it at 3G, and live with it, so you can throw in RAM to your limit
later, when you decide you need to throw RAM at some problem or other.
If you can't afford for the UVA to be as small as 1G, then you are
going to have to make some hard decisions on the amount of physical
RAM you put in the machine.

It's not really that bad: for 3G of KVA, you need 3M for PDE's.  The
problem comes when they are exhausted because of the amount of PDE's
you have lying around to describe UVA pages that are swapped out for
various processes, and for kernel memory requirements that go way up
when you crank up the kernel's ability to handle load (e.g. for network
equipment, I generally take half of physical memory for mbufs, mostly
because that's around the limit of what I can take, and have anything
left over).

That you are using System V shared memory segments is *REALLY* going to
hurt you; each of these shared memory segment comes out of the KVA, so
using shared memory segments with the shm*() calls, rather than using
mmap()'ed files as backing store, can eat huge chunks of KVA, as well
as fragmenting the KVA, particularly over time.

For more details on paged memory management on x86, see:

Protected Mode Software Architecture

and:

The Indispensible PC Hardware Book

You might also want to find a book on bootstrapping protected mode
operating systems (actually, I have yet to find a very good one,
so post about it, if you find one).

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: PAM OpenSSH: two incorrect last login

2002-04-22 Thread Dag-Erling Smorgrav

Bill Fenner [EMAIL PROTECTED] writes:
 You could either pretend that yflag was not set (what ache was
 suggesting) or set yflag if the year of the last login was not
 this year (possibly more useful).

...or always print the year.  Does anybody have any patches?  :)

DES
-- 
Dag-Erling Smorgrav - [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: lock order reversal and panic in kern_descrip.c

2002-04-22 Thread Jun Kuriyama

At Mon, 22 Apr 2002 07:32:39 -0500,
Jacques A. Vidrine [EMAIL PROTECTED] wrote:
 Does it happen every time?  I cannot reproduce it.

Yes.

 What is odd is that fdcheckstd() is only called when exec'ing a
 set[ug]id executable -- any idea what set[ug]id program is being
 exec'd here?

I have no idea about this.  I'm portupgrade'ing linux_base,
linux-netscape-* port and I'll try this again.

  #14 0xc019d01b in fdcheckstd (td=0xe7f8ed50)
  at ../../../kern/kern_descrip.c:1532
  #15 0xc01a04e2 in execve (td=0xe7f8ed50, uap=0xe805bcdc)
  at ../../../kern/kern_exec.c:372
 
 My -CURRENT is a few days old.  I'll see if updating allows me to
 reproduce the problem.

A kern_descrip.c is updated by tanimura after your r1.137.  Could you
try with r1.138?


-- 
Jun Kuriyama [EMAIL PROTECTED] // IMG SRC, Inc.
 [EMAIL PROTECTED] // FreeBSD Project

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: lock order reversal and panic in kern_descrip.c

2002-04-22 Thread Jacques A. Vidrine

On Mon, Apr 22, 2002 at 11:28:41AM +0900, Jun Kuriyama wrote:
 
 Today's -current kernel.  This happend when I invoke linux-netscape.

Does it happen every time?  I cannot reproduce it.

What is odd is that fdcheckstd() is only called when exec'ing a
set[ug]id executable -- any idea what set[ug]id program is being
exec'd here?

 #14 0xc019d01b in fdcheckstd (td=0xe7f8ed50)
 at ../../../kern/kern_descrip.c:1532
 #15 0xc01a04e2 in execve (td=0xe7f8ed50, uap=0xe805bcdc)
 at ../../../kern/kern_exec.c:372

My -CURRENT is a few days old.  I'll see if updating allows me to
reproduce the problem.

Cheers,
-- 
Jacques A. Vidrine [EMAIL PROTECTED] http://www.nectar.cc/
NTT/Verio SME  . FreeBSD UNIX .   Heimdal Kerberos
[EMAIL PROTECTED] .  [EMAIL PROTECTED]  .  [EMAIL PROTECTED]
 


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread David Schultz

Thus spake Terry Lambert [EMAIL PROTECTED]:
 If you want more, then you need to use a 64 bit processor (or use a
 processor that supports bank selection, and hack up FreeBSD to do
 bank swapping on 2G at a time, just like Linux has been hacked up,
 and expect that it won't be very useful).

I'm guessing that this just means looking at more than 4 GB of memory
by working with 2 GB frames at a time.  As I recall, David Greenman
said that this hack would essentially require a rewrite of the VM
system.  Does this just boil down to using 36 bit physical addresses?
Are there plans for FreeBSD to support it, or is everyone just waiting
until 64 bit processors become more common?

 You can't
 really avoid that, for the most part, since there's a shared TLB
 cache that you really don't have opportunity to manage, other than
 by seperating 4M vs. 4K pages (and 2M, etc., for the Pentium Pro,
 though variable page granularity is not supported in FreeBSD, since
 it's not common to most hardware people actually have).

Does FreeBSD use 4M pages exclusively for kernel memory, as in
Solaris, or is there a more complicated scheme?

 If you increase the KVA, then you will decrease the UVA available to
 user processes.  The total of the two can not exceed 4G.

In Linux, all of physical memory is mapped into the kernel's virtual
address space, and hence, until recently Linux was limited to ~3 GB of
physical memory.  FreeBSD, as I understand, doesn't do that.  So is
the cause of this limitation that the top half of the kernel has to
share a virtual address space with user processes?

I'll have to read those books one of these days when I have time(6).
Thanks for the info.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Vizion Communication

Test - Please ignore
- Original Message - 
From: David Schultz [EMAIL PROTECTED]
To: Terry Lambert [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Monday, April 22, 2002 6:09 AM
Subject: Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?


 Thus spake Terry Lambert [EMAIL PROTECTED]:
  If you want more, then you need to use a 64 bit processor (or use a
  processor that supports bank selection, and hack up FreeBSD to do
  bank swapping on 2G at a time, just like Linux has been hacked up,
  and expect that it won't be very useful).
 
 I'm guessing that this just means looking at more than 4 GB of memory
 by working with 2 GB frames at a time.  As I recall, David Greenman
 said that this hack would essentially require a rewrite of the VM
 system.  Does this just boil down to using 36 bit physical addresses?
 Are there plans for FreeBSD to support it, or is everyone just waiting
 until 64 bit processors become more common?
 
  You can't
  really avoid that, for the most part, since there's a shared TLB
  cache that you really don't have opportunity to manage, other than
  by seperating 4M vs. 4K pages (and 2M, etc., for the Pentium Pro,
  though variable page granularity is not supported in FreeBSD, since
  it's not common to most hardware people actually have).
 
 Does FreeBSD use 4M pages exclusively for kernel memory, as in
 Solaris, or is there a more complicated scheme?
 
  If you increase the KVA, then you will decrease the UVA available to
  user processes.  The total of the two can not exceed 4G.
 
 In Linux, all of physical memory is mapped into the kernel's virtual
 address space, and hence, until recently Linux was limited to ~3 GB of
 physical memory.  FreeBSD, as I understand, doesn't do that.  So is
 the cause of this limitation that the top half of the kernel has to
 share a virtual address space with user processes?
 
 I'll have to read those books one of these days when I have time(6).
 Thanks for the info.
 
 To Unsubscribe: send mail to [EMAIL PROTECTED]
 with unsubscribe freebsd-stable in the body of the message
 


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: panic:bremfree with today's current and linux-netscape

2002-04-22 Thread Bruce Evans

On Mon, 22 Apr 2002, Jacques A. Vidrine wrote:

 On Mon, Apr 22, 2002 at 06:25:17PM +1000, Bruce Evans wrote:
  I use the same patch.  Locking here is essentially equivalent to calling
  panic() here (except it gives a more confusing panic message :-).

 :-)  That's a bit of an overstatement; my testing didn't catch it.  I

Yes; I forgot to write the clause about it only being equivalent to a
panic if certain options (mainly INVARIANTS) are configured.

 do see that falloc does lock the file descriptor table too, though ---
 I wonder how it ever worked.

INVARIANTS causes a check of an assertion that non-recursive locks
like fd_mtx aren't actually misused recursively.  The check causes more
panics than the misuse since the misuse is rarely fatal.  I may be
missing something since I rarely use INVARIANTS and haven't checked
that it causes the panic deterministically.

Bruce


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: panic:bremfree with today's current and linux-netscape

2002-04-22 Thread Kris Kennaway

On Mon, Apr 22, 2002 at 12:27:50PM +0300, Adrian Penisoara wrote:

 How-to-repeat: launch linux-netscape while in X11
  
  Are you absolutely sure all kernel modules are up to date?  A
  sure-fire way to cause panics with linux applications is to use an
  out of date linux.ko.
 
  Isn't linux.ko supposed to be rebuilt every time the kernel is
 recompiled ?

Yeah, but some people have NO_MODULES set and forget they're actually
using them.  Anyway, your problem was reported by others so it's
probably real :-)

Kris



msg37521/pgp0.pgp
Description: PGP signature


Re: lock order reversal and panic in kern_descrip.c

2002-04-22 Thread Jacques A. Vidrine

On Mon, Apr 22, 2002 at 10:35:49PM +0900, Jun Kuriyama wrote:
 A kern_descrip.c is updated by tanimura after your r1.137.  Could you
 try with r1.138?

Just updated to today's -CURRENT.  I still cannot reproduce the issue.

I'm curious ... could you send the output of 

  pkg_info -L linux-netscape-navigator-4.79 | xargs ls -l

?

Also, does `/compat/linux/bin/sh' blow up for you?

Cheers,
-- 
Jacques A. Vidrine [EMAIL PROTECTED] http://www.nectar.cc/
NTT/Verio SME  . FreeBSD UNIX .   Heimdal Kerberos
[EMAIL PROTECTED] .  [EMAIL PROTECTED]  .  [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: new expr(1) behaviour breaks libtool

2002-04-22 Thread Garrett Wollman

On Sun, 21 Apr 2002 20:17:29 +0200 (SAT), John Hay [EMAIL PROTECTED] said:

  expr -lgrove : -l\(.*\)
  expr -- -L/export/ports/textproc/jade/work/jade-1.2.1/lib/.libs : -l\(.*\)

  If we are going to leave this behaviour, we will have to teach libtool
  how to call expr(1) differently on -stable and -current and it looks
  like yet again different from the rest of the world. :-(((

No, you just have to teach libtool to properly protect the arguments
to expr(1).  libtool 1.4 does this for some numeric operations, but
not for string parsing.  expr(1) goes into some detail about this, but
the simple answer is:

expr X$arg : 'X-l\(.*\)'

is what's required.  (There is nothing special about the character
`X', other than it not being a hyphen.)

I have just committed code to expr which will cause it to behave more
like the old expr did in the presence of an EXPR_COMPAT environment
variable.  Ports can then be set up to build with this variable
defined until the libtool maintainers fix up their act.

-GAWollman


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: /usr/tmp

2002-04-22 Thread David O'Brien

On Mon, Apr 22, 2002 at 06:03:01PM +0300, Danny Braniss wrote:
 in FreeBSD 5.0 DP1 there is /usr/tmp
   2 drwxrwxrwt  2 root  wheel  512 Apr 11 12:37 /usr/tmp/
 
 any particular reason for this directory? I'm asking because i like to
 keep / and /usr read only, and /tmp is linked to /usr/var/tmp which is 
 writable.

It was created by sysinstall.  One is able to rm it w/o problems.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 5.0 Developer Preview #1 Now Available / diskless booting

2002-04-22 Thread David O'Brien

On Mon, Apr 22, 2002 at 10:40:44AM +0300, Danny Braniss wrote:
 Since im very interested in diskless/dataless, and i've been at it for some
 time now, i made some changes to libstand/bootp.c and /etc/rc.diskless1
 which i wouldn't mind receiving commnets/suggestions, and if possible
 incorporated.
 
 the stuff is in:
   ftp://ftp.cs.huji.ac.il/users/danny/freebsd/diskless-boot/

I'll commit this to rc.diskless1

+if [ -z `hostname -s` ]; then
+hostname=`kenv dhcp.host-name`
+hostname $hostname
+echo Hostname is $hostname
+fi

BUT you've missed the point with this point:
+else
+mount_md 4096 /conf/etc 0
+chkerr $? MFS mount on /conf/etc
+mount -t unionfs /conf/etc /etc
+/bin/chmod 755 /etc
+find /etc  /dev/null
+touch /etc/.sentinel

That being that some of us actually WANT /etc to be left the fsck alone.
We NFS mount / and that / has /etc as it should be.  I also cannot follow
what you are trying to do with /etc/conf.  Perhaps you could comment that
part?

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: PAM OpenSSH: two incorrect last login

2002-04-22 Thread Bill Fenner


Might, might not.  ISTR strftime() can't correctly emulate ctime(),
but some other format might be preferrable.  Do you have a format
string handy?

I'd think something like what last does would be good.

d_first = (*nl_langinfo(D_MD_ORDER) == 'd');

...

(void) strftime(ct, sizeof(ct), d_first ?
(yflag ? %a %e %b %Y %R : %a %e %b %R) :
(yflag ? %a %b %e %Y %R : %a %b %e %R), tm);

except you probably want %T instead of %R if you want the seconds.
You could either pretend that yflag was not set (what ache was
suggesting) or set yflag if the year of the last login was not
this year (possibly more useful).

  Bill

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: panic:bremfree with today's current and linux-netscape

2002-04-22 Thread Adrian Penisoara

Hi,

On Mon, 22 Apr 2002, Bruce Evans wrote:

 Yes; I forgot to write the clause about it only being equivalent to a
 panic if certain options (mainly INVARIANTS) are configured.
 
  do see that falloc does lock the file descriptor table too, though ---
  I wonder how it ever worked.
 
 INVARIANTS causes a check of an assertion that non-recursive locks
 like fd_mtx aren't actually misused recursively.  The check causes more
 panics than the misuse since the misuse is rarely fatal.  I may be
 missing something since I rarely use INVARIANTS and haven't checked
 that it causes the panic deterministically.

 I think you're talking about the WITNESS option rather than
INVARIANTS, that's the one who's at least documented in GENERIC as
being responsible for mutex and deadlock checks. And this is somewhat
confirmed by the face that an alternative kernel that I built doesn't
panic and the difference between them two is that WITNESS is disabled in
the non-panicking one.

 Happy debuggin'
 Ady (@freebsd.ady.ro)
___
| Programming in BASIC causes brain damage.   |
|   (Edsger Wybe Dijkstra)|




To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: DRM in the sys/ tree: looking for testers

2002-04-22 Thread Dag-Erling Smorgrav

Dag-Erling Smorgrav [EMAIL PROTECTED] writes:
 Eric Anholt [EMAIL PROTECTED] writes:
  Could people test this in-kernel DRM and tell me how it works for them?
 Seems to work fine here (trusty ol' Matrox G200 w/8 MB).  Is there a
 particular DRI application I can use to somehow stress-test or
 benchmark the module?  I've gotten kind of tired of Aleph One :)

OK, slight update here: anything that involves uploading textures
fails, and leaves DRI unusable until I restart the server.  For
instance, I can run the gears demo just fine, but if I run the
fire demo, I get lots of error messages about uploading textures,
and can't run the gears demo any more until I restart X.

DES
-- 
Dag-Erling Smorgrav - [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: lock order reversal and panic in kern_descrip.c

2002-04-22 Thread Jun Kuriyama

At Mon, 22 Apr 2002 11:09:26 -0500,
Jacques A. Vidrine [EMAIL PROTECTED] wrote:
 I'm curious ... could you send the output of 
 
   pkg_info -L linux-netscape-navigator-4.79 | xargs ls -l

Attached.

 Also, does `/compat/linux/bin/sh' blow up for you?

No, I can invoke this without problem...


-- 
Jun Kuriyama [EMAIL PROTECTED] // IMG SRC, Inc.
 [EMAIL PROTECTED] // FreeBSD Project


-r-xr-xr-x  1 root  wheel  293 Apr 22 22:37 /usr/local/bin/navigator-linux-4.79
-r--r--r--  1 root  wheel16934 Oct 17  2001 /usr/local/lib/netscape-linux/LICENSE
-r--r--r--  1 root  wheel   323710 Oct 17  2001 
/usr/local/lib/netscape-linux/Netscape.ad
-r--r--r--  1 root  wheel16731 Oct 17  2001 /usr/local/lib/netscape-linux/README
-r--r--r--  1 root  wheel 4674 Oct 18  1994 /usr/local/lib/netscape-linux/XKeysymDB
-r--r--r--  1 root  wheel 4568 Oct 17  2001 
/usr/local/lib/netscape-linux/bookmark.htm
-r--r--r--  1 root  wheel 5861 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties
-r--r--r--  1 root  wheel 6069 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.cs
-r--r--r--  1 root  wheel 6073 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.hu
-r--r--r--  1 root  wheel 9941 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.ja
-r--r--r--  1 root  wheel 7744 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.ko
-r--r--r--  1 root  wheel 6070 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.pl
-r--r--r--  1 root  wheel 6071 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.ru
-r--r--r--  1 root  wheel 7708 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.zh
-r--r--r--  1 root  wheel 8893 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/aix/font.properties.zh_TW
-r--r--r--  1 root  wheel  981 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/awt.properties
-r--r--r--  1 root  wheel 7304 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties
-r--r--r--  1 root  wheel 7757 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.cs
-r--r--r--  1 root  wheel 7713 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.el
-r--r--r--  1 root  wheel 7758 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.hu
-r--r--r--  1 root  wheel13029 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.ja
-r--r--r--  1 root  wheel 9033 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.ko
-r--r--r--  1 root  wheel 7771 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.lt
-r--r--r--  1 root  wheel 7771 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.lv
-r--r--r--  1 root  wheel 7758 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.pl
-r--r--r--  1 root  wheel 7711 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.ru
-r--r--r--  1 root  wheel 7721 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.tr
-r--r--r--  1 root  wheel 8976 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.zh
-r--r--r--  1 root  wheel 8984 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.zh_GB2312
-r--r--r--  1 root  wheel23909 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.zh_TW
-r--r--r--  1 root  wheel 8737 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.zh_TW_Big5
-r--r--r--  1 root  wheel23946 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/font.properties.zh_TW_CNS11643
-r-xr-xr-x  1 root  wheel11179 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties
-r-xr-xr-x  1 root  wheel 8364 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.bg
-r-xr-xr-x  1 root  wheel 8672 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.cs
-r-xr-xr-x  1 root  wheel 8360 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.el
-r-xr-xr-x  1 root  wheel 8670 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.hr
-r-xr-xr-x  1 root  wheel 8676 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.hu
-r-xr-xr-x  1 root  wheel17681 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.ja
-r-xr-xr-x  1 root  wheel14797 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.ko
-r-xr-xr-x  1 root  wheel 8673 Oct  5  1998 
/usr/local/lib/netscape-linux/java/classes/hpux/font.properties.pl
-r-xr-xr-x  1 root  wheel 8665 Oct  5  1998 

Re: FreeBSD 5.0 Developer Preview #1 Now Available / diskless booting

2002-04-22 Thread David O'Brien

On Mon, Apr 22, 2002 at 10:40:44AM +0300, Danny Braniss wrote:
 Since im very interested in diskless/dataless, and i've been at it for some
 time now, i made some changes to libstand/bootp.c and /etc/rc.diskless1
 which i wouldn't mind receiving commnets/suggestions, and if possible
 incorporated.
 
 the stuff is in:
   ftp://ftp.cs.huji.ac.il/users/danny/freebsd/diskless-boot/

The bootp.c.diff looks like it could be useful; but I see you changed
some stuff related to PXE booting and I cannot easily test that.  Maybe
someone else can.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: Adding a 'bpf' group for /dev/bpf*

2002-04-22 Thread sthaug

  For the benefit of packet sniffers and other things that only want
  read-only access to /dev/bpf*, what do people think of adding a 'bpf'
  group for those programs?  This allows bpf devices to be read by
  programs running with an effective gid of 'bpf' instead of the current
  requirement for an effective user of root.  I've been running this way
  on many of our servers for several months now, and things like snort,
  tcpdump, etc., are quite happy with it (under stable).
 
 There's the other small problem that you have to be root to set
 promiscuous mode.

Not on 4.x. Haven't tried -current.

Steinar Haug, Nethelp consulting, [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: lock order reversal and panic in kern_descrip.c

2002-04-22 Thread Jacques A. Vidrine

On Tue, Apr 23, 2002 at 01:20:07AM +0900, Jun Kuriyama wrote:
 At Mon, 22 Apr 2002 11:09:26 -0500,
 Jacques A. Vidrine [EMAIL PROTECTED] wrote:
  I'm curious ... could you send the output of 
  
pkg_info -L linux-netscape-navigator-4.79 | xargs ls -l
 
 Attached.

No setuid executables ... it's a mystery to me how one encounters this
code path when running netscape :-(

-- 
Jacques A. Vidrine [EMAIL PROTECTED] http://www.nectar.cc/
NTT/Verio SME  . FreeBSD UNIX .   Heimdal Kerberos
[EMAIL PROTECTED] .  [EMAIL PROTECTED]  .  [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



A very excite game

2002-04-22 Thread zajv


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: PAM OpenSSH: two incorrect last login

2002-04-22 Thread Andrey A. Chernov

On Mon, Apr 22, 2002 at 14:07:50 +0200, Dag-Erling Smorgrav wrote:
 Bill Fenner [EMAIL PROTECTED] writes:
  You could either pretend that yflag was not set (what ache was
  suggesting) or set yflag if the year of the last login was not
  this year (possibly more useful).
 
 ...or always print the year.  Does anybody have any patches?  :)

IMHO always printing year in pam_lastlog (like unpatched sshd does) will 
not harm.

L10n here currently is not useful for both plain login and sshd because
environment from where LANG may come not yet activated at this moment.
The code must be moved after environment activating first, but because it 
is the very first print, other prints like motd must be moved too.

-- 
Andrey A. Chernov
http://ache.pp.ru/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: kde3 on -current

2002-04-22 Thread Matthew N. Dodd

On Mon, 22 Apr 2002, David W. Chapman Jr. wrote:
 Kde3 seems to work great under -current, kdegames3 fails to compile
 though.  I get the following.  Any ideas would be appreciated.
 In file included from fcs.h:57,
  from freecell.c:23:
 md5.h:17: redefinition of `u_int32_t'
 /usr/include/sys/types.h:116: `u_int32_t' previously declared here
 gmake[3]: *** [freecell.lo] Error 1

There are a couple of things that need  !defined(__FreeBSD__) to them.
(This error and the one after it IIRC.)

There is one later on involving malloc.h in kdegraphics.

-- 
| Matthew N. Dodd  | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD  |
| [EMAIL PROTECTED] |   2 x '84 Volvo 245DL| ix86,sparc,pmax |
| http://www.jurai.net/~winter |  For Great Justice!  | ISO8802.5 4ever |


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Arla Port

2002-04-22 Thread Scott Sipe


Hi, I'm having trouble compiling the Arla (AFS) port on CURRENT, it looks 
like certain structures have changed.  Is anyone running Arla on current?

If it would be the right thing to do, I can post the errors here.

Scott

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Marc G. Fournier

On Mon, 22 Apr 2002, Terry Lambert wrote:

 Marc G. Fournier wrote:
  On Sun, 21 Apr 2002, Terry Lambert wrote:
   No, there's no stats collected on this stuff, because it's a pretty
   obvious and straight-forward thing: you have to have a KVA space large
   enough that, once you subtract out 4K for each 4M of physical memory and
   swap (max 4G total for both), you end up with memory left over for the
   kernel to use, and your limits are such that the you don't run out of
   PTEs before you run out of mbufs (or whatever you plan on allocating).
 
  God, I'm glad its straightforwards :)
 
  Okay, first off, you say (max 4G total for both) ... do you max *total*
  between the two, or phy can be 4g *plus* swap can be 4g for a total of 8g?

 You aren't going to be able to exceed 4G, no matter what you do,
 because that's the limit of your address space.

 If you want more, then you need to use a 64 bit processor (or use a
 processor that supports bank selection, and hack up FreeBSD to do
 bank swapping on 2G at a time, just like Linux has been hacked up,
 and expect that it won't be very useful).

Now I'm confused ... from what I've read so far, going out and buying an
IBM eSeries 350 with 16Gig of RAM with Dual-PIII processors and hoping to
run FreeBSD on it is not possible?  Or, rather, hoping to use more then
4 out of 16Gig of RAM is?


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Terry Lambert

Marc G. Fournier wrote:
  You aren't going to be able to exceed 4G, no matter what you do,
  because that's the limit of your address space.
 
  If you want more, then you need to use a 64 bit processor (or use a
  processor that supports bank selection, and hack up FreeBSD to do
  bank swapping on 2G at a time, just like Linux has been hacked up,
  and expect that it won't be very useful).
 
 Now I'm confused ... from what I've read so far, going out and buying an
 IBM eSeries 350 with 16Gig of RAM with Dual-PIII processors and hoping to
 run FreeBSD on it is not possible?  Or, rather, hoping to use more then
 4 out of 16Gig of RAM is?

FreeBSD doesn't currently support bank selection.  Peter was
working on it, last time I heard.  Linux supports it, at an
incredible performance penalty.

But yes, it means only 4G of the RAM will be usable by you.

Bank selection works by leaving the address space at 4G, and
switching between banks, 2G at a time out of the 16G.

Basically, your kernel code lives in the first 2G, and then
you get to pick which 2G out of the 16G is the last 2G.

As I said, I expect that doing this won't be very useful;
since Itanium's are available, and FreeBSD runs native in
multiuser mode on IA64 now, there's really no reason to
do the 16G, 2G at a time bank selection trick.

The main reason I don't think it'll be useful is DMA: for
the DMA to occur, it will have to occur into the first 2G,
so that it's never selected out.  This is because, no matter
what you do, your address space is limited to 4G total:
adding banks just controls what physical memory is placed
in that 4G window at any given time.

Since the most useful thing you could do with more memory is
buffers for networking and disk I/O for things like web and
file servers... not very useful.

Consider that if I had two processes, and divided the memory
into 8 2G banks.  The 0th bank has the kernel in it, and
can never be selected out, if you expect the kernel to run or
DMA to be possible.  The 1th bank contains the memory for one
process, running on CPU 0.  The 4th bank contains the memory
for one process, running on CPU 1.  Basically, now, you can
not run these processes simultaneously, because they have
conflicting bank selects.

You could jam everything into all the code -- you'd have to
hack the paged memory management, the VM, the scheduler, etc.,
to get it to work -- but, even so, after all that work, what
you have effectively bought yourself is an L3 cache that's
in RAM, rather than in a swap partition.

You are better off just making it usable as swap, semi-directly,
and then making all the paging structures not used for the
kernel itself, swappable.

Even so, your KVA is restricted by whatever your bank size is,
and you can't use it directly (e.g. KVA + UVA + bank_region = 4G).

You really, really ought to look at the books I recommended,
if you are confused about why you can only use 4G with a 32
bit processor and FreeBSD, without additional heroic work.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Marc G. Fournier


First, alot of this stuff is slowly sinking in ... after repeatedly
reading it and waiting for the headache to disapate:)

But, one thing that I'm still not clear on ...

If I have 4Gig of RAM in a server, does it make any sense to have swap
space on that server also?  Again, from what I'm reading, I have a total
of 4Gig *aggregate* to work with, between RAM and swap, but its right here
that I'm confused right now ... basically, the closer to 4Gig of RAM you
get, the closer to 0 of swap you can have?

On Mon, 22 Apr 2002, Terry Lambert wrote:

 Marc G. Fournier wrote:
   No, there's no stats collected on this stuff, because it's a
   pretty obvious and straight-forward thing: you have to have a
   KVA space large enough that, once you subtract out 4K for each
   4M of physical memory and swap (max 4G total for both), you
   end up with memory left over for the kernel to use, and your
   limits are such that the you don't run out of PTEs before you
   run out of mbufs (or whatever you plan on allocating).
 
  ... and translated to english, this means? :)
 
  Okay, I'm going to assume that I'm allowed 4Gig of RAM + 4Gig of Swap, for
  a total of 8Gig ... so, if I subtract out 4K for each 4M, that is 8M for
  ... what?
 
  So, I've theoretically got 8184M of VM available for the kernel to use
  right now?  what are PTEs and how do I know how many I have right now?  as
  for mbufs, I've currently got:

 No.

 Each 4M of physical memory takes 4K of statically allocated KVA.
 Each 4M of backing store takes 4K of statically allocated KVA.

 The definition of backing store includes:

 o All dirty data pages in swap
 o All dirty code pages in swap
 o All clean data pages in files mapped into process or kernel
   address space
 o All clean code pages for executables mapped into process or
   kernel address space
 o Reserved mappings for copy-on-write pages that haven't yet
   been written

 A PTE is a page table entry.  It's the 32 bit value in the page
 table for each address space (one for the kernel, one per process).
 See the books I posted the titles of for more details, or read the
 Intel processor PDF's from their developer web site.


  jupiter netstat -m
  173/1664/61440 mbufs in use (current/peak/max):
  77 mbufs allocated to data
  96 mbufs allocated to packet headers
  71/932/15360 mbuf clusters in use (current/peak/max)
  2280 Kbytes allocated to network (4% of mb_map in use)
  0 requests for memory denied
  0 requests for memory delayed
  0 calls to protocol drain routines
 
  So how do I find out where my PTEs are sitting at?

 The mbufs are only important because most people allocate a
 large number of mbufs up front for networking applications, or
 for alrge numbers of users with network applications that will
 need resources in order to be able to actually run.  There's
 also protocol control blocks and other allocation that occur
 up front, based on the maximum number of system open files
 and sockets you intend to permit.

 The user space stuff is generally a lot easier to calculate:
 do a ps -gaxl, round each entry in the VSZ column up to
 4M, divide by 4K, and that tells you how many 4K units you
 have allocated for user space.  For kernel space, the answer
 is that there are some allocated at boot time, (120M worth),
 and then the kernel map is grown, as necessary, until it hits
 the KVA space limit.  If you plan on using up every byte, then
 divide your total KVA space by 4K to get the number of 4K pages
 allocated there.

 For the kernel stuff... you basically need to know where the
 kernel puts how much memory, based on the tuning parameters
 you use on it.

 -- Terry



To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Terry Lambert

Jeroen Ruigrok/asmodai wrote:
 Take a look at this:
 
http://www.freebsd.org/cgi/getmsg.cgi?fetch=245329+248644+/usr/local/www/db/text/2001/freebsd-hackers/20010624.freebsd-hackers

This is actually no longer valid, since there have been changes
to both the PDE caclcualtions and the kernel base definition to
try and make it more automatic the change the KVA space size.

At the time of the referenced posting, the modifications necessary
were to /sys/conf/ldscript.i386 and /sys/i386/include/pmap.h.

David also neglected to document how he calculated the 511,
which is actually 511 for a UP system, 510 for an SMP system,
which is to divide the kernbase by 0x0040, after subtracting
0x0010, and then subtracting the recursive entry out of the
total.  You also have to subtract out the private entries (if any)
for SMP, etc..  Basically, you have to calculate the number of
descriptor entries required to map the entire KVA space as 4K
pages from 1K of 4K page tables (1K worth of entries in a 4K page
descriptor table for the address space).

Of course, now everyone is going to say how do I... how do I...,
wanting one of the six ways you have to do it, based on the FreeBSD
version and/or intermediate release (-release?  -stable?  -security?
-some-date-here?), rather than figuring out the answer based on a
single known release.

The other issue here is that the number 1 reason for wanting to
dick around with this is to be able to add more physical memory,
and to do that successfully, you have to know a hell of a lot more
about tuning FreeBSD than reading the happy-fun tuning manual
page can ever teach you, without you understanding how the OS
actually does its thing at a low level.  I personally consider the
tuning man page as just a knee-jerk reaction to bad publicity
resulting from naieve benckmarking.

IMO, it's much better to just give elliptical clues, and then
leave the job to the people who can follow the clues and learn
enough that they not only get the right answer, but then end up
knowing enough about *why* it's the right answer to be able to
do the other required tuning.

If FreeBSD would ever sit still long enough for someone to get
a book out, there's probably be a book on the subject (Kirk has
been working on one for a year now, according to several people,
called The Design and Implementation of the FreeBSD Operating
System; no, I don't know what version it's supposed to apply to);
IMO, an architect should set some things in stone, and leave them
there long enough that documentation doesn't immediately go out
of date.

It's a hazard of Open Source projects, in general, that there are
so many people hacking on whatever they think is cool that nothing
ever really gets built to a long term design plan that's stable
enough that a book stands a chance of having a 1 year lifetime.

Basically, it'll boil down to paying someone who knows where the
bodies are buried to do the work for you, if you want to get more
than just a hack job.  8-(.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Terry Lambert

David Schultz wrote:
 Thus spake Terry Lambert [EMAIL PROTECTED]:
  If you want more, then you need to use a 64 bit processor (or use a
  processor that supports bank selection, and hack up FreeBSD to do
  bank swapping on 2G at a time, just like Linux has been hacked up,
  and expect that it won't be very useful).
 
 I'm guessing that this just means looking at more than 4 GB of memory
 by working with 2 GB frames at a time.  As I recall, David Greenman
 said that this hack would essentially require a rewrite of the VM
 system.  Does this just boil down to using 36 bit physical addresses?
 Are there plans for FreeBSD to support it, or is everyone just waiting
 until 64 bit processors become more common?

David Greenman is right.Nevertheless, Peter was planning on
doing the hack, according to his postings to -current.  Please
check the list archives for these things.


 Does FreeBSD use 4M pages exclusively for kernel memory, as in
 Solaris, or is there a more complicated scheme?

FreeBSD starts out using 4K pages for the premapped memory, and
switches over to a 4M page scheme for the initially loaded kernel,
for at least the first 4M.  The PTEs that were for the 4K pages
that are replaced with the 4M mappings are simply lost in the
reload of CR3, and never recovered for the system to use (the
pages containing the PTEs there are leaked, but it's usually
one page, so 4K is not that bad a leak).

For much of the FreeBSD kernel, 4K pages are used.

I'm pretty sure Solaris also used 4K pages for swappable memory
in the kernel, as well: 4M pages don't make much sense, since
you could, for example, exhaust KVA space with 250 kernel modules
(250 X (1 data + 1 code) * 4M = 2G).



  If you increase the KVA, then you will decrease the UVA available to
  user processes.  The total of the two can not exceed 4G.
 
 In Linux, all of physical memory is mapped into the kernel's virtual
 address space, and hence, until recently Linux was limited to ~3 GB of
 physical memory.  FreeBSD, as I understand, doesn't do that.  So is
 the cause of this limitation that the top half of the kernel has to
 share a virtual address space with user processes?

No.  You need to look at the copyin implementation in both OSs to
find the answer.  The way it works is by mapping the address space
of the process in question and the kernel at the same time, and
copying bytes between them.

These are really basic questions about memory layout, which you
should already know the answer to, if you are mucking about in
the KVA size or other parts of the kernel.

I don't know where the Linux limitation comes from; it's really
hard for me to believe ~3G, since it's not an even power of 2,
so I don't really credit this limitation.


 I'll have to read those books one of these days when I have time(6).
 Thanks for the info.

No problem; I think you will have to, if you are planning on
mucking about with more than 4G of physical memory.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: FreeBSD 4.5-STABLE not easily scalable to large servers ... ?

2002-04-22 Thread Terry Lambert

Marc G. Fournier wrote:
 First, alot of this stuff is slowly sinking in ... after repeatedly
 reading it and waiting for the headache to disapate:)
 
 But, one thing that I'm still not clear on ...
 
 If I have 4Gig of RAM in a server, does it make any sense to have swap
 space on that server also?

Yes.  But it (mostly) does not apply to KVA, only to UVA data,
and there are much larger KVA requirements, so the more RAM you
have, the bigger the bottleneck to user space for anything you
swap.


 Again, from what I'm reading, I have a total of 4Gig *aggregate* to
 work with, between RAM and swap, but its right here that I'm confused
 right now ... basically, the closer to 4Gig of RAM you get, the closer
 to 0 of swap you can have?

No.

I think you are getting confused on cardinality.  You get one KVA,
but you get an arbitrary number of UVA's, until you run out of
physical RAM to make new ones.

You have 4G aggregate KVA + UVA.

So if your KVA is 3G, and your UVA is 1G, then you can have 1 3G
KVA, and 1000 1G UVA's.


Certain aspects of KVA are non-swappable.  Some parts of UVA are
swappable in theory, but never swapped in practice (the page
tables and descriptors for each user process).

The closer to 4G you have, the more physical RAM you have to spend
on managing the physical RAM.

The total amount of physical RAM you have to spend on managing
memory is based on the total physical RAM plus the total swap.

As soon as that number exceeds ~2.5G, you can't do it on a 32
bit processor any more, unless you hack FreeBSD to swap the
VM housekeeping data it uses for swapping UVA contents.

Think of physical RAM as a resource.  It's seperate from the
KVA and UVA, but the KVA has to have physical references to
do paged memory management.  You are limited by how many of
these you can have in physical RAM, total.


-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: lock order reversal and panic in kern_descrip.c

2002-04-22 Thread Jun Kuriyama

 No setuid executables ... it's a mystery to me how one encounters this
 code path when running netscape :-(

Hmmm, after patch about FILEDESC_LOCK (1.139), my netscape can run
correctly.  (@_@)


-- 
Jun Kuriyama [EMAIL PROTECTED] // IMG SRC, Inc.
 [EMAIL PROTECTED] // FreeBSD Project

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Status on ftp servers?

2002-04-22 Thread Justin Heath

Any word on when current.freebsd.org or releng4.freebsd.org will be 
available again? I noticed some discussion of this dating back to February 
but did not see a final date. If it is still going to be down for a while 
anyone know of an alternative site? Thanks.

Justin Heath


To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message