Re: objtrm problem probably found

1999-07-12 Thread Peter Jeremy

Doug Rabson [EMAIL PROTECTED] wrote:
We don't need the lock prefix for the current SMP implementation. A lock
prefix would be needed in a multithreaded implementation but should not be
added unless the kernel is an SMP kernel otherwise UP performance would
suffer.

Modulo the issue of UP vs SMP modules, the code would seem to be simply:

#ifdef SMP
#define ATOMIC_ASM(type,op) \
__asm __volatile ("lock; " op : "=m" (*(type *)p) : "ir" (v), "0" (*(type *)p))
#else
#define ATOMIC_ASM(type,op) \
__asm __volatile (op : "=m" (*(type *)p) : "ir" (v), "0" (*(type *)p))
#endif

Or (maybe more clearly):

#ifdef SMP
#define SMP_LOCK"lock; "
#else
#define SMP_LOCK
#endif

#define ATOMIC_ASM(type,op) \
__asm __volatile (SMP_LOCK op : "=m" (*(type *)p) : "ir" (v), "0" (*(type *)p))


John-Mark Gurney [EMAIL PROTECTED] wrote:
actually, I'm not so sure, it guarantees that NO other bus operation
will succeed while this is happening... what happens if a pci bus
mastering card makes a modification to this value?

This is a valid point, but I don't think it's directly related to this
thread.  I think it's up the the various PCI device driver writers to
ensure that objects shared between a PCI device and driver are
correctly locked.  The mechanism to do this is likely to be device-
specific: Lock prefixes only protect objects no larger than 32 or 64
bits (depending on the processor), cards may require locked accesses
to much larger structures.

I believe the API to the PCI-locking routines should be distinct from
the API for SMP locks - even though the underlying implementation may
be common.


Oliver Fromme [EMAIL PROTECTED] wrote:
In my list of i386 clock cycles, the lock prefix is listed with
0 (zero) cycles.
My i486 book states 1 cycle, although that cycle can be overlapped with
several other combinations that add a cycle to the basic instruction
execution time.  I don't know about the Pentium and beyond timings.  In
any case, we have real-world timings, which are more useful.


Mike Haertel [EMAIL PROTECTED] wrote:
Although function calls are more expensive than inline code,
they aren't necessarily a lot more so, and function calls to
non-locked RMW operations are certainly much cheaper than
inline locked RMW operations.
Based on my timings below, this is correct, though counter-intuitive.
Given the substantial cost of indirect function calls, I don't
this this would be acceptable, though.  I think compiling modules
separately for UP/SMP is a better choice.


In Message-id: 19990 [EMAIL PROTECTED],
Matthew Dillon [EMAIL PROTECTED] provided some hard figures
for a dual PIII-450.  Expanding those figures for a range of machines
(all are UP except the PIII-450, which are Matt's SMP figures), and
adding the cost of using indirect function calls (patches to Matt's
code at the end):

i386SX-25   P-133   PII-266  PIII-450  nproc  locks
mode  0   1950.2339.6526.31 9.21 EMPTY
mode  1   3340.5971.7424.4516.48 1  no  tight
mode  2   3237.5771.1825.2723.65 2  no  tight
mode  3   3367.65   282.31   153.2993.02 1 yes  tight
mode  4   3263.64   285.58   152.94   160.82 2 yes  tight
mode  5   9439.15   446.1660.4037.64 1  no  spread
mode  6  10231.96   467.3960.1689.28 2  no  spread
mode  7  10660.05   725.80   153.1888.32 1 yes  spread
mode  8   9990.18   755.87   155.18   161.08 2 yes  spread

mode  9   5544.82   131.3149.96? EMPTY
mode 10   7234.97   174.2064.81? 1  no  tight
mode 11   7212.14   178.7264.87? 2  no  tight
mode 12   7355.46   304.74   182.75? 1 yes  tight
mode 13   6956.54   327.11   180.21? 2 yes  tight
mode 14  13603.72   582.02   100.10? 1  no  spread
mode 15  13443.54   543.97   101.13? 2  no  spread
mode 16  13731.94   717.31   207.12? 1 yes  spread
mode 17  13379.62   800.31   207.70? 2 yes  spread

Modes 9 through 17 are equivalent to modes 0-8 except that the
operation is performed via a call thru a pointer-to-function.
(Mode 9 is a pointer to a nop).

Apart from the noticable improvement in overall speed from left to
right, this shows that the lock prefix is _very_ expensive on
Pentium and above, even in a UP configuration.  It makes no difference
on a 386.  (I can produce the 486 figures tonight after I get home).
It also suggests that moving to an indirect function call (to allow
run-time UP/SMP selection) will be quite painful.

As you can see, the lock prefix creates a stall condition on the locked
memory, but does *NOT* stall other memory.
This is at least CPU dependent (and may also depend on the motherboard
chipset).  The i486 states that `all memory is locked'.

Therefore I believe the impact will be unnoticeable.  On a duel 
450MHz P-III we are talking 37 ns vs 88 ns - 

Re: objtrm problem probably found (was Re: Stuck in objtrm)

1999-07-12 Thread Peter Jeremy

Mike Smith [EMAIL PROTECTED] wrote:
 Although function calls are more expensive than inline code,
 they aren't necessarily a lot more so, and function calls to
 non-locked RMW operations are certainly much cheaper than
 inline locked RMW operations.

This is a fairly key statement in context, and an opinion here would 
count for a lot; are function calls likely to become more or less 
expensive in time?

Based on general computer architecture principles, I'd say that a lock
prefix is likely to become more expensive[1], whilst a function call
will become cheaper[2] over time.

I'm not sure that this is an important issue here.  The sole advantage
of moving to indirect function calls would be that the same object
code could be used on both UP and SMP configurations, without
incurring the overhead of the lock prefix in the UP configuration.
(At the expense of an additional function call in all configurations).
We can't avoid the lock prefix overhead in the SMP case.

Based on the timings I did this morning, function calls are
(unacceptably, IMHO) expensive on all the CPU's I have to hand (i386,
Pentium and P-II) - the latter two presumably comprising the bulk of
current FreeBSD use.

Currently the UP/SMP decision is made at compile time (and has
significant and widespread impact) - therefore there seems little (if
any) benefit in using function calls within the main kernel.

I believe that Matt's patched i386/include/atomic.h, with the addition
of code to only include the lock prefix when SMP is defined, is
currently the optimal approach for the kernel - and I can't see any
way a future IA-32 implementation could change that.

The only benefit could be for kernel modules - a module could possibly
be compiled so the same LKM would run on either UP or SMP.  Note that
function calls for atomic operations may not be sufficient (by
themselves) to achieve this: One of the SMP gurus may be able to
confirm whether anything else prevents an SMP-compiled LKM running
with a UP kernel.

If the lock prefix overhead becomes an issue for LKMs, then we could
define a variant of i386/include/atomic.h (eg by using a #define which
is only true for compiling LKMs) which does use indirect function
calls (and add the appropriate initialisation code).  This is a
trivial exercise (which I'll demonstrate on request).

[1] A locked instruction implies a synchronous RMW cycle.  In order
to meet write-ordering guarantees (without which, a locked RMW
cycle would be useless as a semaphore primitive), it implies a
complete write serialization, and probably some level of
instruction serialisation.  Since write-back pipelines will get
longer and parallel execution units more numerous, the cost of
a serialisation operation will get relatively higher.  Also,
lock instructions are relatively infrequent, therefore there is
little incentive to expend valuable silicon on trying to make
them more efficient (at least as seen by the executing CPU).

[2] Function calls _are_ fairly common, therefore it probably is
worthwhile expending some effort in optimising them - and the
stack updates associated with a leaf subroutine are fairly
easy to totally hide in an on-chip write pipeline/cache.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: LOCK overheads (was Re: objtrm problem probably found)

1999-07-13 Thread Peter Jeremy

Matthew Dillon [EMAIL PROTECTED] wrote:
:mode 1   17.99 ns/loop nproc=1 lcks=no
:mode 3  166.33 ns/loop nproc=1 lcks=yes
...
:This is a K6-2 350. Locks are pretty expensive on them.
Wow, now that *is* expensive!  The K6 must be implementing it in
microcode for it to be that bad.

I wouldn't be surprised if lock prefixes did result in microcode
execution.  As I stated yesterday, I don't believe locked instructions
are implemented frequently enough to warrant special handling, and are
therefore likely to be implemented in whichever way need the least
chip area.

Since you need to be able to track and mark the memory references
associated with the instruction, the cheapest implementation (in terms
of dedicated chip area) is likely to be something like: wait until all
currently executing instructions are complete, wait until all pending
memory writes are complete (at least to L1 cache), assert the lock pin
and execute RMW instuction without allowing any other instructions to
commence, deassert lock pin.  This is (of course) probably the worst
case as far as execution time as seen by that CPU - though it's not
far off optimal as seen by other CPUs.

(`Assert lock pin' should also be mapped into a `begin locked memory
reference' using whatever cache coherency protocol is being used).

I'm not saying that you can't implement a locked RMW sequence a lot
better, but until the chip architects decide that the performance is
an issue, they aren't likely to spend any silicon on it.  The big
IA-32 market is UP systems running games - and locked RMW instructions
don't affect this market.  Intel see the high-end of the market (where
SMP and hence locked RMW is more of an issue) moving to Merced.  This
suggests that it's unlikely that the IA-32 will ever acquire a decent
lock capability (though at least the PIII is no worse than the PII).

That said, the above timings make a lock prefix cost over 50 core
clocks (or 15 bus clocks) - even microcode couldn't be that bad.  My
other timings (core/bus cycles) were: 486DX2: 20/10, Pentium: 28/7,
P-II: 34/8.5, P-III 34/7.5.

I suspect that these timings are a combination of inefficient on-chip
implementation of the lock prefix (see above for my reasoning behind
this), together with poor off-chip handling of locked cycles.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Using float emulator on a system with FPU?

1999-07-14 Thread Peter Jeremy

Hi Warner,

Without commenting on the need to have an emulator in the kernel,
doesn't -msoft-float work faster?
Yes it does.  The kernel has to emulate an 80x87, giving you the
following sequence:
a) trap to kernel
b) decode FP instruction
c) fetch operands - possibly including copyin()
d) perform operation [this is the only bit the soft-FP code does]
e) store result - possibly including copyout()
f) return from trap

The trap/decode/copyin()/copyout()/rtt are all non-trivial.

  Or is that a MIPS only thing...
It's documented under i386 as well.  There is a note that gcc can
still emit some hardware FP instructions (to return FP values on
the 80x87 stack), but this can be prevented with -mno-fp-ret-in-387

The disadvantage of soft-FP is that you wind up with lots of different
libc's - one for h/w FP, one for soft FP and maybe a few more for
different FP formats (eg FFP on M68K, or a `cut-a-few-corners-but-
give-almost-the-correct-answer-much-faster' IEEE, or single-precision
only IEEE).  This is less of a problem in an embedded system, where
the FP requirements (if needed at all) are well defined.  (The problem
is that using the wrong library generally doesn't show up until
run-time - when the results aren't what you expected).

Note that the GNU newlib defines `iprintf()' (and family) as an
integer-only printf() to avoid pulling in all the FP overhead when
it's not really needed.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Make world broken

1999-07-28 Thread Peter Jeremy

cvs-cur 5518 breaks building libgcc with:
c++ -c -I/usr/obj/3.0/cvs/src/tmp/usr/include/g++ -O -pipe 
-I/3.0/cvs/src/gnu/lib/libgcc/../../../contrib/egcs/gcc/config 
-I/3.0/cvs/src/gnu/lib/libgcc/../../../contrib/egcs/gc
c -I. -fexceptions -DIN_GCC -I/usr/obj/3.0/cvs/src/tmp/usr/include 
-I/3.0/cvs/src/gnu/lib/libgcc/../../../contrib/egcs/gcc/cp/inc -nostdinc++ -DL_op_new 
-o _op_new.o /3.0/cvs/s
rc/gnu/lib/libgcc/../../../contrib/egcs/gcc/cp/new1.cc
In file included from /usr/obj/3.0/cvs/src/tmp/usr/include/stddef.h:39,
 from /usr/obj/3.0/cvs/src/tmp/usr/include/g++/new:8,
 from 
/3.0/cvs/src/gnu/lib/libgcc/../../../contrib/egcs/gcc/cp/new1.cc:28:
/usr/obj/3.0/cvs/src/tmp/usr/include/machine/ansi.h:103: syntax error before 
`__attribute__'
/usr/obj/3.0/cvs/src/tmp/usr/include/machine/ansi.h:104: syntax error before 
`__attribute__'
In file included from /usr/obj/3.0/cvs/src/tmp/usr/include/g++/new:9,
 from 
/3.0/cvs/src/gnu/lib/libgcc/../../../contrib/egcs/gcc/cp/new1.cc:28:
/usr/obj/3.0/cvs/src/tmp/usr/include/g++/exception:9: syntax error before string 
constant
*** Error code 1

Stop.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: make.conf on CURRENT question

1999-07-28 Thread Peter Jeremy

"Scot W. Hetzel" [EMAIL PROTECTED] wrote:
You'll also want to use:

make world -DWANT_AOUT=YES

to have the a.out libraries built.

You'll also need the a.out X11 libraries, and last time I tried,
they built OK, but wouldn't work.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Broken world

1999-07-28 Thread Peter Jeremy

"David O'Brien" [EMAIL PROTECTED] wrote:
  from 
/usr/src/gnu/lib/libgcc/../../../contrib/egcs/gcc/cp/new1.cc:28:
 /usr/obj/usr/src/tmp/usr/include/g++/exception:9: syntax error before string 
constant
 *** Error code 1


This is due to the Bison-Yacc change.  I did builds of the compiler and
all, but not a full make world before commiting.  I am *VERY* surprised
it has taken nearly 24hrs for someone to yell Ouch!  

I wrote exactly the same thing as Greg, just over 9 hours ago.

Petrer


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: gnu tar upgrade?

1999-08-16 Thread Peter Jeremy

Alec Wolman [EMAIL PROTECTED] wrote:
The version of tar that comes with freebsd (v1.11.2 with local
freebsd modifications) has a bug: if you attempt to copy large
files ( 2GB) it will silently truncate the large file.
...
There is a new version of gnu tar (v1.13) that has support for
large files.

  FreeBSD has added the following behavior over
the years:
  the --unlink option:
  the --norecurse option:
  the --bzip and --unbzip options:
  the --fast-read option:

A check through the CVS logs reveals lots of other changes and
enhancements such as:
- the builtin regex() code has been replaced with -lgnuregex
- 21-bit minor numbers (same as v1.13)
- Support for hard-links to files with long names
- FreeBSD-specific knowledge of potential names/locations of rsh(1)
- support 8-bit filenames
- assorted minor bugfixes and compiler-quietenings

If we do import GNU tar 1.13 (which I think is probably a good idea),
we need to make sure all the `invisible' fixes go in (if they're not
there already), as well as the additional options.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



RE: Dropping connections without RST

1999-08-17 Thread Peter Jeremy

Geoff Rehmet [EMAIL PROTECTED] wrote:
sysctl -d needs fixing. :-)

No, sysctl -d needs _implementing_.  I've looked into this myself.
I brought it up on -hackers in mid-February, and here in early June.

"sysctl -d" invokes sysctl({0,5,...},...) (sysctl.c:show_var()).

kern_sysctl.c documents (and implements) the following:
/*
 * "Staff-functions"
 *
 * These functions implement a presently undocumented interface 
 * used by the sysctl program to walk the tree, and get the type
 * so it can print the value.
 * This interface is under work and consideration, and should probably
 * be killed with a big axe by the first person who can find the time.
 * (be aware though, that the proper interface isn't as obvious as it
 * may seem, there are various conflicting requirements.
 *
 * {0,0}printf the entire MIB-tree.
 * {0,1,...}return the name of the "..." OID.
 * {0,2,...}return the next OID.
 * {0,3}return the OID of the name in "new"
 * {0,4,...}return the kind  format info for the "..." OID.
 */

It would be a fairly trivial exercise (a couple of hours, if that)
to implement sysctl({0,5,...},...).  The downside is the the loaded
kernel would bloat by the size of the sysctl descriptions.  I don't
feel that this is acceptable (and it's definitely unnecessary).

My feeling is that each sysctl should include a 1-line brief
description (which includes any relevant units[0]), followed by a much
longer description of what the sysctl does and when it should (or
shouldn't) be tweaked.  A new option (-D maybe) to sysctl would
display the long description.  An example of what I mean by the latter
is the following (from one of Matt's commits on 1999/02/18 11:57:33
PST):

   vm.swap_async_max: 4
   vm.swap_cluster_max: 16
  
  Recommended values are a cluster size of 8 or 16 pages.  async_max is
  about right for 1-4 swap devices.  Reduce to 2 if swap is eating too much
  bandwidth, or even 1 if swap is both eating too much bandwidth and sitting
  on a slow network (10BaseT).
  
  The defaults work well across a broad range of configurations and should
  normally be left alone.

I currently have a partially implemented solution which puts the
documentation strings into a separate ELF section, which isn't loaded.
struct sysctl_oid is extended to include a `pointer' to this string
(and will probably need a pointer to the loaded filename[1]).  A new
`staff function' [2] will return the string offset and filename.

There are ~600 sysctl's, so this approach would add ~4.8K to the
loaded i386 kernel image size (and about twice that on an alpha[2]).
(Whereas the actual description strings should be at least an order of
magnitude more).

sysctl(8) will then need to open the specified file and extract the
string at the specified offset.  I'd like to arrange for the offset to
actually be the physical offset within the file - which should be
possible, but I'm having trouble convincing the linker to do this.
Otherwise the sysctl will need to actually decode the ELF structure.

If necessary, the relevant kernel section could be stripped off to
reduce the size of the kernel image on disk.

An alternative approach would be to suitably mark the documentation
strings and extract them into man pages (using a perl tool or
similar).  This would remove `sysctl -d'.

[0] Eg most of the IP timers are in units of 500msec.  This is not
obvious and has caught out a couple of people here recently.

[1] This is needed to support description strings in KLDs.  The
pathnames are currently available in KVM.

[2] Probably {0,6,...} to prevent compatibility problems with old
sysctl(8) programs (since the return information will be
different).

[3] Depending on the implementation, and the layout of struct
sysctl_oid, this could either be ~9.6K (appending both fields)
or ~4.8K (append the pathname pointer, put the 32-bit offset
field into some existing padding).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: HEADS UP! ATA driver (atapi DMA)..

1999-08-31 Thread Peter Jeremy

David Scheidt [EMAIL PROTECTED] wrote:
Almost all fast (12X or so) CD-ROM drives are variable speed.

To put this a different way, the data on CD's is stored at a constant
lineal density (closely related to the wavelength of the laser).
Audio CDs are read using constant-linear-velocity, the angular
(rotational) speed reduces from the inside edge to the outside edge,
to maintain a constant data rate.  CD-ROM drives initially just copied
this approach.

Almost all fast CD-ROM drives now use constant-angular-velocity (which
simplifies the rotational mechanics and should improve seek times, at
the expense of slightly more complex read logic).  This means the the
lineal velocity (and hence data rate) increases from the inside
(start) to the outside (end) of the disk.

The quoted CD-ROM speeds are always based on the peak data rate
(and the better manufacturers actually quote the speed range).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Problems with the latest changes to ifconfig (I guess) - Badguess...

1999-08-31 Thread Peter Jeremy

Ian Whalley [EMAIL PROTECTED] wrote:
My card is identified as 3Com 3c905B-TX Fast Etherlink XL.

FWIW, I'm running a kernel about 30 hours old with a 3Com 3c905-TX
Fast Etherlink XL and I'm not seeing this problem.

At a quick quess, something in the miibus support broke the 3C905B
support.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Building PicoBSD

1999-09-14 Thread Peter Jeremy

I'm trying to build a `net' PicoBSD on a -current system and am
running into problems that appear to be inside make(1) (and how it
imports/exports/treats MAKEFLAGS).

I thought I'd try to shrink things by stripping the ISDN and RADIUS
support out of ppp (amongst other things), but ppp/i4b.c kept getting
included in the crunch1.cache and crunch1.mk.  I eventually wound up
with the patches below which:
1) Within crunchgen(8), explicitly pass the environment MAKEFLAGS to
   the subordinate make as a commandline option.
2) Add various options to net/crunch1/Makefile's MAKEFLAGS to disable
   unwanted bits and pieces.
3) Explicitly pass MAKEFLAGS to both crunchgen and the `build it all`
   make.
4) Remote ipfw and -lradius from the crunch.conf file.
5) Fix the password diffs to suit a -current system.  (Note that there
   is a generic problem with including the $FreeBSD$ tag in the diffs).

The result of all this is that ppp/i4b.o _has_ been left out (suggesting
that the -DNOI4B in MAKEFLAGS is being seen), but ppp/nat_cmd.o and
ppp/chap_ms.o are still being included (suggesting that the -DNONAT,
-DNOALIAS, -DRELEASE_CRUNCH and -DNOCRYPT _aren't_ being seen).

Does anyone have any ideas?  I _hope_ I'm doing soething silly...

Index: src/usr.sbin/crunch/crunchgen/crunchgen.c
===
RCS file: /home/CVSROOT/./src/usr.sbin/crunch/crunchgen/crunchgen.c,v
retrieving revision 1.11
diff -u -r1.11 crunchgen.c
--- crunchgen.c 1998/09/14 11:33:38 1.11
+++ crunchgen.c 1999/09/15 05:41:23
@@ -544,7 +544,7 @@
 fprintf(f, "crunchgen_objs:\n\t@echo 'OBJS= '${OBJS}\n");
 fclose(f);
 
-sprintf(line, "make -f %s crunchgen_objs 21", tempfname);
+sprintf(line, "make 'MAKEFLAGS=$MAKEFLAGS' -f %s crunchgen_objs 21", tempfname);
 if((f = popen(line, "r")) == NULL) {
warn("submake pipe");
goterror = 1;
Index: src/release/picobsd/net/crunch1/Makefile
===
RCS file: /home/CVSROOT/./src/release/picobsd/net/crunch1/Makefile,v
retrieving revision 1.5
diff -u -r1.5 Makefile
--- Makefile1999/08/28 01:33:39 1.5
+++ Makefile1999/09/15 05:47:50
@@ -3,6 +3,8 @@
 #
 SRC?=/usr/src
 
+MAKEFLAGS+=-DNOPAM -DNOSECURE -DNOCRYPT -DRELEASE_CRUNCH -DNOI4B -DNORADIUS -DNONAT
+
 all: crunch
 
 crunch:
@@ -11,8 +13,8 @@
touch /usr/src/usr.bin/passwd/.picobsd.patched; \
fi
@cat crunch.conf|sed -e "s@/usr/src@${SRC}@" crunch1.conf
-   @crunchgen ./crunch1.conf
-   @${MAKE} -f crunch1.mk -DNOPAM all \
+   @MAKEFLAGS="${MAKEFLAGS}" crunchgen ./crunch1.conf
+   @${MAKE} ${MAKEFLAGS} -f crunch1.mk -DNOPAM all \
"CFLAGS=${CFLAGS} -DRELEASE_CRUNCH -DCRUNCHED_BINARY -DNOSECURE -DNOCRYPT" 
#21 /dev/null
 
 clean:
Index: src/release/picobsd/net/crunch1/crunch.conf
===
RCS file: /home/CVSROOT/./src/release/picobsd/net/crunch1/crunch.conf,v
retrieving revision 1.9
diff -u -r1.9 crunch.conf
--- crunch.conf 1999/08/28 01:33:39 1.9
+++ crunch.conf 1999/09/15 05:52:33
@@ -25,7 +25,8 @@
 
 progs pwd ppp telnet more
 progs passwd date
-progs mount_cd9660 mount_nfs ping traceroute routed ipfw
+progs mount_cd9660 mount_nfs ping traceroute routed
+# ipfw
 
 ln mount_cd9660 cd9660
 ln mount_nfs nfs
@@ -42,4 +43,4 @@
 libs -lncurses -lmytinfo -lipx -lz -lpcap -lalias -lwrap
 libs -ledit -lutil -lmd -lcrypt -lmp -lgmp -lm -lkvm
 libs -lgnuregex -ltelnet
-libs -lradius # used by ppp
+# libs -lradius # used by ppp
Index: src/release/picobsd/net/crunch1/passwd.diff
===
RCS file: /home/CVSROOT/./src/release/picobsd/net/crunch1/passwd.diff,v
retrieving revision 1.5
diff -u -r1.5 passwd.diff
--- passwd.diff 1999/08/28 01:33:39 1.5
+++ passwd.diff 1999/09/13 03:08:52
@@ -2,14 +2,14 @@
 --- Makefile   Sat Aug  1 20:40:38 1998
 ***
 *** 2,24 
-  # $FreeBSD: ./src/release/picobsd/net/crunch1/passwd.diff,v 1.5 1999/08/28 01:33:39 
peter Exp $
+  # $FreeBSD: ./src/usr.bin/passwd/Makefile,v 1.34 1999/09/06 17:30:02 peter Exp $
   
   PROG=passwd
 ! SRCS=local_passwd.c passwd.c pw_copy.c pw_util.c pw_yp.c \
 !  yp_passwd.c ypxfr_misc.c ${GENSRCS}
   GENSRCS=yp.h yp_clnt.c yppasswd.h yppasswd_clnt.c \
yppasswd_private.h yppasswd_private_clnt.c yppasswd_private_xdr.c
-  CFLAGS+=-Wall -DPASSWD_IGNORE_COMMENTS
+  CFLAGS+=-Wall
   
 ! DPADD=   ${LIBCRYPT} ${LIBRPCSVC} ${LIBUTIL}
 ! LDADD=   -lcrypt -lrpcsvc -lutil
@@ -26,14 +26,14 @@
   
   CLEANFILES= ${GENSRCS}
 --- 2,21 
-  # $FreeBSD: ./src/release/picobsd/net/crunch1/passwd.diff,v 1.5 1999/08/28 01:33:39 
peter Exp $
+  # $FreeBSD: ./src/usr.bin/passwd/Makefile,v 1.34 1999/09/06 17:30:02 peter Exp $
   
   PROG=passwd
 ! SRCS=local_passwd.c passwd.c pw_copy.c pw_util.c 
 ! 
   GENSRCS=yp.h yp_clnt.c yppasswd.h 

Re: ports/13729: strip(1) exits with an error on script file - causessevere portability problems

1999-09-15 Thread Peter Jeremy

On Tue, 14 Sep 1999 02:39:23 +1000, Chuck Robey [EMAIL PROTECTED] wrote:
(Judging by the headers, this item spent a couple fo days getting from
Chuck to hub).
This kind of thing, where there is no bug ... where the subject is a
request for a new feature, isn't this kind of thing the wrong way for
folks to be using the trouble reporting system?

I don't think so.  send-pr(1) allows the following classes:
Class:  [ sw-bug | doc-bug | change-request ] (one line)

And we have no other tracking mechanism for users' change requests.
There are frequent requests on the general mailing lists to send-pr
things so they don't get lost.  It also allows non-committers (such
as myself) to formally submit new features.

 It seems to me that allowing
such use of gnats makes it miserably hard for folks to close some PRs.

I agree that mis-classifying feature requests as bugs (which Patrick
has done) causes problems, but this should be handled via a mechanism
to re-classify the PR.  I don't see a problem with having a large
number of `new feature requests' outstanding - as long as they can be
identified as such.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: EGCS, or EGCS?

1999-09-26 Thread Peter Jeremy

Peter Wemm [EMAIL PROTECTED] wrote:
Peter Jeremy wrote:
 I appear to have two complete copies of gcc - one in src/contrib/gcc
 and another in src/contrib/egcs/gcc.

src/contrib/gcc is where gcc used to live.  Then along came egcs with a
cygnus-style tree that ended up in src/contrib/egcs (v1.1.1 and later
1.1.2).  Now, egcs has become gcc 2.95, so it's going back to src/contrib/
gcc again.

That's the way I remembered things.  What threw me is that I currently
have two _different_ gcc directories, both claiming to be EGCS 1.1.2,
and both being updated.

  David O'Brien is working on this now but I think he's 
suffering from gcc-induced insanity. :-)

I guess that's one explanation...

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: HEADS UP: sigset_t changes committed

1999-09-29 Thread Peter Jeremy

On Thu, Sep 30, 1999 at 08:17:16AM +1000, Adam Strohl wrote:
On Wed, 29 Sep 1999, Jim Bloom wrote:

I believe this must be fixed.  At some point in time, there is going
to be another change to the kernel such that some older version of the
code cannot run on a new kernel.

FTPing a GENERIC kernel from somewhere would solve this, then just
single user the box, make world.  Build your kernel.  Reboot.

This assumes that your old world can work correctly with your new
kernel (remembering that `make world' is seen as our standard system
stress test).  The whole justification for the `install the world
then upgrade the kernel' approach has been that we _do_not_ guarantee
this.

Furthermore, for when 4.0 becomes a -R or -S, ftping in a compiled kernel
shouldn't be that hard of a price to pay for going from 3.2.  

We've never required this before.  I managed to convert from 2.2.6 to
-current using `make upgrade'.  Why should I need to FTP a kernel
from another machine to go from 3.x to 4.x?

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: HEADS UP: sigset_t changes committed

1999-09-29 Thread Peter Jeremy

On Thu, Sep 30, 1999 at 01:29:40AM +1000, Marcel Moolenaar wrote:
Before attempting to build world, you must make and install a new
kernel. The new kernel will contain new syscalls that are needed during
build world. doscmd is currently not being build because it needs fixing
first.

I'd like to voice my concerns with this change as well.  The `normal'
upgrade procedure has always been to build and install a new world
before the new kernel.  The only exception I'm aware of has been the
aout-elf transition (where a special build procedure was provided).

This commit seems to create the same upgrade hurdle as existed with
the conversion to ELF[1].  That conversion came with an extensive
description of how to upgrade the system, as well as clear CVS tags
delimiting it.  This change has no tags and a single paragraph
warning us to install a new kernel before building world.

I can't configure or compile a -current kernel on a -stable system
(and I have no idea whether I can run a -current kernel with a -stable
world, but I suspect I can't).  This makes it very difficult to
convert a -stable system to -current.  Whilst -current is for people
who like battering their way around such hurdles, it's also the
testbed for 4.0-RELEASE, and we will need a tested and documented
upgrade procedure in place before then.

That said, I think that backing out the changes at this stage will
only make things worse.  I would like to see a clear upgrade procedure
that will enable someone to cross this hurdle when their running
system can't build or run a current kernel.

[1] Actually, in many ways this change is worse.  Recent 2.2 kernels
will happily execute many 3.x ELF programs as long as the shared
libraries are present.  This change makes any program that uses
signals not backward compatible.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: HEADS UP: sigset_t changes committed

1999-09-29 Thread Peter Jeremy

On Thu, Sep 30, 1999 at 02:37:45PM +1000, Warner Losh wrote:
In keeping notes, what would need to happen would be that you'd have
to build config as well as all the tools to build binaries.

We might be able to get away with temporarily replacing the
i386/include/atomic.h[1] with the one from -stable and using gcc 2.7.

At some stage, we need to confirm that a -current kernel will work
with a -stable world - or add the incompatible bits to the list of
things to build first.

The current Makefile.upgrade would have to be massively gutted and
rewritten since it deals with the aout - elf transition, but it
shouldn't be too horrible.

At least, no more horrible than the aout-elf procedure :-).

I think that we need to do this upgrade proceedure.  I wish I had more
time than to just sketch out this outline...

I think that a suitable upgrade procedure should have been thought
through and implemented before the changes were committed.  It would
also have been nice to have a CVS tag.  It's a bit late now though.

[1] It's just occurred to me that the solution for atomic.h is to
make the asm constraints dependent on the gcc version.  Anyone
willing to commit a change if I submit patches?

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: new sigset_t and upgrading: a proposal

1999-09-30 Thread Peter Jeremy

On Fri, Oct 01, 1999 at 08:48:34AM +1000, David O'Brien wrote:
On Thu, Sep 30, 1999 at 06:25:56AM -0700, Rodney W. Grimes wrote:
  Cons
  
  o  Upgrading from 3.3 and before to -current is only possible after
 an upgrade to post 3.3.
 
 Not good.

We recommend that 2.2.x people upgrade to the latest 2.2-STABLE offering
before trying to jump to 3.x via `make {upgrade,world,aout-to-elf}.

I thought it was `2.2.8-RELEASE or later', not `the latest
2.2-STABLE'.  (I know this has been stated regularly in various
mailing lists, but I can't find the exact wording in the FAQ or
handbook).  AFAIK, this recommendation was more along the lines of
`it's known to work with 2.2.8 and isn't guaranteed to work with
earlier releases'.  I'm fairly certain I've upgraded machines from
2.2.6 or 2.2.7 to 3.x without installing 2.2.8.  If we implement this
change, it will be `the upgrade is guaranteed not to work unless you
are running 3.x from 1st October 1999 or later'.

I believe that the above also implies that there should be a
3.4-RELEASE before 4.0-RELEASE (I'm not sure what has been planned)
so that there's a -release that is upgradeable to -current.

In any case, I'm not sure that this is a particularly clean solution:
1) It is a specific work-around for this problem and does not solve
   the general problem:  Buildworld should not use be using tools
   built for the new world with the current kernel.
2) It's not a change that can be applied to -current, tested and then
   MFC'd.  This increases the likelihood that it will break -stable.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: System crash on vinum start

1999-09-30 Thread Peter Jeremy

On Fri, Oct 01, 1999 at 09:20:53AM +1000, Jeffrey J. Mountin wrote:
At 12:59 PM 9/28/99 +1000, Peter Jeremy wrote:
On Tue, Sep 28, 1999 at 09:11:31AM +1000, Greg Lehey wrote:
  Good software shouldn't panic.
I wish _I_ could convince some people of this :-(.

It can be difficult to consider what a user can do and tends to bloat the
code a bit.  Frustrates my instructors, since it very much a habit when I
script and carried over to C. 8-)

I was thinking of some `production' code (written by a sister company)
that I used to provide customer support for.  It would regularly core
dump (but was automatically restarted).  After a few years they did
manage to fix the core dump, but that exposed a memory leak which they
never did manage to fix (fairly embarrasing in a supposedly
permanently running process).

Speaking of, why is (would) root filesystem support necessary for non
root/swap automagical recovery.

I gather that root support is the most-asked-for feature in Vinum (when
I asked Greg about it some weeks ago, I got about as far as `when will
Vinum' before he answered me).  As long as recovery works, making it
nicer to use is (should be) a lower priority.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: HEADS UP: sigset_t changes committed

1999-09-30 Thread Peter Jeremy

On Fri, Oct 01, 1999, you wrote:
On Sep 30,  4:14pm, John-Mark Gurney wrote:
} Subject: Re: HEADS UP: sigset_t changes committed
}  
}  In this particular case, the only thing cross-compilation would buy us
}  is the ability to build (but not install) 4.x binaries on a machine
}  running 3.x.  It sounds like some folks would be satisfied just having
}  that.
} 
} I'm sorry, this is easy to fix... have a set of tools you copy to /ibin
} that are used for the install (all staticly compiled binaries hopefully)
} and run the install world out of /ibin...  maybe include some binaries
} for system recovery to make sure...

... but as soon as you run the stuff in /ibin to install the new userland,
you won't even be able to run a shell script, because the newly installed
/bin/sh will be using the new signal syscalls.

No, all it means is that the tools to install the new userland need
to have special shellscripts with '#!/ibin/sh' or similar, or be
explicitly invoked (eg /ibin/sh foo).  I don't think this is a real
problem.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: make install trick

1999-10-05 Thread Peter Jeremy

On 1999-Oct-06 07:42:39 +1000, Warner Losh wrote:
Nearly full?  It is a 32M file system with 15M free.  That's not
nearly full.  The problem is that the update rate is faster than the
softupdate code can deal with.  Running sync between each install
shows that this is a bug.

As Ollivier Robert pointed out, it is a known problem.  Kirk says
it's non-trivial to fix (though I'm sure he'll welcome your patches :-).
The suggested work-around is not to run softupdates on root.

In any case, you should not be doing lots of writes to root, so the
lack of softupdates should not be a problem.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: make install trick

1999-10-05 Thread Peter Jeremy

On 1999-Oct-06 09:02:12 +1000, Nate Williams wrote:
 In any case, you should not be doing lots of writes to root, so the
 lack of softupdates should not be a problem.

So, are you suggesting make /tmp it's own disk, otherwise anytime you do
development alot of writes are done to /.

I've pretty well always used a separate /tmp partition.  I've always
used MFS with FreeBSD.  This makes the root partition virtually
read-only (which has robustness and security advantages).

And, if you do lots of development, then you'll have the same problem on
/tmp as you did on / unless you waste a huge disk for /tmp. :(

Solutions:
1) Use -pipe
2) Don't use softupdates
3) Use a RAMdisk (eg MFS) instead of a physical disk (which makes the
   previous point irrelevant).

I don't recall seeing anyone mention problems like this (though they
might be on lists I don't read).

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: make install trick

1999-10-05 Thread Peter Jeremy

On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
I've seen softupdates nearly eliminate disk io for systems that used
an abmornal amount of temp files, but the fact that it can destabilize
a system worries me greatly.

What do you mean by `destabilize'?  There are bugs in softupdates
which mean that an application may receive ENOSPC when writing to a
filesystem even though space on that filesystem has been recently
freed.  Any application that cannot handle this situation is _broken_.

Another option for /tmp - which I forgot last time, and seems to
avoid the known problems with MFS and softupdates - it to mount
/tmp async.  Since you're going to blow it all away on the next
reboot anyway, it doesn't really matter if the a crash trashes the
FS (which is the major problem with async mounts).

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: make install trick

1999-10-05 Thread Peter Jeremy

On 1999-Oct-06 11:33:51 +1000, Alfred Perlstein wrote:

On Wed, 6 Oct 1999, Peter Jeremy wrote:

 On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
 I've seen softupdates nearly eliminate disk io for systems that used
 an abmornal amount of temp files, but the fact that it can destabilize
 a system worries me greatly.
 
 What do you mean by `destabilize'?
...
 softupdates can crash the system when this
happens, not a mere application, but the entire system can lock up.

I don't recall this coming up in this thread.  Last I recall, that
bug was supposed to be fixed (and Kirk(?) wanted anyone who still
saw it to report it).  Unfortunately, I can't find that thread in
the archives :-(.

Which isn't an option unless you dedicate a partition for /tmp
which is pretty wasteful imo.

I guess we disagree on this.  My feeling is that write activity on
root should be minimised to minimise the risk that root will be
inconsistent following a crash.  Secondly, there's a security
viewpoint that users should not have write access anywhere in the root
or /usr filesystems.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: make install trick

1999-10-06 Thread Peter Jeremy

On 1999-Oct-06 16:14:19 +1000, Brian Somers wrote:
 On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
 I've seen softupdates nearly eliminate disk io for systems that used
 an abmornal amount of temp files, but the fact that it can destabilize
 a system worries me greatly.
 
 What do you mean by `destabilize'?  There are bugs in softupdates
 which mean that an application may receive ENOSPC when writing to a
 filesystem even though space on that filesystem has been recently
 freed.  Any application that cannot handle this situation is _broken_.
[.]

:-]  You're joking right ?  Most applications don't even *notice* 
the situation let alone handle it.

I wasn't really joking.  Applications _should_ handle this situation.
Most coding standards tell you to check for error conditions.  Most
programmers (including myself most of the time) just don't bother.
(One reason being that C suffers from extreme code bloat and
substantial loss of readability.  C++ exceptions are a definite
improvement here).

  Whaddaya do when /var/log runs 
out of space ?  Show me an app that handles it.

The only app that normally writes in /var/log is syslogd.  And the
behaviour you want in this case depends on the sysadmin's level of
paranoia (anything from `ignore it' to `halt the system').

I guess you can argue the case, but in real life, running out of any 
machine resource can cause all sorts of possibly unrecoverable 
problems.
Agreed.  One reason I (and presumably others) don't bother to check
for some errors is that it's not clear what you can or should do if
you get the error.

  IMHO the best way to deal with it in a generic way is to 
have the give-me-the-resource function block if it really thinks it's 
a temporary thing.

How do you work out whether the resource shortage is temporary or not?
This situation is orthogonal to the ever-popular `out of swapspace'
issue (normally brought up in conjunction with swap overcommit).  The
difficulty with any recovery strategy is avoiding deadlocks.

(I suspect that a major reason why root is not constrained by UFS
MINFREE was to try and avoid the situation where the _system_ (as
against the users) ran out of disk space).

In the case of softupdtes, I'd be surprised if it couldn't easily 
figure out that the problem is *probably* transient and block,

To quote an excerpt from the response I got from Kirk when I
asked him about this problem:

: I experimented with keeping a count of space that was pending
:removal. If the filesystem was about to return a space full message,
:it would check the pending count for the filesystem and if there was
:space that was going to show up in the future, it would request that
:the space freeing be expedited then go to sleep and wait for it. The
:problem is that the space freeing can take up to a minute (typically
:more like 30 seconds) during which time the file's inode is
:locked. If the locked file is say a log file, then you can get a
:temporary lock race to the root which make for really terrible
:perceived performance. If the inode is unlocked during the wait, then
:file inconsistencies can sneak in. So, the short answer is that you
:should not run soft updates on filesystems where you are running with
:less than a 60 second reserve of free space.

Note the last sentence...

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: {a}sync updates (was Re: make install trick)

1999-10-06 Thread Peter Jeremy

On 1999-Oct-07 09:15:42 +1000, Matthew D. Fuller wrote:
Is this good, bad, ugly, or just inconsistent?  On the one hand, you can
argue that 'sync should be sync should be sync, I don't bloody care, just
don't do anything async at all', since that's what it's supposed to do:
mount(8):
  syncAll I/O to the file system should be done synchronously.

How detailed should the man page be?  If it stated "all file data will
be written synchronously, but inodes where the only update is atime
and free block bitmaps are written asynchronously", would that be any
clearer to a user who didn't have a detailed understanding of UFS?
If you would like it to say something different, write some patches
and send them in as a PR (keeping in mind phk's recent e-mail about
green bikesheds).

  sync atime updates will slow it
down, but on the flip side, if you're mounting sync in the first place
you don't care much for speed anyway.

There should be fairly few writes to the root partition, so having
these writes synchronous is not a big performance hit.  On the other
hand, there are probably a _lot_ of read accesses to devices in /dev
and files in /bin (how many of your scripts begin #!/bin/sh?).  Unless
you specify NOATIME, each of these read accesses implies an atime
update within the inode.  Making these synchronous probably would
be a big performance hit.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: {a}sync updates (was Re: make install trick)

1999-10-07 Thread Peter Jeremy

On 1999-Oct-08 08:13:12 +1000, David O'Brien wrote:
 mount(8):
   syncAll I/O to the file system should be done synchronously.
 
 How detailed should the man page be?  If it stated "all file data will
 be written synchronously, but inodes where the only update is atime
 and free block bitmaps are written asynchronously", would that be any
 clearer to a user who didn't have a detailed understanding of UFS?

Yes.  I know the difference between sync/async and data/metadata.

My point was that the average user probably doesn't.  I agree that
the current description glosses over the difference (and probably
shouldn't).  We need to strike a balance between providing enough
detail for the knowledgeable user (who wants to know exactly what
a sync mount does to different types of writes within the FS) and
the novice user (who doesn't understand the details of UFS and
is more likely to become confused).

IMHO, `sync' does behave in a reasonable manner.  I'm not sure how I'd
go about explaining its behaviour to someone who didn't understand the
UFS though.

  Guess I go off editing mount(8) again.

I was going to suggest that the relationship between the sync option
in mount(2,8) and O_FSYNC in open(2) be noted.  Only problem is that
O_FSYNC isn't documented :-(.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Majordomo problems (Re: The eventual fate of BLOCK devices.)

1999-10-10 Thread Peter Jeremy

On 1999-Oct-11 08:35:41 +1000, Oliver Fromme wrote:
Nate Williams wrote in list.freebsd-current:
  Interesting.  It appears that somehow I got 'unsubscribed' from arch.
  Not sure why, but in May I was subscribed, but I'm no longer subscribed.
  
  Did everyone get unsubscribed when it went idle?

FWIW, I got unsubscribed, too.

Same here.

  While I were at it, I sent a
"which" command to the majordomo, and the result did not list
me as a member of freebsd-announce and freebsd-security-
notifications.

I did the same and also found significant differences between the
mailing lists that I am receiving mail from and the ones that
majordomo thinks I am subscribed to.  I sent a note to
[EMAIL PROTECTED] last Friday, but haven't seen anything
back yet (I'm not particularly fussed).

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Proper Whacking Weeding

1999-10-12 Thread Peter Jeremy

On 1999-Oct-13 01:03:54 +1000, Marc van Woerkom wrote:
It took me some fiddling to get a -current system from begin of September 
to build present -current.

It might be useful if you documented exactly what you did.  This is
likely to be a very popular topic for the next few months...

Is there a way to ensure that /usr/include, /usr/share/.. and such
are rebuilt properly? (No files missing, no old files messing things up)

It shouldn't be possible to miss files during an installworld without
the installworld bombing out, though I'm not sure how to actually
confirm this.

I have had a couple of occurrences where my checked-out source tree
got out of step with my CVS tree (I'm not sure how this happened).
My solution here is to occasionally do a checkout into a temporary
tree and compare it with my active tree.  (An alternative would be
to squirrel away my local patches and blow away the active tree).

As for obsolete cruft in the root and /usr filesystems, you could try
"rm -fr /" :-).  Seriously, since installworld installs everything
with the current date, one way is to run "ls -ltr" on all all affected
directories and delete all files older than the installworld.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Deadlock in nbufkv

1999-10-24 Thread Peter Jeremy

I've recently upgraded a system from 3.2-RELEASE to -CURRENT as of
30-Sept (just before the signal changes).  I now find that when
I try to do a CVS checkout, the system hangs, with cvs in `nbufkv'.

The CVSROOT is on a filesystem with standard 8K/1K blocks.  The
target FS is 32k/4k.  Both FS are running softupdates.  This worked
without problem under 3.2.  The kernel config files are basically
the same (modulo config(8) changes).  FWIW, it has 'maxusers 5'
and no other options over-riding default kernel memory sizes.

I notice that Matt Dillon found his FS stress program also hung in
nbufkv, but that was with 64k blocks.  Other than that, I can't
find any reference to nbufkv here, in the PR list or in the sys
commitlogs.

Does anyone have any ideas?  Should I just go back to 8K/1K blocks?

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



SCSI CCB timeouts accessing disk

1999-10-24 Thread Peter Jeremy

I've temporarily attached a Seagate ST32151N to a 1542B on a -current
system to try and setup the disk for another system.  Unfortunately,
although the disk reports it is OK, I can't read or write.  The
relevant probe messages are:

Oct 24 15:05:50 /kernel: aha0 at port 0x334-0x337 irq 11 drq 6 on isa0
Oct 24 15:05:50 /kernel: aha0: AHA-1540/1542 64 head BIOS FW Rev. 0.5 (ID=41) SCSI 
Host Adapter, SCSI ID 7, 16 CCBs
...
Oct 24 15:05:51 /kernel: da0 at aha0 bus 0 target 0 lun 0
Oct 24 15:05:51 /kernel: da0: SEAGATE ST32151N 9470 Fixed Direct Access SCSI-2 
device 
Oct 24 15:05:51 /kernel: da0: 5.000MB/s transfers (5.000MHz, offset 7)
Oct 24 15:05:51 /kernel: da0: 2049MB (4197405 512 byte sectors: 64H 32S/T 2049C)

When I try to read or write the disk (eg `disklabel da0', `dd
if=/dev/rda0 ...' or `dd of=/dev/rda0 ...'), I get:

Oct 24 15:10:39 /kernel: (da0:aha0:0:0:0): CCB 0xc2d78508 - timed out
Oct 24 15:11:21 /kernel: (da0:aha0:0:0:0): CCB 0xc2d78508 - timed out
Oct 24 15:11:21 /kernel: aha0: aha_cmd: Timeout waiting for adapter idle
Oct 24 15:11:21 /kernel: ahainitmboxes: Initialization command failed
Oct 24 15:11:21 /kernel: aha0: No longer in timeout

Does anyone have any ideas?

Peter
--
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: SCSI CCB timeouts accessing disk

1999-10-26 Thread Peter Jeremy

On 1999-Oct-25 07:41:59 +1000, I wrote:
I've temporarily attached a Seagate ST32151N to a 1542B on a -current
system to try and setup the disk for another system.  Unfortunately,
although the disk reports it is OK, I can't read or write.

I finally got it to work by disabling the SYNC negotiation on the
AHA-1542B.  Has anyone seen this before?

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: GENERIC build broken

1999-11-02 Thread Peter Jeremy

On 1999-Nov-03 15:59:38 +1100, Matthew Dillon wrote:
:Well, bootp in the kernel has to die too.

Huh?  And replace it with what?  BOOTP is the only way to get an NFS
root and swap.

Sun uses reverse ARP to do this.  Reverse ARP _is_ a hack, but it _is_
an alternative to BOOTP.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: diskless boot roadmap (was:L Re: GENERIC build broken )

1999-11-02 Thread Peter Jeremy

On 1999-Nov-03 16:15:03 +1100, Mike Smith wrote:
 On 1999-Nov-03 15:59:38 +1100, Matthew Dillon wrote:
 :Well, bootp in the kernel has to die too.
 
 Huh?  And replace it with what?  BOOTP is the only way to get an NFS
 root and swap.
 
 Sun uses reverse ARP to do this.  Reverse ARP _is_ a hack, but it _is_
 an alternative to BOOTP.

RARP would be a really bad move at the moment, since everyone else is
using DHCP.

I'm not suggesting we move to RARP.  Matt was stating that BOOTP is
the only way.

  There's also no call for the kernel 
to do any of this work; the loader will do it all and pass it in to the 
kernel in a format it can use directly.

Sounds excellent.  (Not that I'm offering at this point in time).

Initialisation code in the permanently resident kernel is basically
just wasting physical RAM.  The less of it there is, the better.
How much of the probe code can we move into userland?

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Pageable kernels (was Re: diskless boot roadmap)

1999-11-03 Thread Peter Jeremy

[This is heading more towards -arch than -current]

On 1999-Nov-03 21:53:07 +1100, Richard Wackerbarth wrote:
 How much of the probe code can we move into userland?
Or alternately, transient kernel memory. Use it, then lose it.

IMHO, there are some items (like the PNP ID tables) that really belong
in a user-editable file, rather than the kernel source.  This file
could then be read by /boot/loader (and maybe a runtime daemon similar
to pccardd).

And taking the idea of transient kernel memory a bit further: How
about supporting paging within the kernel?  At least for text space,
this would seem to be a fairly simple matter of tagging functions as
pageable (eg add '__attribute__((section("text_pageable")))' to the
function declaration), and treating that section of KVM the same way
as user text space (though probably with a higher preference for being
resident and a higher priority for paging in).  Paging data may be a
bit trickier, but would seem to be amenable to a mixture of variable
attributes and a new `pageable' flag for kernel malloc.

Initialisation code is an obvious candidate for paging, but I suspect
that everything except interrupt handlers, the paging code and
device drivers associated with the paging devices could be made
pageable.

In fact, the above all sounds so obvious and simple that I wonder
if I'm missing something...

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: CTM-deltas generation sptopped ?

1999-11-03 Thread Peter Jeremy

On 1999-Nov-03 23:58:00 +1100, [EMAIL PROTECTED] wrote:
   There are no new CTM-deltas on 'ctm.freebsd.org'
at least 22 hours.

The last e-mail delta I have is cvs-cur.5804, which arrived here at
0808UTC (about 5 hours before your message).  I would have expected to
receive 5805 (or at least an initial part) about 5 hours ago now, so I
suspect something is wrong.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: show stopper for Gcc 2.95.2 conversion

1999-11-08 Thread Peter Jeremy

On 1999-Nov-09 05:01:43 +1100, Luoqi Chen wrote:
 + "=D" (addr), "=c" (count)   :
 + "r" (bsh + offset), "0" (addr), "1" (count) : 
 + "%eax", "memory");

You may use "+D" and "+c" for the in-out operands,
   "+D" (addr), "+c" (count)   :
   "r" (bsh + offset)  :


Just as a word of caution:  "+" isn't supported prior to gcc 2.8.x,
so you can't use it for code that might be MFC to 3.x.  And just to
make it more of a pain, I've found code similar to the original
patch also fails on gcc 2.7 when the operands are volatile :-(.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ambiguity between -STABLE and -RELEASE

1999-11-08 Thread Peter Jeremy

On 1999-Nov-09 09:53:44 +1100, Theo PAGTZIS wrote:
 If there are bugs that are resolved in 
3.3-STABLE then the 3.4-RC should entail NO new functionality even if this is 
supplemental.

This (presumably) means that new functionality should only be
introduced at major releases (eg 3.0, 4.0 etc).  This is probably
unacceptably slow for most people.  Instead, FreeBSD takes the view
that minor enhancements can be introduced with new minor releases.
In some cases, the line between `minor enahncement' and `bugfix'
can get blurred.

In that sense I would recommend some change in the naming (or rather 
numbering) convention which in my book should be

3.2-RELEASE - 3.3-STABLE - 3.3-RC - 3.3-RELEASE - 3.4-STABLE

and NOT

3.2-RELEASE - 3.2-STABLE - 3.3-RC - 3.3-RELEASE - 3.3-STABLE

The only difference here is how the -STABLE branches are numbered.
The 3.x-STABLE numbering does not exist in the CVS tree and is used
solely as a mechanism to designate positions on the RELENG_3 branch
relative to actual releases (which exist as tags in the CVS tree).

For whatever reason, the decision was taken that a reference to
x.y-STABLE means a location on the RELENG_x CVS branch later than
RELENG_x_y_z_RELEASE.  Changing that at this point in time would
only cause confusion.

It's also worth noting that the -STABLE branch of a release may extend
beyond the last release.  In particular, a couple of serious problems
have been fixed on RELENG_2_2 branch since 2.2.8-RELEASE (and it's
possible that future problems may lead to additional fixes on this
branch), though there is no intention to ever provide a later -RELEASE
off this branch.

 So the RELEASE could be upgradable to the next STABLE by applying a
patch (no CVS interaction here).  I trust that such patches are indeed
existing.

-STABLE is not a fixed point in the CVS tree.  It refers to everything
between x.y-RELEASE and x.(y+1)-RC.  It is fairly simple to produce a
set of patches from the CVS tree to convert say 3.3-RELEASE to the
`current' -STABLE ("cvs diff -u -r RELENG_3_3_0_RELEASE -r RELENG_3"),
but this doesn't help someone who is running -STABLE from last month.
In general, it is assumed that anyone who wants to follow -STABLE will
be tracking the CVS tree.

If you want bugfixes, but can't track -STABLE, you have three options:
- wait for the next -RELEASE
- use the -STABLE snapshots that appear from time-to-time.
- find someone who can provide you with customised support.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



panic: nexus_setup_intr: NULL irq resource!

1999-11-11 Thread Peter Jeremy

I'm trying to enable a generic ISA multiport SIO card in -current from
just before the signal changes and get presented with the above panic
when the first SIO port on the card is attached.  Since it seemed to
be a problem with the resource allocation, I tried turning on
RMAN_DEBUG, but that just suggests that the problem is elsewhere.

The relevant part of my config is:

options COM_MULTIPORT
device  sio0at isa? disable port IO_COM1
device  sio1at isa? port IO_COM2 irq 9
device  sio2at isa? port IO_COM3 irq 3
device  sio3at isa? port 0x2e8 flags 0x605
device  sio4at isa? port 0x2f0 flags 0x605
device  sio5at isa? port 0x3e0 flags 0x605
device  sio6at isa? port 0x2e0 flags 0x605 irq 4

I originally set this up under 2.2.6 with sio3 as the master port,
and it worked (as long as sio0 was disabled).  When it didn't work
under -current, I RTFM and noticed that the master port should be
the highest number, but that didn't help.

The relevant part of the boot response is:

sio0: not probed (disabled)
rman_reserve_resource: I/O ports request: [0x2f8, 0x2ff], length 0x8, flags 0, 
device sio1
considering [0x1f8, 0x333]
truncated region: [0x2f8, 0x300]; size 0x9 (requested 0x8)
candidate region: [0x300, 0x2f8], size 0x9
splitting region in three parts: [0x1f8, 0x2f7]; [0x2f8, 0x2ff]; [0x300, 0x333]
sio1 at port 0x2f8-0x2ff irq 9 flags 0x30 on isa0
rman_reserve_resource: I/O ports request: [0x2f8, 0x2ff], length 0x8, flags 0, 
device sio1
considering [0x1f8, 0x333]
truncated region: [0x2f8, 0x300]; size 0x9 (requested 0x8)
candidate region: [0x300, 0x2f8], size 0x9
splitting region in three parts: [0x1f8, 0x2f7]; [0x2f8, 0x2ff]; [0x300, 0x333]
sio1: type 16550A, console
rman_reserve_resource: Interrupt request lines request: [0x9, 0x9], length 0x1, 
flags 4, device sio1
considering [0x7, 0xa]
truncated region: [0x9, 0xa]; size 0x2 (requested 0x1)
candidate region: [0xa, 0x9], size 0x2
splitting region in three parts: [0x7, 0x8]; [0x9, 0x9]; [0xa, 0xa]
rman_reserve_resource: I/O ports request: [0x3e8, 0x3ef], length 0x8, flags 0, 
device sio2
considering [0x338, 0x3ef]
truncated region: [0x3e8, 0x3ef]; size 0x8 (requested 0x8)
candidate region: [0x3ef, 0x3e8], size 0x8
allocating at the end
sio2 at port 0x3e8-0x3ef irq 3 on isa0
rman_reserve_resource: I/O ports request: [0x3e8, 0x3ef], length 0x8, flags 0, 
device sio2
considering [0x338, 0x3ef]
truncated region: [0x3e8, 0x3ef]; size 0x8 (requested 0x8)
candidate region: [0x3ef, 0x3e8], size 0x8
allocating at the end
sio2: type 16450
rman_reserve_resource: Interrupt request lines request: [0x3, 0x3], length 0x1, 
flags 4, device sio2
considering [0x3, 0x5]
truncated region: [0x3, 0x4]; size 0x2 (requested 0x1)
candidate region: [0x4, 0x3], size 0x2
allocating from the beginning
rman_reserve_resource: I/O ports request: [0x2e8, 0x2ef], length 0x8, flags 0, 
device sio3
considering [0x1f8, 0x2f7]
truncated region: [0x2e8, 0x2f0]; size 0x9 (requested 0x8)
candidate region: [0x2f0, 0x2e8], size 0x9
splitting region in three parts: [0x1f8, 0x2e7]; [0x2e8, 0x2ef]; [0x2f0, 0x2f7]
sio3 at port 0x2e8-0x2ef flags 0x605 on isa0
rman_reserve_resource: I/O ports request: [0x2e8, 0x2ef], length 0x8, flags 0, 
device sio3
considering [0x1f8, 0x2f7]
truncated region: [0x2e8, 0x2f0]; size 0x9 (requested 0x8)
candidate region: [0x2f0, 0x2e8], size 0x9
splitting region in three parts: [0x1f8, 0x2e7]; [0x2e8, 0x2ef]; [0x2f0, 0x2f7]
sio3: type 16550A (multiport)
panic: nexus_setup_intr: NULL irq resource!
Debugger("panic")
Stopped at  Debugger+0x37:  movl$0,in_Debugger
db trace
Debugger(c01f5fe3) at Debugger+0x37
panic(c02072a0,7,c033e0c0,c029ff2c,c029fe94) at panic+0x74
nexus_setup_intr(c04b7280,c04b8b00,0,81,c01e81d8) at nexus_setup_intr+0x19
BUS_SETUP_INTR(c04b7280,c04b8b00,0,81,c01e81d8) at BUS_SETUP_INTR+0x45
isa_setup_intr(c04b7100,c04b8b00,0,81,c01e81d8) at isa_setup_intr+0x26
BUS_SETUP_INTR(c04b7100,c04b8b00,0,81,c01e81d8) at BUS_SETUP_INTR+0x45
sioattach(c04b8b00,c04b8b00,c029ff64,c014b3ef,c04b8b00) at sioattach+0x5d9
DEVICE_ATTACH(c04b8b00,c04b8b00,7,2a4000,c029ff88) at DEVICE_ATTACH+0x33
device_probe_and_attach(c04b8b00) at device_probe_and_attach+0x4f
isa_probe_children(c04b7100) at isa_probe_children+0x47
configure(0,29dc00,2a4000,0,c01191e6) at configure+0x50
mi_startup(c029ffb4,ffe,2a4000,c014fd39,c02a7000) at mi_startup+0x70
begin() at begin+0x4b
db

Any immediate suggestions?

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: panic: nexus_setup_intr: NULL irq resource!

1999-11-11 Thread Peter Jeremy

On 1999-Nov-12 12:35:01 +1100, Mark Newton wrote:
Peter Jeremy wrote:

  I'm trying to enable a generic ISA multiport SIO card in -current from
  just before the signal changes and get presented with the above panic
  when the first SIO port on the card is attached.

Well... the first port that doesn't mention an IRQ.

I can't recall right now whether the panic occurred with sio3 or sio4
when sio3 was master (and had an interrupt).

The problem is that the BUS_SETUP_INTR() method for ISA seems to
absolutely require the specification of an IRQ, even though IRQ
specification is absolutely prohibited for non-master ports in 
AST-compatible multi-port sio cards.  Gah.

That's what the code looked like to me as well.  Since multi-port SIO
cards are supported, but will not have interrupts for the slave ports,
I assumed I was missing something in the maze of nested, indirect
function calls :-).

As an aside, why doesn't our SIO driver work with ports that don't
have interrupts?

I'm not completely sure that this patch does the right thing

Since your patch effectively turns isa_setup_intr() into a nop for
this case, a better patch would seem to be to skip the call to 
BUS_SETUP_INTR() (and presumably bus_alloc_resource()) at the end
of sioattach() when you're attaching a slave SIO port.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FreeBSD security auditing project.

1999-11-23 Thread Peter Jeremy

On 1999-Nov-24 06:35:16 +1100, Kris Kennaway wrote:
 o unsafe use of the str*(3) functions; strcat/strcpy/sprintf c.

I wonder how many instances of the potentially unsafe functions there are
in the source tree? :)

A 'grep | wc' equivalent over the source tree gives:

gets110
strcat 2860
strcpy 4717
strncat 167
strncpy1514
sprintf6839
vsprintf133

Note that (particularly in the case of gets()), this includes the
definition(s) in libraries and declarations in various headers as well
as occurrences in comments, strings and structure/union members.
There are also occurrences in dead or unused code (eg
gnu/usr.bin/as/config/tc-vax.c calls gets() 10 times as well as
referring to it in a comment).

These counts are based on tokens, not strings, so (eg) fgets doesn't
get counted as gets.

A string search for (roughly) "scanf.*%s" also picks up 74 cases of
un-bounded string scans.

And these are the easy ones...

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FreeBSD security auditing project.

1999-11-23 Thread Peter Jeremy

On 1999-Nov-24 09:26:26 +1100, David O'Brien wrote:
  A 'grep | wc' equivalent over the source tree gives:
  
  gets110
  strcat 2860
  strcpy 4717
  strncat 167
  strncpy1514
  sprintf6839
  vsprintf133
 
 *ouch* :-)

This means nothing out of context.  I hope we don't go on a witch hunt.

Agreed.  I wasn't suggesting that all these occurrences are examples
of unsafe use.  They just give an order-of-magnitude indication of
the number of places they are used.

That said, I'm not sure that going through the code and checking every
call to strcpy() (for example) is the right way to go about things.
It _is_ possible to use strcpy() safely, at the same time, it is
possible to use strlcpy() or snprintf() _unsafely_ (mainly mis-
interpreting the return value when the result is larger than the
destination buffer).  In any case, you still miss the case where
someone has implemented their own string copy function and is using it
incorrectly.

I believe the correct way is a line-by-line audit of all the code,
checking for the various security problems.  One thing that would
help with this task is a list of code patterns that indicate security
problems.  This list will make it easier for auditors (and will
expand over time).

I suspect that a 'cvs diff' of the OpenBSD code tree is the best
starting point.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FreeBSD security auditing project.

1999-11-23 Thread Peter Jeremy

On 1999-Nov-24 10:21:17 +1100, [EMAIL PROTECTED] wrote:
a) This is what an unsafe function call looks like

Without checking a lot of the call context, it is very difficult
to categorically state that a particular function call is safe or
not.  As an example, consider the following:

foo(const char *ibuf, ...)
{
char buf[MYBUFSIZ];
...
strcpy(buf, ibuf);
...
}

In general, this call is unsafe because there's no apparent
restriction on the size of ibuf, but in the particular program, it may
be quite safe because the length of ibuf has been checked previously.
In this case, it's probably safer to change the strcpy() to a
strlcpy() or similar - the cost (and risk) of making the change is
probably less than the cost of checking all the places where foo()
is called.  Now consider the case where `buf' is also passed
as an argument - now you don't immediately know the length of
either the source _or_ destination buffers.

And the unsafe code may not be a function call at all.  It's quite
easy to have an off-by-one error when working with arrays.

If you want to look at standard library functions used unsafely, I
think there's a range you need to consider.  At one end you have
"virtually impossible to use safely" (ie [v][f]scanf("...%s..."),
gets(), system() and popen()).  At the other end, you have "fairly
easy to use without introducing buffer overflows" (ie fgets(),
[v]snprintf(), strlcpy()).  The other string functions, [v]sprintf()
and [v]sscanf("...%s...") fall somewhere in the middle.  Note that the
range does not extend to "can't be used unsafely" or even "difficult
to use unsafely" (at least in C).

In fact, I'll go further: If someone can point out a reliable resource
on the Net for a) and b), I'll be happy to write up a first draft of
"The FreeBSD Security Audit for Beginners".  I'm sure that any number
of programmers out there would be happy to review it for technical
accuracy before putting it into circulation.

A good start would be to read the general `secure programming'
information on the web and look for things that are being done
differently.  Aleph One [EMAIL PROTECTED] posted a good
summary in BUGTRAQ last December as Message-id:
[EMAIL PROTECTED]

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



buildworld across signal changes not quite right

1999-11-23 Thread Peter Jeremy

Thanks to Marcel's latest Makefile.inc1 changes (1.92), a -current
buildworld running on an older -current system now progresses much
further - in fact it now completes :-).

There are, however still a few problems - as far as I can tell, these
are all related to the wrong version of perl being called whilst perl
is being built in the " Building everything.." section.

I'm not sure how to fix this problem.  Unlike our other build tools,
perl is not designed to be able to be cross-built:  It builds bits
of itself and assumes they can be safely executed to build other bits.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FreeBSD security auditing project.

1999-11-23 Thread Peter Jeremy

On 1999-Nov-24 12:02:59 +1100, Jordan K. Hubbard wrote:
  I don't see any reason, for
example, why anyone should still be using gets()

To take gets() as an example, of the 110 occurrences that gid found in
-current, the following files contain actual calls to gets() (rather
than declarations, comments, defines etc):

contrib/binutils/gas/hash.c   - only if compiled -DTEST
contrib/cvs/lib/getdate.y - only if compiled -DTEST
contrib/gperf/tests/test.c- part of gperf test suite
contrib/libreadline/tilde.c   - only if compiled -DTEST
contrib/texinfo/info/tilde.c  - only if compiled -DTEST
gnu/lib/libregex/test/fileregex.c - part of libregex test suite
gnu/lib/libregex/test/iregex.c- part of libregex test suite
gnu/usr.bin/as/config/tc-m68k.c   - only if compiled -DTEST1
gnu/usr.bin/as/config/tc-vax.c- only if compiled -Dtest or -DTEST
gnu/usr.bin/tar/getdate.y - only if compiled -DTEST
sys/boot/pc98/boot2/boot.c- asking for boot device
sys/i386/boot/biosboot/boot.c - asking for boot device
sys/i386/boot/cdboot/boot.c   - asking for boot device
sys/kern/vfs_conf.c   - prompting user for root filesystem
sys/pc98/boot/biosboot/boot.c - asking for boot device

So the only live code that contains gets() is in the boot loader
(where space is a serious problem) and when reading a user-specified
root filesystem name in the kernel.  In either case, it's not clear
that exploiting the resultant buffer overflow would allow someone to
gain additional privileges (beyond those they already have as a result
of being able to type input into gets()).

I would prefer to see the gets() in vfs_conf.c go away - the actual
gets() definition is right below the (sole) call to gets() and could
easily be changed to bounds check its input.

The boot code is less obvious.  Adding input bounds checking could
make the difference to the code fitting or not fitting.  This is
probably an area where compliance to Standard C Library interfaces
is less important than code size.

 and our implementation even gets whiney about it if you do.
I like this and have previously suggested that it could probably
be usefully extended to other functions.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FreeBSD security auditing project.

1999-11-23 Thread Peter Jeremy

On 1999-Nov-24 15:33:14 +1100, Brian Fundakowski Feldman wrote:
I'd like to note something.  Strcat isn't necessarily unsafe, and strncat()
isn't necessarily safe.

I wasn't implying that.  In fact, I believe the semantics of strncat()
put it into the `hard to use correctly' category (or maybe `very likely
to be misused').

   if (fscanf(file, "%d:foo:%.*s", smurf, sizeof(something),
   something)  /* This is safe, of course. */
Beep.  You lose.  "%.*s" doesn't exist in *scanf() [I thought it did,
but it's not mentioned in either scanf(3) or the source].  You have
to specify field widths as literals (which makes this sort of code
a real PITA).

#define SNPARGS(buf, len) buf + len, sizeof(buf)  len ? sizeof(buf) - len : 0
char action2[32], proto[47], name[18], fragment[17];
/* Print command name */
snprintf(SNPARGS(name, 0), "ipfw: %d", f ? f-fw_number : -1);

Despite the fact that the buffer name[] was made to be exactly the
largest size, where sprintf() _would_be_safe_,

Not necessarily true.  Consider a system where sizeof(int)==8 (such C
compilers exist today).  In this case "%d" can take 20 characters, but
the code above code assumes an int can always be printed in 11
characters.

  Don't get caught doing this.
If you find a strcat() (for example), see if it's safe.  If it is,
then why replace it?

Confirming that it is safe (checking all the paths by which the
strcat() can be reached) might take substantial effort (if the buffers
and/or range checks are widely separated from the strcat() call.

In addition, someone might add a new path to the strcat(), or might
change a buffer size, without properly checking all the ramifications.

I tend towards the approach that unless it's immediately obvious that
it's safe, you are better off using strlcat() (or maybe strncat()).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ST-506, ESDI and BAD144 ?

1999-11-25 Thread Peter Jeremy

On 1999-Nov-26 05:09:20 +1100, Greg Lehey wrote:
I don't suppose it would be that big a deal to remove the old driver,
but what speaks against leaving it in the tree along with a comment in
the GENERIC config file saying "if you use antediluvial disk hardware,
you may prefer to use an old driver too"?

The primary problem I can see is bitrot.

There seem to be a number of fairly intrusive architecture changes
coming up - block/character device merging, CardBus, `proper' thread
support and SMP improvements.  The WD driver will presumably live long
enough to see the end of the block/character merge.  The other changes
seem further off, but have the potential to require changes to all
drivers.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Process cleanup (was Shared memory ...)

2000-03-01 Thread Peter Jeremy

On 2000-Mar-02 07:10:39 +1100, Lyndon Nerenberg [EMAIL PROTECTED] wrote:
I'm too lazy to look right this second ;-) ... do atexit() functions get
run when a process takes (say) a segmentation fault?

Nope.  If there's no userland signal handler, the process will never
return to userland after a signal.  If you really wanted to, you
could add handlers for all the signals and make them call the
atexit() functions before dying, but that would get messy (since
the process state is probably indeterminate if it got an unexpected
signal).

As various people have pointed out, SysV IPC is mostly an abomination.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ssh strangeness in -current...

2000-03-06 Thread Peter Jeremy

On Mon, 6 Mar 2000, Oliver Fromme wrote:
 Apart from my stupidness of not checking the location of the binary
 first -- what did I do wrong, and what's the recommended way of
 handling this?  Am I supposed to rm /usr/bin/ssh each time I install a
 new release or snapshot?  I can't believe that.

I avoid the problem by structuring my paths along the lines of
$HOME/bin:/usr/local/bin:/usr/bin:/bin (everythere, not just on
FreeBSD).

This way, if I (as sysadmin) install something in /usr/local, it
over-rides whatever the vendor supplied.  (Otherwise, I probably
wouldn't have installed my own version).  Likewise, anything I
put in my private bin directory over-rides anything in the common
areas.

In this case, it would mean that the version of ssh installed
(in /usr/local/bin) from the ports would over-ride the /usr/bin/ssh
in the base system.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ssh strangeness in -current...

2000-03-07 Thread Peter Jeremy

On Mon, 6 Mar 2000, Oliver Fromme wrote:
 Apart from my stupidness of not checking the location of the binary
 first -- what did I do wrong, and what's the recommended way of
 handling this?  Am I supposed to rm /usr/bin/ssh each time I install a
 new release or snapshot?  I can't believe that.

I avoid the problem by structuring my paths along the lines of
$HOME/bin:/usr/local/bin:/usr/bin:/bin (everythere, not just on
FreeBSD).

This way, if I (as sysadmin) install something in /usr/local, it
over-rides whatever the vendor supplied.  (Otherwise, I probably
wouldn't have installed my own version).  Likewise, anything I
put in my private bin directory over-rides anything in the common
areas.

In this case, it would mean that the version of ssh installed
(in /usr/local/bin) from the ports would over-ride the /usr/bin/ssh
in the base system.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: current lockups

2000-03-07 Thread Peter Jeremy

On 2000-Mar-07 06:29:17 +1100, Dave Boers [EMAIL PROTECTED] wrote:
It is rumoured that Arun Sharma had the courage to say:
 Compiling Mozilla with make -j 2 got -current to lock up, twice in
 succession. I'm running a fairly recent snapshot (a week or two old)
 on a Dual celeron box (BP6) with UDMA66 enabled.

Finally. I've been complaining about this on several occasions. I'm also
running UDMA66 and Dual Celeron BP6. No overclocking. 

Later postings mention possible problems with UDMA66.  The other
possibility that has been discussed recently is potential priority
inversions for processes using rtptio and idprio.

Note that ntpd will use rtprio if the Posix P1003.1b extensions aren't
enabled in the kernel.  (These were enabled by default in GENERIC on
i386 in mid-January).  If you have the new ntpd (rather than xntpd)
and are running a kernel without options P1003_1B,
_KPOSIX_PRIORITY_SCHEDULING and _KPOSIX_VERSION=199309L, you could
potentially get a lockup due to a priority inversion.  (Though I
think the probability is very small).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: ssh strangeness in -current...

2000-03-07 Thread Peter Jeremy

On 2000-Mar-08 13:55:45 +1100, Oliver Fromme [EMAIL PROTECTED] 
wrote:
 I also had to remove
/etc/ssh.  Somehow, /usr/local/bin/scp seems to pick up data
from /etc/ssh and tries to invoke /usr/bin/ssh, no matter
what.  :-(

I can't explain that.  My installed-from-ports scp exec's
/usr/local/bin/ssh1 and that ssh doesn't know anything about
/etc/ssh.  Try running "ktrace -i" on the scp and looking at
what is actually exec'd and opened.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: current lockups

2000-03-06 Thread Peter Jeremy

On 2000-Mar-06 21:39:11 +1100, Matthew Sean Thyer [EMAIL PROTECTED] wrote:
My computer had been stable all winter (with setiathome runnning full
time) but suddenly come the Australian summer it started freezing.

And it's been the coldest summer for something like 5 years...

 How about these Peltier (sp ?) cooling devices I have heard about ?

A Peltier cell is just a semiconductor heat pump.  It effectively just
reduces the junction-to-heatsink thermal resistance, allowing you (in
theory) to use a less efficient heatsink (or have the CPU run cooler
with the same heatsink.  The downside is they they're relatively
inefficient - your power supply will need to supply an extra 3-4A at
12v and you need to dissipate that extra power.  Unless you
significantly improve the airflow through the case, you'll probably
find that the internal temperature rises significantly - further
stressing everything except the CPU.

Note that the chip that most needs cooling may not be the CPU - the
big support chips can also run very hot.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: current lockups

2000-03-06 Thread Peter Jeremy

On 2000-Mar-07 06:29:17 +1100, Dave Boers [EMAIL PROTECTED] wrote:
It is rumoured that Arun Sharma had the courage to say:
 Compiling Mozilla with make -j 2 got -current to lock up, twice in
 succession. I'm running a fairly recent snapshot (a week or two old)
 on a Dual celeron box (BP6) with UDMA66 enabled.

Finally. I've been complaining about this on several occasions. I'm also
running UDMA66 and Dual Celeron BP6. No overclocking. 

Later postings mention possible problems with UDMA66.  The other
possibility that has been discussed recently is potential priority
inversions for processes using rtptio and idprio.

Note that ntpd will use rtprio if the Posix P1003.1b extensions aren't
enabled in the kernel.  (These were enabled by default in GENERIC on
i386 in mid-January).  If you have the new ntpd (rather than xntpd)
and are running a kernel without options P1003_1B,
_KPOSIX_PRIORITY_SCHEDULING and _KPOSIX_VERSION=199309L, you could
potentially get a lockup due to a priority inversion.  (Though I
think the probability is very small).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: MAX_UID ?

2000-03-12 Thread Peter Jeremy

On 2000-Mar-13 12:01:03 +1100, Paul Richards [EMAIL PROTECTED] wrote:
id = strtoul(p, (char **)NULL, 10);
if ((errno == ERANGE) || (id = UID_MAX)) {
warnx("%s  max uid value (%lu)", p, UID_MAX);
return (0);
}

You can do this now.  Just add the following:
pid_t   UID_MAX = ~0;
somewhere before the code.

When you see it written out like that the latter is a lot more
informative. It also provides the flexibility to limit the parameters
max value even if the type allows it to be larger. This is of particular
significance to UIDs which are currently limited to a far smaller value
than would fit in a uid_t.

AFAIK, there's no real hard limit.  adduser(8) limits uids to 32000
to prevent portability problems with systems using a signed 16-bit
uid_t (whilst allowing for a few `special' uids near the top of the
range).  Other potential limits are ~65000 (16-bit unsigned) and
9 (5 decimal digits).

It would be nice if there was a system-wide constant that could be
used for this, but it's not obvious where this should be located.
(And note that adduser(8) is a perl script, so it can't use a C
header file without some contortions).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: SMP on Alpha?

2000-03-09 Thread Peter Jeremy

On 2000-Mar-10 12:06:18 +1100, Mikhail Teterin [EMAIL PROTECTED] wrote:
Will the -current version of FreeBSD  run on a multi-CPU axp machine and
use all of  the CPUs?

Not yet, but Real Soon Now.

 Would that  be a reliable box  (assuming the admin
sometimes knows  what he  is doing)?

-current comes with all the usual `not for production use' caveats,
and (without any slur on Doug Rabson, who's doing the work), you'd
be very game taking FreeBSD's first cut at Alpha/SMP and putting it
into production.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: MAX_UID ?

2000-03-12 Thread Peter Jeremy

On Mon, 13 Mar 2000 13:07:47 +1100, Paul Richards [EMAIL PROTECTED] wrote:
Peter Jeremy wrote:
 You can do this now.  Just add the following:
 pid_t   UID_MAX = ~0;
 somewhere before the code.

I assume you meant uid_t,

Ooops, I did :-(.  And this should probably be
uid_t   UID_MAX = ~0L;
to avoid problems if int != long (as John mentioned in passing).

The actual limit is 4294967295, which is the largest number that will
fit into the the 32-bit uid_t. This does work,

I had an idea that NFSv2 only allowed 16-bit UIDs, but I can't find
any code in FreeBSD, or the relevant RFCs, to support this (possibly
some NFS clients/servers ignore the top 16 bits).  I did notice,
however, that VNOVAL (-1) is used to mean the UID isn't present in
some places - and (uid_t)2^32-1 == (uid_t)-1.

On 2000-Mar-13 13:14:40 +1100, Paul Richards [EMAIL PROTECTED] wrote:
#define UID_MAX ((uid_t)0-1)
...
I can see the flaw in that straight away in that uid_t isn't available
in sys/syslimits.h

Not a problem.  C macros are just text expansions.  The `uid_t' isn't
evaluated unless/until the macro is used - at which point uid_t must
be available.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Weak symbols in libc_r broken?

2000-03-12 Thread Peter Jeremy

On 2000-Mar-13 14:45:16 +1100, John Birrell [EMAIL PROTECTED] wrote:
the linker gives me the weak symbol version which refers to
_thread_sys_nanosleep (i.e. the syscall), instead of the nanosleep
function in libc_r.

Out of interest, why does nanosleep appear in libc_r.a as a weak
symbol version of _thread_sys_nanosleep at all?  I would have thought
this was unnecessary (and based on my experiments, undesirable).

Is there someone with a recent -current system who has time to try
this out?

I have -current from 9th March and get the same results you do.
Looking at last night's buildworld results, there doesn't appear to
be any change in the symbols in libc_r.a and when I try manually
performing the link with the ld in /usr/obj/..., I still get the
same result.

... then libc_r is broken.

This is a particularly bad time to discover what appears to be a
fairly serious bug in the threaded libraries and/or linker.

I just tried fiddling with the order in which uthread_nanosleep.o and
nanosleep.o appear in libc_r.a.  It seems that the linker always picks
the first definition of the symbol found after the first reference to
the symbol - whether it is weak or not.  Since nanosleep.o appears
before uthread_nanosleep.o, the non thread-safe version will be
picked.

I suspect the only solution is to dispose of the weak symbols -
re-ordering the object file isn't an option given the requirement
for the non-weak symbol to always appear between the reference and
the weak symbol.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: [ID 20000306.004] unpack(NaN) big baddaboom

2000-03-06 Thread Peter Jeremy

On 2000-Mar-07 05:35:04 +1100, Ian Grigg [EMAIL PROTECTED] wrote:
my $packed = "\0\0\xc0\x7f";
print STDERR "len: ", length($packed), " bytes: ", unpack("H*", $packed), "\n";
my $float = unpack("f", $packed);
print STDERR "float done\n";
print STDERR "float: $float\n";

Looking at the resultant core file, perl is getting SIGFPE inside
cast_i32() (probably when it tries to cast the NaN to an int).
Unfortunately, the code for cast_i32() does not handle NaNs (though it
does handle infinity).  (And it's not clear what integer value should
be associated with NaN in any case).

As to whether it should trap or not, that is less clear-cut.  FreeBSD
takes the view that trying to convert a NaN into an integer is not a
sensible operation, so it signals SIGFPE (the i386 NPX gives an
invalid operand exception).  Other implementations may differ.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: MAX_UID ?

2000-03-13 Thread Peter Jeremy

On 2000-Mar-14 15:42:52 +1100, Bruce Evans [EMAIL PROTECTED] wrote:
On Tue, 14 Mar 2000, Giorgos Keramidas wrote:

 On Mon, Mar 13, 2000 at 05:28:47PM +1100, Bruce Evans wrote:
 ...
  #define isschar(type)  (!isfloat(type)  issigned(type)  sizeof(type) == 1)

 This is marvellous in it's simplicity of interface.
 
 Yet, using sizeof(char) and assuming that it's going to be 1, strikes me
 like a dangerous thing to do.  I have never heard of machines where this

It is sure to be 1 (the C standard requires this).  The problem is
going in the opposite direction -- sizeof(long) may also be 1.

If this was comp.std.c, then I'd agree that such constructs may fail
in some environments.  It's not so clear that we need to worry about
this here (this being FreeBSD).  I believe that most or all such
machines belong to Seymour Cray's legacy (CDC6600, Cray etc), and
aren't likely to be running FreeBSD (or any other free Unix).

I don't see any real impediment to such constructs being used within
FreeBSD.  A comment could be added warning of the problems if the
code is used on `unusual' architectures.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: panic during make depend

2000-03-15 Thread Peter Jeremy

On 2000-Mar-16 15:04:00 +1100, Donn Miller [EMAIL PROTECTED] wrote:
Basically, gcc is a very good compiler.  But, it isn't exactly the
best compiler to use for optimization.  Someone told me that Sun's and
DEC's compilers, for example, blow away gcc in terms of speed.  But,
they aren't portable.

I think it's very dependent on the target architecture and how popular
that architecture is.  Traditionally, the M68K and SPARC families were
the most popular, and gcc generated the best (or equal best) code on
them.  The i386 is now a more popular family but, at the high end,
optimisation is restricted by the difficulty of obtaining accurate
documentation.

The Alpha is a very difficult processor to generate good code for
(and I suspect the IA64 will be worse).  DEC (and now Compaq) have
put a lot of effort into tweaking their compiler - I don't believe
anything like the same amount of effort has been expended on the
gcc backend (and I wouldn't expect it).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Neat kernel development environment.

2000-04-04 Thread Peter Jeremy

On Sat, Apr 01, 2000 at 04:32:36AM +1000, Matthew Dillon wrote:
I can just see all the coolness seeping out.  Now guys, we have to
have as a goal something at least as comparable as what IBM did
with one of their mainframes.  Oh, say, lets shoot for being able to
run 4000 copies or so of linux under VMWare on FreeBSD :-)

Since VMWare is our equivalent of IBM VM/CMS's CP, how about being able
to run 4000 _nested_ copies of FreeBSD :-).  (Run FreeBSD on real H/W,
start VMWare, run FreeBSD under VMware and recurse).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Using packed structs to gain cheap SMP primatives

2000-04-05 Thread Peter Jeremy

On Fri, Mar 31, 2000 at 12:50:51AM +1000, Alfred Perlstein wrote:
What I'm calling for is a vote if we'll rely on this type of behavior
(32 bit stores being atomic with respect to readers) or not, or
perhaps to rely on it but mark it somehow so people can "fix it"
if the need arises later by using other locking primatives on what
should be atomic updates.

I can't imagine any implementation with an N-bit bus where N-bit
aligned transfers were not atomic WRT other N-bit aligned transfers.
It may not be (and probably isn't) guaranteed for non-aligned
transfers, or for smaller bus sizes (anyone for a 386SX SMP box?).

Larger bus sizes are also problematical - they may require some form
of RMW sequence to update an object smaller than the bus.  If more
than one writer is allowed to the bus-word containing the object, some
form of locking will probably be necessary anyway (the Alpha, and
presumably the IA64, special-case 32-bit R/W, but future architectures
may not).

Since we started talking about packed structs, it's worth remembering
that struct { short foo[2];} will only require short alignment and
therefore may not wind up with 32-bit alignment. 

IMHO, we can't rely on 32-bit stores being atomic in the future, but
there will be some size where this will be true.  This suggests that
the objects or object accesses should be marked somehow.  (Of course,
since this size will be the `most efficient' size for that
architecture, it should be C's `int' type).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Kernel trace question (kernel doesn't compile _without_ -O)

2000-04-05 Thread Peter Jeremy

On Wed, Mar 29, 2000 at 04:44:08AM +1000, David Gilbert wrote:
Also... I have noticed that several modules don't want to compile in
the kernel without -O, including kern_synch.c (undefined reference to
__cursig) and atomic.c (inconsistent operand constraints in an `asm').

I am aware of problems with atomic.c, but haven't had the time to
fix it properly.  My original problem was trying to make the code
work in both 3.x and 4.x since gcc 2.7.x and egcs/gcc 2.9.x have
imcompatible constraint requirements.

I suspect the current approach (inline functions) may not be optimal,
but haven't spent enough time experimenting to be certain.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Sysctl

2000-04-12 Thread Peter Jeremy

On Tue, Apr 04, 2000 at 06:43:03PM +1000, Omachonu Ogali wrote:
I don't know whether to hand this to -doc or here, so I'll take the risk
of here, patch simply adds comments to sysctl variables.

Before you get too enthusiastic about adding description strings to the
declarations, you might like to work out:
1) How to store the descriptions so they are accessible to sysctl(8),
   but don't bloat the loaded kernel image.
2) How sysctl(8) should locate the descriptions.

Note that your solution needs to include support for dynamically
loaded sysctls.

This issue tends to come up regularly.  The last thread I recall was
at the end of November last year in a thread on cvs-all.  Have a look
at http://people.freebsd.org/~green/sysctl.descr.patch for one
implementation.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: MLEN and crashes

2000-04-13 Thread Peter Jeremy

On  3/04, John Polstra wrote:
[don't allocate big structs on kernel stack]

Many years ago, I wrote a tool that analysed stack requirements by
parsing the assembler output from the compiler.  It determined the
stack frame requirements and built a call flow graph to determine
total stack depth.  It had some hooks to allow indirect function
calls to be specified manually.  It couldn't handle alloca() (and
equivalents), but they were forbidden by the design standards.

Anyone who does embedded work probably has something similar (or maybe
better). 

It occurs to me that something along these lines would be quite useful
for picking overly expensive (in stack space) subroutine threads
within the kernel.  (And maybe identifying areas where we could
safely allow malloc'd structs to be made auto).

The downside is that indirect function calls would need to be manually
identified - which could be a significant amount of effort.

What are other people's opinions on the usefulness of something
like this?

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: Problems with MAKEDEV.

2000-04-14 Thread Peter Jeremy

On 2000-Apr-14 22:49:40 +1000, Steve Ames [EMAIL PROTECTED] wrote:
That's always struck me a bit odd... I thought 'MAKEDEV std' made
the generic set of devices and that 'MAKEDEV all' should make... well..
_ALL_. *shrug*

What do you define as `all'?  Say I have a big FTP server with 8 wide
SCSI controllers, each with 15 disks - that's da0..da119.  I might
have a big shell (or similar) server that needs a few thousand PTYs.
I could have all sorts of other wierd hardware.  "MAKEDEV all" has to
draw the line somewhere.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: MLEN and crashes

2000-04-15 Thread Peter Jeremy

On 2000-Apr-14 23:40:53 +1000, Poul-Henning Kamp [EMAIL PROTECTED] wrote:
In message [EMAIL PROTECTED], Peter Jeremy write
s:
Many years ago, I wrote a tool that analysed stack requirements by
parsing the assembler output from the compiler.
...
Commit it either as a general tool or as a kernel targeted tool
under src/tools.  And the faster the better :-)

I'll have to see if I can still find it.  In any case, it was
designed to handle M68K assembler from a Microtec compiler.  It
would need some re-work before it could work with FreeBSD.

On 2000-Apr-14 23:49:58 +1000, Thomas David Rivers [EMAIL PROTECTED] wrote:
 How did you address recursive functions, or were they also forbidden
 by design standards?

They were forbidden from memory.  (Which will be a nuisance here).
They would show up as loops in the call graph.

Even if I can't find my previous code, the design of such a tool is
fairly trivial (given the assembler, or a suitably patched compiler).
The first part reads the assembler: When a function is defined, you
note the local framesize and keep track of other stack pushes/pops to
work out total stack requirements for the function.  When a function
is called, you output a triplet of caller, called function and stack
usage.  (From memory I cheated a bit and output caller/called pairs,
with a separate maximum stack used line).  This information should
be fairly trivially available within the compiler as an alternative.

The second part reads all the output from the first part and forms a
call flow graph, generating maximum stack needs from each node.  The
top level node(s) give you the total stack needs.

One reason for splitting it into two bits is to make it easier to
add manual entries for indirect function calls.  (Some of this
`manual' work could be partially automated).

Recursive calls (which there are a fair number of in the FreeBSD
kernel) aren't that easily handled and would probably entail
manual (or semi-automatic) editing.

I'll have a go at a more detailed design (and see if I can find
or re-implement it) if no-one else has one already.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: FreeBSD Build status

2000-04-17 Thread Peter Jeremy

On 2000-Apr-18 08:07:45 +1000, "Jordan K. Hubbard" [EMAIL PROTECTED] wrote:
 As for the lists being tedious and long: I've sorted the content by
 relevance, and it was my hope that over time they would shrink to
 zero if we annoyed people enough with them.

I think that's too much annoyance, really.  I can see most people
simply unsubscribing from -current in the face of a mail that long and
tedious

IMHO, it's not such a bad idea.  freebsd-current _is_ the place to
report problems with -current, why can't the report be produced by a
daemon?  All the MUA's I've ever used allowed me to delete a mail
before reading all of it, so I don't see that having detailed
information (at the end) is a real issue (though it might be for
someove paying by the byte).

Comments:
- I hope it has a well-defined subject so it can be easily recognized.
- I presume it's built using the default make.conf.
- It would be nice if it reported changes since the previous day (or
  successful build).  This would make it easier to see what has been
  broken recently.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: asm_pci.h,v Holy cow!

2000-04-27 Thread Peter Jeremy

On Tue, Apr 25, 2000 at 09:48:18PM +1000, Sheldon Hearn wrote:
If that's the _only_ point, then Garrett Wollman's idea should work
perfectly.  Stick the files under CVS, just agree that they should
never be revised, but rather that new versions should be imported in a
different directory and the old versions punted to the Attic.

AFAIK, ctm (not sure about CVSup) doesn't support a rename function -
when a file is deleted into the Attic, ctm deletes the old file and
then creates a new file in the Attic (transferring the content).  So
this approach probably wouldn't solve Julian's immediate problem.

Apart from that, this approach seems significantly better than
(effectively) committing diffs to binary files.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: OpenSSL asm optimizations

2000-04-27 Thread Peter Jeremy

On Fri, 21 Apr 2000, Kris Kennaway wrote:
 OpenSSL includes asm code for several platforms to speed up various
 operations. Currently we don't build any of this - the attached patch
 turns on asm code for Pentiums and above (it relies on an uncommitted
 patch to sys.mk which defined MACHINE_CPU ?= i386). Set MACHINE_CPU to
 "i586" or "i686" (both are actually identical at present) and rebuild.

On Sat, Apr 22, 2000 at 02:35:51PM +1000, Matthew N. Dodd wrote:
Can these be turned on at runtime?

Not the way the libraries are currently structured.  There are a
number of libraries where we would get significant speedups by
supporting target-dependent code.  I can think of three possible
ways of supporting this:
1) Use machine-depend shared libraries to replace functions in the
   standard shared libraries.  This approach is used on Solaris -
   the rtld automatically loads a machine-specific library (if it
   exists) before libc.so.  The disadvantages are:
   - no support for static executables or in-line functions
   - slower startup due to the extra libraries (particularly if
 Kris's idea of an ordered list of machine architectures)
   - increased VM usage due to multiple function versions in
 the process address space
   - I'm not sure how difficult it would be to integrate into
 FreeBSD's lazy binding scheme
2) Use indirect function calls, with run-time initialisation to
   setup the pointers.  This approach is used in the kernel for
   bzero/bcopy.  The disadvantages are:
   - no support for in-line functions
   - need to invoke a library initialisation routine (not too
 difficult with ELF)
   - increased function call overheads (indirect rather than
 direct calls).
   - increased VM usage due to multiple function versions in
 the process address space
3) Have separate library versions for each target.
   - Significant increase in disk space occupied by libraries

All the approaches increase the build time since multiple copies
of functions need to be built.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: mktemp() patch

2000-06-18 Thread Peter Jeremy

As one of the proponents of the change, my apologies for not taking
part in this thread earlier - I am way behind in my reading of most of
the lists.

On Thu, Jun 08, 2000 at 11:11:42AM +1000, Kris Kennaway wrote:
Instead of using only alphabetic characters, the patch uses the following
character set:

0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz@#%^-_=+:,.~

This was discussed in -audit as well - and most of the points brought
up here have already been discussed there.

 Peter, I hope I've got the right version of the patch

Looks about right.  The character set matches my last suggestion (the
only other item was making base64 and padchar `const').

On Thu, Jun 08, 2000 at 04:08:18PM +1000, Matthew Dillon wrote:
It would be a good idea to avoid any punctuation.

Could you please provide some rationale for this statement.  I agree
there are valid reasons for not using some specific characters, but I
don't see any justification for a blanket ban.

On Fri, Jun 09, 2000 at 04:15:45AM +1000, Garance A Drosihn wrote:
context that reads blah@blah:blah as userid@hostname:fname

Good point.  I hadn't considered the impact of letting [rs]cp
loose on the output from mktemp.

At the same time, I do like to see the set expanded as much as
possible, without causing any problems.

That was my rationale as well.  I went through the 7-bit printable set
and wiped out the characters I though would cause problems.

On Thu, Jun 08, 2000 at 09:51:34PM +1000, Bruce Evans wrote:
Why are we still using the pid?  It is highly non-random.  It was
originally used to ensure a separate starting point for separate
processes,

The other advantage is that it was easy to locate and delete temp
files associated with dead processes.  Given the change to a BASE64
PID, this is no longer true and I agree with Kris that it can be
dropped.  If you need to clean up after dead processes, you'll need
to use lsof or fstat to work out what files are still wanted.

On Fri, Jun 09, 2000 at 01:48:20PM +1000, Kris Kennaway wrote:
I'm not sure that weird filesystems are a valid argument against mktemp()
naming - there are LOTS of UNIX code which assumes UNIX namespace
conventions, and it's not just mktemp() which is going to break on weird
filesystems. For example, should we limit all FreeBSD file names to 8.3
single-case in case someone wants to run from an old-style MSDOS
partition?

I agree with Kris here.  I briefly considered the problem of support
for non-UFS filesystems and decided that it was reasonable for
standard Unix utilities to make use of the facilities offered by the
native filesystem - ie UFS.

In any case, I believe that mktemp(3) will always return a valid
filename for the filesystem (though it might take some time to pick
one).  _gettemp() [the function underlying mktemp(3), mkstemp(3) etc]
invokes lstat(2) on each sample path and requires that lstat() returns
ENOENT for it to succeed.  If lstat() is given an invalid pathname, it
should return something like EINVAL - which will make _gettemp() try
again.  If a filesystem implementation returns ENOENT when given a
pathname that is not valid on that filesystem, then the filesystem
implementation is broken and should be fixed.

Note that AFAIK, currently FreeBSD cannot meet mkstemp guarantees on
_any_ non-local filesystem (and some shared filesystem types) because
the relevant locking mechanisms either don't exist or don't work.

On Fri, Jun 09, 2000 at 02:57:02PM +1000, Garance A Drosihn wrote:
Should the new mktemp check some kind of environment variable,
and use a different list (or maybe even a totally different
algorithm) depending on the value?

I think that selecting between a number of different algorithms based
on an environment variable might be over-kill.  However it could be
useful to allow the user to control the character set used by mktemp()
via an environment variable.  For example
MKTEMP_CHARSET="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._-"
would give you the POSIX portable filename character set (apart from
the `no leading hyphen' bit).  Maybe support two variables - one to
defines the allowable initial characters and another to define the
allowable following characters if it's different.

The downside is that this means that the user can pretty much control
the temporary filenames used by processes - which means it would need
to be disabled for setuid (and maybe setgid) processes.

On Fri, Jun 09, 2000 at 03:08:57PM +1000, Dan Nelson wrote:
I still suggest not using symbols at all, since I'd like to be able to
quickly remove tempfiles by hand without worrying if I have to escape #
or ^, etc.  ...
I'd rather stick with easy-to-read and type tempnames.

`easy-to-read and type' is basically incompatible with
`hard-to-guess'.  I'd suggest that most people will be deleting mktemp
filenames using wildcards, shell-completion or cut-and-paste.  Any
shell that can't understand a filename that it has globbed is broken.
Similarly, any shell that 

Re: cvs commit: src/sys/contrib/softupdates softdep.h ffs_softdep.c

2000-06-22 Thread Peter Jeremy

On 2000-Jun-22 15:22:12 -0500, Chris Dillon [EMAIL PROTECTED] wrote:
I think it would be a very good idea to enable softupdates by default
when a new filesystem is created.  Modify newfs to do this and use
tunefs only if you want to _disable_ softupdates on a filesystem.  

My only concern with doing this is softupdates behaviour when disk
space is freed.  If you delete a file, it takes about 20 seconds for
the space occupied by the file to appear in the free list.  If you
have relatively little free disk space and write another file, you
can get `disk full', even though there will be sufficient free space
when the freed blocks become available.

This has bitten a number of people who have turned softupdates on for
their root filesystems - and had installworld die.

Whilst the solution to this appears obvious (if you can't allocate a
block, but there are free blocks on the to-be-commited list, wait for
free space to become available), actually implementing it is quite
difficult if you want to avoid deadlocks.  Kirk has previously
recommended that softupdates not be enabled on a filesystem unless it
has sufficient free space to absorb about 1 minute's writes.

A number of people have also claimed that softupdates interacts with
Vinum (particularly RAID-5) in undesirable ways.  I believe that at
least some of this was a misunderstanding of the way softupdates
handles write re-ordering (or at least the write semantics that
softupdates expects).

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Re: dc driver and underruns (was: Strangeness with 4.0-S)

2000-07-19 Thread Peter Jeremy

On Fri, Jul 14, 2000 at 08:46:40AM +0200, Wilko Bulte wrote:
That theory is not correct, I have seen multiple Alpha machines reporting 
buffer underruns as well. No ATA disk in sight there..

I get the same thing on AS4000/AS4100 machines running Tru64.  I'm
inclined to believe it's a design flaw in the chip.

Peter


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message



Getting from 5.1-RELEASE to RELENG_5_1

2003-08-20 Thread Peter Jeremy
I am looking at moving various hosts from 4.x to 5.1 but have run into
a problem with my test machine.  I've successfully installed
5.1-RELEASE (from CD) but want to rebuild the system to customise it
to its environment.

The machine in question does not have enough local disk space to hold
both /usr/src and /usr/obj.  When I tried to NFS mount the space, the
'make buildworld' would consistently wedge the machine in stage 1:
bootstrap tools building games/fortune/strfile.  I tried using local
disk for /usr/obj and NFS mounting just /usr/src.  This got somewhat
further but again wedged the machine.  Wedged means no response via
network or local keyboard, needing reset to recover.

Has anyone else seen NFS problems with 5.1-RELEASE?  The client is a
P-133 with 96MB RAM.  The server is a PIII running roughly 4.6.  They
are both using 100baseTX full-duplex 802.1Q trunks to a common switch.

My alternative (preferred) option is to do a build world/kernel on
another faster machine (the NFS server used above).  This fails with
multiple definitions of _sigaction and _sigprocmask building
libpthread.  Looking back thru the archives, I gather this has been
fixed in -CURRENT but it's not in RELENG_5_1 because it's not
security-related.

Any suggestions for a way forward?

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: cvsup failing with ASSERT failed

2003-08-27 Thread Peter Jeremy
On Tue, Aug 26, 2003 at 11:02:08PM -0500, [EMAIL PROTECTED] wrote:
A couple of times, now, on both FBSD-5.1-CURRENT and FBSD-4.8-STABLE whilst 
running with 2MB of RAM, cvsup has croaked with the following error:

Out of interest, how do you get either 4.x or 5.x to run in 2MB?  I
found running 2.x in 4MB was painful and my 4.x kernel is more than 2MB.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: aac related panics

2003-08-30 Thread Peter Jeremy
On Fri, Aug 29, 2003 at 10:54:10PM -0500, Chip Norkus wrote:
My company is working on a new hosting infrastructure, and I'd like to use 
FreeBSD if possible, so any help at all would be greatly appreciated as 
we plan to use these machines for some time.

Note that 5.x is not yet production quality.  It is still primarily
intended for early adopters to help shake out any bugs.  Unless you
specifically need 5.x features, you might find it less painful to
start with 4.x as your main platform and just experiment with 5.x on
some non-critical boxes.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: .fsck_snapshot file

2003-09-02 Thread Peter Jeremy
On Tue, Sep 02, 2003 at 08:27:00AM +0200, Christoph Kukulies wrote:
I have a file .fsck_snapshot in /usr (of 7 GB ?!)
-r   1 root   wheel  7220781056 Aug 22 18:08 .fsck_snapshot

The '7GB' does not mean you'll free up 7GB of disk space by freeing
it.  IIRC, it's actually the size of the filesystem.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


NFS writes over a VLAN trunk wedges system

2003-09-03 Thread Peter Jeremy
I have a system running 5.1-RELEASE-p2 which is an NFS client of
another FreeBSD (4.x) machine.  When I have the NFS mount via a VLAN
the system reliably hangs (no response to console, including
Ctrl-Alt-Esc).  This is a default NFS mount (no options) and I am
trying to do a buildworld with /usr/{src,obj} NFS mounted.  The VLAN
is using an Intel fxp NIC.  The kernel has DEVICE_POLLING specified
but not enabled.

- NFS mount to the same server via normal Ethernet (using a 3Com 905
  NIC) successfully manages buildworld
- Adding WITNESS and INVARIANTS didn't help (though it lasted longer
  before dying)
- Enabling device polling didn't help
- Adding DISABLE_PSE allows it to complete a buildworld

Any suggestions on where to go next?

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: ATAng and CF cards

2003-09-08 Thread Peter Jeremy
On Mon, Sep 08, 2003 at 05:30:28AM +0200, YazzY wrote:
Isn't the ATAng code great?
It makes it affordable to get a 9007199253773098MB CF for the price of a 
 32 MB card.
Now I am taking backups of the internet on it.
:)

The old ATA code (in -stable) can only manage to expand my 3102MB disk
to 43402MB.  I might install -CURRENT into the spare 40GB and see if I
can manage to expand it by a few billion TB :-).

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: new rc system

2003-09-08 Thread Peter Jeremy
On Mon, Sep 08, 2003 at 12:59:11PM +0200, Philipp Grau wrote:
Next problem is that /etc/rc.d/ntpd is evaluated before /etc/rc.d/devfs (see
the output of rcorder /etc/rc.d*) So the start of ntpd fails because it is
requires the devfs link. Ntpd likes to open /dev/refclock-0.

What should I do to circumvent this problem? By simply adding a 
# REQUIRE: devfs to the /etc/rc.d/ntpd file? Or this there some other
mechanism? 

This is a known shortcoming in the new rc system.  Luke Mewburn
commented on it in a talk recently but does not yet have a
satisfactory solution.

Assuming there is no implicit dependency on ntpd by devfs, adding
# REQUIRE: devfs to /etc/rc.d/ntpd should work as would adding
# BEFORE: ntpd to /etc/rc.d/devfs.  You will need to remember
to merge this change into future updates.  Another option may be to
create a dummy new rc.d file that contains both the above lines.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: new rc system

2003-09-11 Thread Peter Jeremy
On Wed, Sep 10, 2003 at 05:00:49PM -0700, Doug Barton wrote:
[re-ordering rc.d scripts]
 This is a known shortcoming in the new rc system.  Luke Mewburn
 commented on it in a talk recently but does not yet have a
 satisfactory solution.

Can you describe in more detail what you mean by this is a known
shortcoming?

The files in /etc/rc.d/ include dependency information in the form of
'BEFORE' and 'REQUIRE' entries.  The default entries are appropriate
for normal configurations but may require changes in some cases (eg
Philipp's situation).

The new rc system currently has no mechanism for over-riding these
defaults other then by editing the individual rc files.  These changes
need to be re-merged if the rc files are updated.

Luke is currently looking at options to allow administrators to alter
the dependency order without requiring the rc files to be edited.  Two
possibilities are:
1) An option to rcorder that allows dependency information to be
   included on the command line.
2) Add a hack to rcorder so that given a file /etc/rc.d/foo, it will
   check for dependency information in /etc/rc.cnf/foo.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Bad performance

2003-09-13 Thread Peter Jeremy
On Sat, Sep 13, 2003 at 02:52:29PM +0200, sebastian ssmoller wrote:
then i moved the disk from the hardware used during install into the
production environment which includes VIA 82C8363 (Apollo KT133A)
...
everything worked fine again. BUT: launching gdm needs a lot of time,
same for gnome2. when i start moz-firebird i am unabled to use it for
minutes (!) until it reacts on user events (typing inet adress into
address bar), same for gaim.

I agree with the general concensus that this shows all the symptoms of
a network or DNS problem - though the switch from SIS to nVidia may
have disturbed X.

Did you change any system configuration (hostname etc) when you moved
the disk?  Is the 'production' environment identical network-wise to
your test environment?  Have you re-configured X to use the different
video card?

How are you starting gdm, gnome2 etc?  I gather gdm isn't started
via /etc/ttys but manually from a vty.  I presume you are using gdm
to start X.

Can you log in from a second system?  If so, what is happening during
the startup delay?  Does top show the system is very heavily loaded or
doing nothing (all processes waiting)?

Before you start gdm, can you ping your system by hostname?  Are there
any other hostname mentioned in your gdm configuration file?  Can you
ping them all?

Have you checked your /etc/nsswitch.conf and /etc/resolv.conf?  Is the
output from 'ifconfig -a' and 'netstat -r' correct?

Have a look through all the files in /var/log that have been updated
recently and check for errors - especially XFree86.0.log, daemon and
messages.  Have a look in the gdm log file (I'm not sure where this
is by default).  Are there any messages on either the console or
vty from which you started gdm?  (Use Ctrl-Alt-Fn to get from X to
vtyn and then Alt-Fn to switch between vtys.  You can use ScrollLock
and PgUp/PgDn/Up/Down to scroll back.  Press ScrollLock again to
get back to normal).

Is any part of your system NFS-mounted?  Is X using a fontserver?
Are all these servers responding?

Are you running a GENERIC or custom kernel?  Do you have any firewall
functions enabled?

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: mystery kernel spew

2003-09-15 Thread Peter Jeremy
On Sun, Sep 14, 2003 at 07:13:09PM -0700, Doug White wrote:
Incidentally, if you are getting wrapping even without this, you can use a
serial console to capture the output.  I've had to do this for doing nasty
ACPI debugging with lots of the options enabled.

For kernel spew, you can also increase the kernel message buffer:
# Size of the kernel message buffer.  Should be N * pagesize.
options MSGBUF_SIZE=40960

And for the VTYs, you can increase the scroll-back size:
options SC_HISTORY_SIZE=200 # number of history buffer lines

Both these are kernel compile-time options.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Bad performance

2003-09-18 Thread Peter Jeremy
On Wed, Sep 17, 2003 at 09:15:50PM -0600, Scott Long wrote:
sebastian ssmoller wrote:
here is my lmmon output. 

 Motherboard Temp   Voltages

 255C / 491F / 528KVcore1:   +3.984V
   Vcore2:   +3.984V
Fan Speeds + 3.3V:   +3.984V
   + 5.0V:   +6.654V
1:0 rpm+12.0V:  +15.938V
2:0 rpm-12.0V:  -15.938V
3:0 rpm- 5.0V:   -6.654V
...
I wonder if lmmon is unaware that acpi provides temperatures in 
deciKelvins, not Kelvins.  25.5C would be a much more reasonable value,
though my system typically runs in the 40-50C range.

528 deciKelvin == 52.8K  == -220C which isn't reasonable for a system
temperature unless you have cryogenic cooling.

OTOH, 255 == 0xff, 3.984V == 255/256*4V, 15.938V = 255/256*16V
These all look like dummy values - as does 0rpm.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: 4.9 stability update

2003-09-19 Thread Peter Jeremy
On Thu, Sep 18, 2003 at 11:21:20PM -0600, Scott Long wrote:
We'd like to get a new poll on the stability and readiness of 4.9.

Last night I upgraded my laptop and it hung partway thru the boot
(immediately after the pcm0: line in the dmesg below).  Powering
it off and back on made it boot normally.  It's not done this
before and I don't know whether to blame this on a glitch or the
new kernel.

Copyright (c) 1992-2003 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 4.9-PRERELEASE #0: Thu Sep 18 20:46:38 EST 2003
[EMAIL PROTECTED]:/home/obj/i586/usr/src/sys/pj1592
Timecounter i8254  frequency 1193182 Hz
CPU: Pentium/P55C (quarter-micron) (233.86-MHz 586-class CPU)
  Origin = GenuineIntel  Id = 0x581  Stepping = 1
  Features=0x8001bfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,MMX
real memory  = 100663296 (98304K bytes)
avail memory = 94388224 (92176K bytes)
Preloaded elf kernel kernel at 0xc036e000.
Intel Pentium detected, installing workaround for F00F bug
Using $PIR table, 5 entries at 0xc00f5c00
apm0: APM BIOS on motherboard
apm0: found APM BIOS v1.2, connected at v1.2
npx0: math processor on motherboard
npx0: INT 16 interface
pcib0: Host to PCI bridge on motherboard
pci0: PCI bus on pcib0
isab0: PCI to ISA bridge (vendor=1045 device=c700) at device 1.0 on pci0
isa0: ISA bus on isab0
ohci0: OHCI (generic) USB controller mem 0x4100-0x41000fff irq 11 at device 16.0 
on pci0
usb0: OHCI version 1.0, legacy support
usb0: OHCI (generic) USB controller on ohci0
usb0: USB revision 1.0
uhub0: (0x0e11) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
pcic0: TI PCI-1131 PCI-CardBus Bridge mem 0x7fffe000-0x7fffefff irq 15 at device 
17.0 on pci0
pcic0: TI113X PCI Config Reg: [ring enable][speaker enable][CSC serial isa irq]
pccard0: PC Card 16-bit bus (classic) on pcic0
pcic1: TI PCI-1131 PCI-CardBus Bridge mem 0x7000-0x7fff irq 11 at device 
17.1 on pci0
pcic1: TI113X PCI Config Reg: [ring enable][speaker enable][CSC serial isa irq]
pccard1: PC Card 16-bit bus (classic) on pcic1
pci0: Chips  Technologies 68554 SVGA controller at 18.0
atapci0: Generic PCI ATA controller port 0x1000-0x100f at device 20.0 on pci0
atapci0: Busmastering DMA not supported
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
orm0: Option ROM at iomem 0xc-0xc9fff on isa0
pmtimer0 on isa0
fdc0: NEC 72065B or clone at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: 1440-KB 3.5 drive on fdc0 drive 0
atkbdc0: Keyboard controller (i8042) at port 0x60,0x64 on isa0
atkbd0: AT Keyboard flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
psm0: PS/2 Mouse irq 12 on atkbdc0
psm0: model GlidePoint, device ID 0
vga0: Generic ISA VGA at port 0x3c0-0x3df iomem 0xa-0xb on isa0
sc0: System console at flags 0x100 on isa0
sc0: VGA 12 virtual consoles, flags=0x300
sio0 at port 0x3f8-0x3ff irq 4 on isa0
sio0: type 16550A
pca0 at port 0x40 on isa0
ppc0: Parallel port at port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
lpt0: Printer on ppbus0
lpt0: Interrupt-driven port
ppi0: Parallel I/O on ppbus0
unknown: PNP can't assign resources
pca1: AT-style speaker sound at port 0x61 on isa0
unknown: PNP0303 can't assign resources
unknown: CPQae3d can't assign resources
unknown: PNP0401 can't assign resources
unknown: PNP0501 can't assign resources
unknown: PNP0511 can't assign resources
unknown: PNP0700 can't assign resources
sbc0: ESS ES1878 at port 0x220-0x22f,0x388-0x38b,0x330-0x331 irq 5 drq 1,5 on isa0
pcm0: ESS 18xx DSP on sbc0
ipfw2 initialized, divert enabled, rule-based forwarding enabled, default to accept, 
logging limited to 100 packets/entry by default
DUMMYNET initialized (011031)
pccard: card inserted, slot 0
pccard: card inserted, slot 1
pccard: card removed, slot 0
pccard: card removed, slot 1
ad0: 3102MB IBM-DYKA-23240 [6304/16/63] at ata0-master BIOSPIO
acd0: CDROM UJDA120 at ata0-slave BIOSPIO
Mounting root from ufs:/dev/ad0s1a
pccard: card inserted, slot 0
pccard: card inserted, slot 1

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: 4.9 stability update

2003-09-21 Thread Peter Jeremy
On Fri, Sep 19, 2003 at 06:35:12PM +1000, Peter Jeremy wrote:
On Thu, Sep 18, 2003 at 11:21:20PM -0600, Scott Long wrote:
We'd like to get a new poll on the stability and readiness of 4.9.

Last night I upgraded my laptop and it hung partway thru the boot
(immediately after the pcm0: line in the dmesg below).  Powering
it off and back on made it boot normally.  It's not done this
before and I don't know whether to blame this on a glitch or the
new kernel.

Please ignore this problem and sorry for replying without checking
the mailing list.  (It was a newly built system with a newly updated
CVS repository, but someone forgot the 'cvs update' in between).  I
do have some problems with 4.9 but I'm discussing that in -stable.

Mea culpa.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: What's happened to CDIOCREADAUDIO friends?

2003-09-24 Thread Peter Jeremy
On Tue, Sep 23, 2003 at 09:54:06PM -0400, Michael W. Oliver wrote:
Content-Description: signed data
Well, I didn't know somebody was patching it, so I started using the 
following in ripit.pl (not exactly as below) instead of 'dagrab':

dd if=/dev/acd0t01 ibs=2352 obs=2048 | sox -t raw -r 44100 \
-s -c 2 -w - -t wav -r 44100 -s -c 2 -w track01.wav

Funny thing is, it is a hell of a lot faster than dagrab was.  Before, 
dagrab would read the data in at about the same rate as flac would encode 
it, but now I can rip a 12 track disc in the time it takes flac to encode 
the first 6 tracks.  Go figure...

dagrab reads each block of data multiple times until it gets identical
results.  The code mentions jitter correction but doesn't include
any detailed explanation.  For a more equitable comparison, I suggest
you hack dagrab.c:cd_read_audio() to use read() ISO ioctl().

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Where is my SLIP interface

2003-09-25 Thread Peter Jeremy
On Thu, Sep 25, 2003 at 02:43:21PM +0930, Greg 'groggy' Lehey wrote:
 Perhaps 'The Complete FreeBSD' by Greg Lehey will help out.

Not any more.  I removed that chapter from the book.  The chapter's
available (also covers UUCP) if anybody wants it; just ask.  But it
seems that Willem has already had SLIP up and running.

I can (mostly) understand removing SLIP, but I though UUCP still
had a niche as a way to implement e-mail pull delivery without
losing the envelope - even when you don't have/want IP connectivity.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Where is my SLIP interface

2003-09-25 Thread Peter Jeremy
On Thu, Sep 25, 2003 at 09:39:12AM +0200, Willem Jan Withagen wrote:
But PPP has the same problem:
the interfaces do not show up when listing them with 'ifconfig -a'
And trying to config it:

router# ifconfig sl0 1.2.3.4 4.5.6.7
ifconfig: interface sl0 does not exist

Try 'ifconfig sl0 create' first.  PPP (and presumably SLIP) interfaces
are now cloned on request rather than created statically.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Why is em nic generating interrupts?

2003-10-11 Thread Peter Jeremy
On Fri, Oct 10, 2003 at 12:54:55PM -0800, Terry Lambert wrote:
If your FXP is not generating any interrupts at all, i think that the polling
code in it is probably broken.

Is the polling code in -current different to that in -stable?

I have a system running 4.6-STABLE (or so) with DEVICE_POLLING and
turning kern.polling.enable on/off makes the fxp interrupt count go
from 0 to several thousand/sec.  The system is primarily routing
packets thru various dummynet pipes and natd and is not heavily
loaded.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: hiding e-mail adresses needed badly

2003-10-17 Thread Peter Jeremy
[This may get duplicated if my outgoing work e-mail gets fixed]

On 2003-Oct-16 11:29:36 -0700, Terry Lambert [EMAIL PROTECTED] wrote:
Earthlink often sucks in terms of customer service.  If they would
just designate a couple of common markers as known SPAM, the
problem would have gone away

There's a fine line between 'blocking a couple of common markers'
and arbitrarily blocking domains, IP addresses and all mails containing
specific words - which some large ISPs do.  What's needed is a filter
system that allows users to control what they receive - not one where
the ISP gets to decide what is/isn't delivered.

When W32.Swen first hit, I was getting mailbox near quota messages
if I didn't empty my home mailbox for about 8 hours.  I asked my ISP
when they would be implementing something to let me control what was
delivered into my mailbox and eventually managed to get a we're
looking into the problem response.  I started running fetchmail as a
work-around (which stops the quota DOS but does nothing to help my
download bandwidth).  AFAIK, they still haven't done anything.

And Australia's biggest ISP (Telstra BigPond) is currently getting
unfavourable mentions in Parliament and the media because it's e-mail
system can't cope - users are claiming e-mails are being delayed a
week or more, or just aren't arriving.

people forced to use Earthlink (forced, because no matter where
I go, Earthlink buys up my damn ISP -- no one talks about *that*
monocoluture being a threat).

Mumble years ago, I heard a talk on this phenomenom.  They problem
boils down to ISP interconnect agreements - they generally wind up
meaning the small ISP has to pay the big ISP (or Internet wholesaler)
whatever the big ISP asks because their customers need to exchange
packets with IP addresses owned by the big ISP and the big ISP
doesn't have as much incentive to route packets to the smaller ISP.
This is a positive feedback loop with the bigger ISP absorbing all the
smaller ones.

This is an inherent flaw in a store-with-quota+pickup-transiently
model, which is what any POP3/IMAP4 forces their users into, and
that means *any* ISP, even ones that give you full time connections,
when they refuse to let you run your own mail server, either by
explicitly disallowing it, or by not providing you a static IP.

Optus Internet (my home ISP) state that they block incoming traffic
to TCP/25 to prevent them being being black-listed for allowing
people to run promiscuous SMTP relays.  This is probably at least
partly true.

  A non-quotaed maildrop would fix it.

How do you stop the weenies never deleting e-mail so their mailboxes
grow indefinitely?  A better solution would be a soft-quota'd
maildrop.  As long as you get to it every few days you don't get DOS'd
but if you never delete your mail you get bitten.  Of course, from an
ISP perspective, there's the problem of several thousand mailboxes
each receiving several hundred 200KB mails each day - that's an awful
lot of maildrop disk space to have to find in a hurry.

Can you imagine if someone wrote one of these things to *actively*
target an ISP with a stupid network topology like Earthlink?

Do you know of any ISPs that do a better job of upstream filtering?

  You
could drive the company out of business by chasing all their
subscribers away by denying them the ability to receive communications
from almost anyone else on the Internet.  I'm really surprised these
idiots are unwilling to do anything about saving their business model
from extinction.

The problem is that it doesn't really hurt the ISP - they (typically)
charge for downlink usage, so they're making more money by not blocking
SPAM.  The customers have to put up with it because they know the
competing ISPs aren't any better.

Death of USENET predicted ... Film at 11 can probably be updated.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Random signals in {build,install}world recently?

2003-10-21 Thread Peter Jeremy
On Mon, Oct 20, 2003 at 11:45:21PM -0700, Terry Lambert wrote:
I've noticed a lot of bad problems with Hynix memory lately; your
mileage may vary.  At Whistle we had a problem with memory with Gold
contacts, and didn't have any problems with the ones with Tin.

A good rule of thumb is to make sure that the finish on the DIMM
contacts are the same as the ones on the DIMM socket - both gold or
both tin.  Note that whilst gold doesn't oxidise, it's fairly easy
to make a gold coating so thin that it's gas permeable allowing the
underlying metal to oxidise.

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: ethercons: ethernet console driver for 5-current

2003-10-21 Thread Peter Jeremy
On Mon, Oct 20, 2003 at 12:13:27PM -0400, Robert Watson wrote:
  After reading a FREENIX
paper this summer on a Linux ethernet console driver, I took a pass at
implementing ethernet console support for FreeBSD.

A very worthy cause.  I'm sure this has come up before but I think
you're the first person to produce actual code.  Something that you
don't seem to address is security - which surprised me.

  This driver is similar
to the Linux driver, although not binary-compatible on the wire,
...
As with the Linux driver, communication happens at the ethernet link
layer, using protocol number 0x0666 (entertaining choice).

If Linux is using 0x0666, we should probably pick a different number
since we're not wire compatible.  Though coming up with a common
protocol would be even better.

  In general, the wire protocol is
probably the weakest part of the endeavor, but I'm having trouble finding
documentation for a decent wire console protocol that doesn't come with an
entire network stack attached.

MOP (as you point out later) or LAT have the advantage of being more
standard, but I'm not sure how well documented they are.

  A series of tunables and sysctls is available to tune the
behavior of ethercons:
kern.ethercons.ifnet_raise
kern.ethercons.interface_preference
kern.ethercons.target

Is there any way to specify ifconfig options?  media and mediaopts
in particular may need to be specified to get the interface to talk to
the associated switch.

I presume kern.ethercons.target only specifies the MAC address
inserted into transmitted packets.  Is there any way to restrict the
src address(es) of received packets?  Does ethercons have any concept
of a current session or will it accept incoming packets from anywhere
at any time?  The latter case would seem undesirable as (IMHO) it
makes it too easy to accidently send a command to the wrong system

Peter
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Anyone working on missing sysv* ipc functionality

2001-09-10 Thread Peter Jeremy

[I'm a long way behind with my e-mail]
On 2001-Aug-12 14:22:00 +0200, Michael Reifenberger [EMAIL PROTECTED] wrote:
at least the linux emulation is missing some ipc functionality:
[SEM|SHM]_INFO [SEM|SHM]_STAT.

Whilst not Linux related, there's a lot of general SysV Semaphore
cleanup in PR kern/12014.  Following the latest round of Giant
pushdown's, the patches in the PR don't apply, but I have an
updated-but-untested set of patches.

Peter

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: stdin/out/err changes kill world

2001-09-23 Thread Peter Jeremy

On 2001-Sep-21 10:45:42 +0300, Ruslan Ermilov [EMAIL PROTECTED] wrote:
Also, this error may only be caused if:

1.  The previous ``buildworld'' installed new headers in
${WORLDTMP}/usr/include.

2.  The next ``buildworld'' was run with -DNOCLEAN.
(Only in this case ${WORLDTMP}/usr/include will be
non-empty.)

I have an old -CURRENT system[1] that can't do a buildworld any more,
even with the latest bsd.{prog,lib}.mk changes (1.101 and 1.98).  I
delete /usr/obj before the buildworld, which writes off the above
scenarios.  Is anyone else seeing problems?

[1] From January.  I know it's old but I don't seem to have found the
time to update lately (and I didn't realise it has been that long).

Peter

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: stdin/out/err changes kill world

2001-09-24 Thread Peter Jeremy

On Mon, Sep 24, 2001 at 01:26:37PM +1000, I wrote:
 I have an old -CURRENT system[1] that can't do a buildworld any more,
 even with the latest bsd.{prog,lib}.mk changes (1.101 and 1.98).  I
 delete /usr/obj before the buildworld, which writes off the above
 scenarios.  Is anyone else seeing problems?

On 2001-Sep-24 12:31:30 +0300, Ruslan Ermilov [EMAIL PROTECTED] wrote:
Yes I know.  I have just committed the actual fix:

Thanks for that.  With src/gnu/usr.bin/binutils/ld/Makefile v1.16,
buildworld now works without any other work-around.

Peter

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: kernel won't build - atomic.c/atomic.h errors...

2001-11-12 Thread Peter Jeremy

Sorry for mot responding sooner.

On Wed, Oct 31, 2001 at 09:27:58PM -0600, Jim Bryant wrote:
cc -c -g -pipe  -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes  
-Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual  -fformat-extensions -ansi  
-nostdinc -I-  -I. -I../../.. -I../../../dev -I../../../contrib/dev/acpica 
-I../../../contrib/ipfilter -I../../../../include  -D_KERNEL -include opt_global.h 
-elf  -mpreferred-stack-boundary=2 -fomit-frame-pointer ../../../i386/i386/atomic.c
In file included from ../../../i386/i386/atomic.c:48:
machine/atomic.h: In function `atomic_set_char':
machine/atomic.h:214: inconsistent operand constraints in an `asm'

On Fri, Nov 02, 2001 at 03:38:56PM -0600, Jim Bryant wrote:
Is anyone else seeing this problem?

There is a problem with the way gcc handles asm constraints.  The
quick solution is to compile with `-O'.  For a more detailed analysis
and my then suggested solution, refer to the thread kernel compile
failure without -O option in -current from Jul 2000 (I can't quote
the exact message ID of my response because our firewall helpfully
rewrites it).

Since those patches were written, the code has been cleaned up to
remove support for older variants of gcc, but the constraint changes
for gcc = 2.8 should still be valid.

Peter, VK2PJ

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: problems with 'make buildworld' on current

2001-11-22 Thread Peter Jeremy

On 2001-Nov-22 11:26:19 -0800, Landon Stewart [EMAIL PROTECTED] wrote:
When I do a make buildworld I get an error about an incomplete type for 
field 'inc4_route'.  I've read the /usr/src/UPDATING and found no real 
references to this type of problem.

Am I jumping too far between 4.3-REL and current?
Do I need some compile options?
Do I need the compat4.x.i386 installed?
Any other ideas?

A recent change to netinet/tcp_var.h has broken world.  You probably
need to include net/route.h in systat/tcp.c

Peter

To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message



Re: RFC: mod for 'du'

2001-09-28 Thread Peter Jeremy

On Thu, Sep 27, 2001 at 02:00:26PM -0700, Julian Elischer wrote:
The following patch to replace the linear array (which it realocs if too
small)
(which it scans linearly) with a hash-table can makle a DRASTIC change
to how DU perfomrs for us in this environment.

Sounds good.

I must stress that the new code is slightly slower for small numbers 
of linked files but the old algorythm is O(N^2) and so started taking 
sizable portions of whole days on our filesystems as they grew.

Note: Since this change has no impact on files with only 1 link,
'file' means 'hard-linked file' below. 

The memory requirements also increase significantly - there's a
(NHASHSLOTS * sizeof(ID*) overhead and the larger ID is allocated
singly rather than in blocks.  (Though the avoidance of realloc()
will help here).  The increased number of malloc's also slows
processing of new files.

My guess is that much of the initial slowdown is due to the explicit
malloc/bzero of the hash array.  Since all (non-broken) directories
have multiple hard-links, the hash table will be allocated in all but
degenerate cases, so I don't see any advantage in allocating it at
runtime.  Try the attached instead (note - manually edited patch, use
with caution).

Note that a hash table only has a constant cost when it is sparse.
Once you get large numbers of files (~NHASHSLOTS), the algorithm is
still O(N^2) - it's just a constant factor (~NHASHSLOTS) faster.  For
very large numbers of files, you want a bigger value for NHASHSLOTS -
but that is wasteful (and increases overheads) for small numbers of
files.  To efficiently support a wide range of sizes, you may need to
implement dynamic re-resizing of the hash table - though this needs a
bigger change to the code and re-hashing can be slow.  100,000 is
probably small enough not to be too expensive for small invocations
(though it's probably at the upper end).  Is it big enough for the
large end (eg Vicor)?

Peter


Index: du.c
===
RCS file: /home/ncvs/src/usr.bin/du/du.c,v
retrieving revision 1.21
diff -u -r1.21 du.c
--- du.c2001/09/04 09:43:31 1.21
+++ du.c2001/09/27 20:57:13
@@ -2,6 +2,14 @@
  * Copyright (c) 1989, 1993, 1994
  * The Regents of the University of California.  All rights reserved.
  *
+ * This version of du has been modified by Andre de Bruin and Scott Macy for Vicor.
+ * The change is related to the handling of file links, in the official
+ * release, du keeps a simple linked list of all visited i-nodes with
+ * multiple links. This list is now implemented as a hash table (actually
+ * a fixed size array) with link list nodes providing a 
+ * performance of up to 30% better than the original version.
+ * The only changes are in file create.c and marked with AdB.
+ *
  * This code is derived from software contributed to Berkeley by
  * Chris Newcomb.
  *
@@ -104,6 +112,15 @@
 void   ignoreclean __P((void));
 intignorep __P((FTSENT *));
 
+typedef struct _ID {
+   dev_t   dev;
+   ino_t   inode;
+struct _ID *next;
+} ID;
+
+#define NHASHSLOTS 10
+#define NALLOC 1024
+
 int
 main(argc, argv)
int argc;
@@ -310,35 +327,42 @@
 }
 
 
-typedef struct _ID {
-   dev_t   dev;
-   ino_t   inode;
-} ID;
-
 
 int
 linkchk(p)
FTSENT *p;
 {
static ID *files;
-   static int maxfiles, nfiles;
+   static int nfiles = NALLOC;
-   ID *fp, *start;
+   ID *lp;
ino_t ino;
dev_t dev;
+   int hashval;
+   static ID *ahash[NHASHSLOTS];
 
ino = p-fts_statp-st_ino;
dev = p-fts_statp-st_dev;
-   if ((start = files) != NULL)
-   for (fp = start + nfiles - 1; fp = start; --fp)
-   if (ino == fp-inode  dev == fp-dev)
-   return (1);
+   hashval = ino % NHASHSLOTS;
+
+
+   /* AdB Use simple hash with linklist to record visited inodes */
+   for (lp = ahash[hashval]; lp != NULL; lp = lp-next) {
+   if (ino == lp-inode  dev == lp-dev)
+   return (1);
+   }
+
+   /* Not found, add it */
 
-   if (nfiles == maxfiles  (files = realloc((char *)files,
-   (u_int)(sizeof(ID) * (maxfiles += 128 == NULL)
-   errx(1, can't allocate memory);
+   if (nfiles = NALLOC) {
+   files = malloc(sizeof(ID) * NALLOC));
+   if (files == NULL)
+   errx(1, can't allocate memory);
+   nfiles = 0;
+   }
files[nfiles].inode = ino;
files[nfiles].dev = dev;
+   files[nfiles].next = ahash[hashval];
+   ahash[hashval] = files[nfiles];
++nfiles;
return (0);
 }
 



  1   2   3   4   >