Re: Base packaging
On Fri, 2003-09-19 at 02:02, M. Warner Losh wrote: Why would you want to package sbin? Where do you see this work going? What problems do you think this will solve? Doing things a top level directory at a time isn't very interesting, but since it looks like a demo, perhaps you could sketch out what the polishing you'd envision. Well, as I mentioned in a followup. I'm actually doing this work so that I can use the system mk files for our product development, but I also wanted to be able to registered all the installed files from our products in the pkg db too. Having tried parallel trees, and hacking the ports mk files to get our code directly out of CVS and various other attempts eventually I found out that it's quite easy to link the two together, so that's what I did. So that's the problem it solves :-) I only chose sbin as a demo, you can put the PORTNAME entry in any Makefile and the granularity of the package it creates is going to be based on the content of the pkg-plist and not where in the tree the Makefile is. I'll reply to your other mail with specific points. Paul. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Base packaging
On Fri, 2003-09-19 at 02:10, M. Warner Losh wrote:
P.S. How do you handle the packlist generation? The ports system
doesn't automatically generate these things, as far as I can tell, and
I didn't see anything that you've added to do this.
My agenda, if you will, on this is to deal with:
upgrades: portupgrade can grok packages. If you had a good way to
generate the package list, then we could make it a lot easier to do
binary upgrades. Thie would let me have a big meta-port that
covers all the 'standard' things on the machine, including the os.
Chances are good that some care would need to be taken in
portupgrade to make sure that it doesn't use binaries in place that
will be replaced.
It should be able to do this. Anything you can do with ports I hope
you'll be able to do in the standard tree, i.e. have dependencies and
meta ports etc.
subsetting: with the proper set of subsetting, one could easily
create packages such that they could install just what they
needed. It might be good to have a few like minimal to boot with
rc.d and the like.
This will probably work already. There's no real magic to what I did,
you can convert any makefile to understand ports just by adding PORTNAME
to it, but it doesn't hook into all the ports targets, just fake-pkg. So
basically if you do make install with one of these modified Makefiles
then it will register a package based on the pkg-plist in that
directory. That plist could contain anything you want.
So, you can create a Makefile in the main tree to build a package with
any files you want, you just have to craft the plist e.g. you could have
/usr/src/dists/{minimal,full, sources} etc and in each dir you'd have a
skelton port, with a Makefile, pkg-descr, pkg-plist etc. The pkg-plist
would determine the list of files in each dist. You could of course
break this down to be much more fine grained in the makefiles that
actually build the code to create lots of small packages and then have a
meta package at the dist level to pull them all together.
nopriv: it should be possible to build a release w/o any privs at
all. NetBSD does this with hacks to install and pax to journal
installation stuff in a certain mode and a new mkfs program to take
that info and create a file system image that can be used in the
target environment.
I haven't done any work on this, but it's on my todo list for different
reasons. I'd like to be able to run a dummy install to generate a plist
i.e., if I wanted to package sbin then in /usr/src/sbin I could run make
plist to create the pkg-plist for me based on everything under
/usr/src/sbin, just to save the effort of it being done manually.
To achieve that I'll do something similar to NetBSD and have an option
to install that registers the installation in a file rather than
actually doing it, so you could run this dummyinstall step and then make
package and you'll end up with a load of package files that can be
installed later as root.
Paul.
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Base packaging
On Thu, 2003-09-18 at 11:25, Alexander Leidinger wrote: On Wed, 17 Sep 2003 16:27:03 +0100 Paul Richards [EMAIL PROTECTED] wrote: However, I suspect that a marginally better place to use these would be in the make distribute target that make release uses. This way, the files are already separated out into directory structures, and it may be easier to build complex pkg-plist's with find(1). ALSO, it may be easier to make more fine-grained packages (DISTRIBUTION=foo) with this. I looked into this originally so that I could use the standard BSD make includes for a project in work but I needed some way to have install wrappered so that any files installed by my project were registered in a package. Therefore, I wouldn't want it restricted to just FreeBSD release scripts since I want to be able to use it outside of the FreeBSD tree. We have programs in the ports tree which use our bsd.*.mk infrastructure. Will there be a problem if such a program gets installed from ports (will it try to register itself 2 times)? I don't know, have you got an example port I can look at? Paul. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Base packaging
On Thu, 2003-09-18 at 12:09, Alexander Leidinger wrote: On Thu, 18 Sep 2003 11:28:31 +0100 Paul Richards [EMAIL PROTECTED] wrote: We have programs in the ports tree which use our bsd.*.mk infrastructure. Will there be a problem if such a program gets installed from ports (will it try to register itself 2 times)? I don't know, have you got an example port I can look at? sysutils/portupgrade for sure, probably games/freebsd-games and net/freebsd-uucp too. No they won't. If there's no PORTNAME in the Makefile then the old mk files will behave the same as they always have. The way I'm doing it I modify the install target if PORTNAME is defined. Paul. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Base packaging
I've got a prototype setup that packages the base tree. It turned out to
be very simple. It needs a lot more polishing and testing but it looks
like this can definitely be made to work with just some tidying up and
re-arranging of our existing make files. I've succesfully created
packages of /sbin by adding the following to /usr/src/sbin/Makefile
--
PORTNAME= FreeBSD-sbin
PORTVERSION= 1.0
COMMENT=sbin
CATEGORIES=misc
--
host# pkg_info -Im FreeBSD\*
FreeBSD-sbin-1.0sbin
host#
Patches below:
Similar patches are needed in bsd.lib.mk and bsd.prog.mk
if you need to create packages in leaf directories. I've
tested that but not included the diffs here since it's
more likely a package would be at a higher directory level.
*** bsd.subdir.mk Wed Sep 17 12:47:11 2003
--- new.subdir.mk Wed Sep 17 13:07:01 2003
***
*** 90,95
.if !target(afterinstall)
afterinstall:
.endif
! install: beforeinstall realinstall afterinstall
! .ORDER: beforeinstall realinstall afterinstall
.endif
--- 90,99
.if !target(afterinstall)
afterinstall:
.endif
! .if defined(PORTNAME)
! .include bsd.syspkg.mk
! .else
! install: beforeinstall realinstall afterinstall fake-pkg
! .ORDER: beforeinstall realinstall afterinstall fake-pkg
! .endif
.endif
---
# bsd.syspkg.mk
LOCALBASE=/
WRKDIR=${.OBJDIR}
NO_WRKSUBDIR=YES
NO_CHECKSUM=YES
NO_BUILD=YES
fetch:
extract:
patch:
configure:
install: beforeinstall realinstall afterinstall generate-plist fake-pkg
.include bsd.own.mk
.include ${PORTSDIR}/Mk/bsd.port.mk
intY has scanned this email for all known viruses (www.inty.com)
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Base packaging
On Wed, 2003-09-17 at 15:45, Mark Murray wrote: Paul Richards writes: I've got a prototype setup that packages the base tree. It turned out to be very simple. It needs a lot more polishing and testing but it looks like this can definitely be made to work with just some tidying up and re-arranging of our existing make files. I've succesfully created packages of /sbin by adding the following to /usr/src/sbin/Makefile -- PORTNAME= FreeBSD-sbin PORTVERSION= 1.0 COMMENT=sbin CATEGORIES=misc -- ... etc. This is excellent! However, I suspect that a marginally better place to use these would be in the make distribute target that make release uses. This way, the files are already separated out into directory structures, and it may be easier to build complex pkg-plist's with find(1). ALSO, it may be easier to make more fine-grained packages (DISTRIBUTION=foo) with this. I looked into this originally so that I could use the standard BSD make includes for a project in work but I needed some way to have install wrappered so that any files installed by my project were registered in a package. Therefore, I wouldn't want it restricted to just FreeBSD release scripts since I want to be able to use it outside of the FreeBSD tree. I was thinking of adding an option to install so it registers the file in a plist rather than actually doing the install. A seperate make plist target could then be used as a helper target to automate the generation of plists. If we want to get even more resilient, we could pass a plist file to install and have install abort if the file to install is missing from the plist e.g. return an out of date package error or something. Paul. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Base packaging
On Wed, Sep 17, 2003 at 06:53:41PM +0100, Nik Clayton wrote: On Wednesday, September 17, 2003, at 04:27 pm, Paul Richards wrote: I was thinking of adding an option to install so it registers the file in a plist rather than actually doing the install. A seperate make plist target could then be used as a helper target to automate the generation of plists. I think the NetBSD guys have already done something like this. Luke Mewburn (IIRC) mentioned this in his 'cross building' talk at BSDCon. Yeah, I've been looking at that. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Text file busy
On Thu, 2003-09-04 at 19:20, Tim Kientzle wrote: Depends on how you're installing the binary. It has always been safe to do either of the following: * Rename the current executable and then install the new one. * Unlink the current executable and then install the new one. Many tools that claim to overwrite really do the latter, which causes a certain amount of understandable confusion. (I'm pretty sure install does unlink/copy by default and will do rename/copy if you specify -b.) I thought I remembered a discussion from the very early days where a solution was implemented to copy the pages if a file was overwritten into memory or swap but I can find no record of that now other than a suggestion in a thread that Solaris might do this. I think I'm confusing the above impression with an actual problem that was fixed where you'd still get the error even though the program had stopped executing. Paul. intY has scanned this email for all known viruses (www.inty.com) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Text file busy
Overwriting a file that's currently executing results in a Text file busy error. When did this start happening? This was something that was fixed way back on FreeBSD but it seems to be a problem again. Paul. intY has scanned this email for all known viruses (www.inty.com) ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Email accounts on FreeBSD 5.1-RELEASE
On Fri, Jun 20, 2003 at 05:16:11PM -0400, Alex Ayala wrote: Ok, maybe...yes I read what I wrote and didn't quite explain what I really wanted to say. I want to setup accounts on my box so users can retrieve emails by accessing my pop server. Do I need to setup user accounts on my box with the adduser command? I don't want them to be able to have access to the shell by any means. Is like when I wanted to give someone access to my ftp server I just created an account and took out the shell part in the passwd file. Sorry my english is not the greatest. Trying to explain something and can't find the right words. Is that a bit better to understand? The answer depends a bit on which pop server you're using. If you don't want to create user accounts on the box for mail users then use something like Cyrus, which manages it's own authentication and so doesn't require any system accounts to be created for mail users. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kernel: lnc0: Missed packet -- no reviece buffer QWE
On Wed, Jun 11, 2003 at 09:51:02PM +1000, Anthony Wyatt wrote: Hi Everyone, I'm in the process of hand building a FreeBSD 5.1 CURRENT #2 box. I now have a booting system that I can log onto and use, but my network interface does not work :-( I get lots of: kernel: lnc0: Missed packet -- no recieve buffer and the occasional kernel: lnc0: Device timeout -- resetting Can you send the the dmesg output and pciconf -v -l -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
sh job control
I've installed a current built last night and job control no longer works in /bin/sh or /usr/local/bin/zsh, but it does with csh. ctr-c and ctrl-z are just ignored with both the sh style shells. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: CURRENT console setttings borked
On Wed, Jun 11, 2003 at 11:21:16PM -0400, Andrew Lankford wrote: Info about my buildworld: FreeBSD bogushost2 5.1-CURRENT FreeBSD 5.1-CURRENT #0: Wed Jun 11 21:33:34 EDT 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/ARL5KERNEL i386 In addition to my pppoe/adsl connection no longer working, I find that I cannot use control-c to kill running programs when I'm logged in directly to the console (without invoking stty intr ^c first) yerprompt$ stty #(syscons console, TERM=cons25) speed 115200 baud; lflags: echoe echoke echoctl pendin oflags: -oxtabs cflags: cs8 -parenb discard dsusp eof erase intrkilllnext quitreprint undef undef undef undef undef undef undef undef undef start stopsuspwerase undef undef undef undef Ahh, that's what I'm seeing too, but not with csh, just bourne style shells. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kernel: lnc0: Missed packet -- no reviece buffer QWE
On Wed, Jun 11, 2003 at 03:39:33PM +0100, Bruce Cran wrote: I get this all the time on my FreeBSD 4.8-RELEASE system, which is a P75 with a lnc NIC. The man page does say this driver is one of the more verbose ones, and I think the message about no recieve buffer is just that the system cannot keep up with the incoming data, and so there is no buffer left. Certainly on my system doing NAT across lnc0 and sis0, the system uses 30-40% CPU just to act as a router! I still get 600KB/s through it, which I have heard is fairly reasonable from a 10Mbit ISA network card. That is what the error message means. However, the driver was able to handle 900k-1M load on a 486 when I wrote without maxing out the cpu so there's something wrong somewhere. It might be the sis driver though, I don't know much about that but I've not heard good things about them. The lance does most of it's work on the chip so all that the CPU does is copy the buffers into an mbuf, which shouldn't put much load on the cpu. Is it possible for you to do some testing on that box by disabling each interface in turn and doing some load testing on each individually? -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Panic - blockable sleep lock
Just got this: panic: blockable sleep lock (sleep mutex) sellck @ /usr/src/sys/kern/sys_generic.c:1190 Debugger(panic) Stopped at Debugger+0x54: xchgl %ebx,in_Debugger.0 db t Debugger(c029bc34,c02de7a0,c029e847,df12da20,1) at Debugger+0x54 panic(c029e847,c02a9469,c029f08e,c029f065,4a6) at panic+0xab witness_lock(c03041e0,8,c029f065,4a6,c029cf40) at witness_lock+0xaf _mtx_lock_flags(c03041e0,0,c029f065,4a6,c6594030) at _mtx_lock_flags+0xb1 selwakeup(c6594004,c6594030,73,c6594030,df12dad0) at selwakeup+0x2f ptcwakeup(c6594030,1,df12dadc,c01b0beb,c6594030) at ptcwakeup+0x2e ptsstart(c6594030,df12daf4,c01b2984,c6594030,c6594030) at ptsstart+0x37 ttstart(c6594030,c6594030,7,df12dbf4,df12db10) at ttstart+0x1b tputchar(73,c6594030,1,c02af00f,c21abab0) at tputchar+0x54 putchar(73,df12dbf4,0,c02e1b10,0) at putchar+0x6b kvprintf(c02af00e,c019c650,df12dbf4,a,df12dc14) at kvprintf+0x7d printf(c02af00e,7,c02dfdc0,7,296) at printf+0x57 sched_ithd(7,c02df540,c21aa84c,c21abab0,df12dc58) at sched_ithd+0x63 i386_unpend(570cb146,c02dcae0,c21a9180,df12dcb4,c0184585) at i386_unpend+0xc7 cpu_unpend(a8,df12dcc0,c0184354,c21abab0,df12dce0) at cpu_unpend+0x2a cpu_critical_exit(c21abab0,df12dce0,c017533d,c02dcae0,1) at cpu_critical_exit+0x15 critical_exit(c02dcae0,1,c029b144,1ba,c21a9180) at critical_exit+0x24 _mtx_unlock_spin_flags(c02dcae0,0,c0299a15,231,c21aa790) at _mtx_unlock_spin_flags+0xbd ithread_loop(c21a9180,df12dd48,c02998d2,30c,0) at ithread_loop+0x27a fork_exit(c016b200,c21a9180,df12dd48) at fork_exit+0xc0 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xdf12dd7c, ebp = 0 --- -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Wed, Jun 04, 2003 at 02:43:20PM +0100, Doug Rabson wrote: On Wed, 2003-06-04 at 14:16, Paul Richards wrote: On Wed, Jun 04, 2003 at 01:33:46PM +0100, Doug Rabson wrote: Interfaces actually can be added at runtime. Existing objects (i.e. objects instantiated before the new interface was added) will continue to work as before. If methods from the new interface are called on old objects, the default method will be called. How can you add an interface at runtime? By loading a kernel module. If I load e.g. the agp kernel module, I add the agp_if interface to the kernel. Yes, I know that you can load a pre-compiled interface at runtime and thereby add that interface, but you can only create interfaces at build time because each method in the interface is uniquely identified by a kobjop_desc struct, which is what I was referring to. The code which is doing the method dispatch has no real idea what methods (or what interfaces for that matter) that the object's class implements. You can't use the classes method table layout for the ops table since the caller has no way of knowing that layout (and the layout will be different for almost every class in the system). One possible way of making this slightly simpler might be to make the class point at a table indexed by interface ID, each entry of which is a table indexed by a method ID from that interface. This sounds fine in theory but in practice, it would end up slower due to the two memory accesses. That's along the lines of what I suggested at the start of the thread :-) -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Thu, Jun 05, 2003 at 04:06:16PM +0100, Doug Rabson wrote: On Thu, 2003-06-05 at 15:51, Paul Richards wrote: On Wed, Jun 04, 2003 at 02:43:20PM +0100, Doug Rabson wrote: On Wed, 2003-06-04 at 14:16, Paul Richards wrote: On Wed, Jun 04, 2003 at 01:33:46PM +0100, Doug Rabson wrote: Interfaces actually can be added at runtime. Existing objects (i.e. objects instantiated before the new interface was added) will continue to work as before. If methods from the new interface are called on old objects, the default method will be called. How can you add an interface at runtime? By loading a kernel module. If I load e.g. the agp kernel module, I add the agp_if interface to the kernel. Yes, I know that you can load a pre-compiled interface at runtime and thereby add that interface, but you can only create interfaces at build time because each method in the interface is uniquely identified by a kobjop_desc struct, which is what I was referring to. What has build time got to do with anything? At build time, you have no idea what interfaces are available either. Besides modules can be built at a different time from the rest of the kernel. My point was simply that interfaces can't be created other than at compile time because the kobjop_desc structures are used as unique identifiers. I wasn't making this point in relation to the discussion on knowing what methods are in the class, just from the point that interfaces can't be created or modified at runtime. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Way forward with BIND 8
On Fri, Jun 06, 2003 at 03:01:02AM -0700, Doug Barton wrote: On Fri, 6 Jun 2003, Brad Knowles wrote: At 12:09 AM -0700 2003/06/06, Doug Barton wrote: FYI, for those wondering why I'm not considering BIND 9 for import, please see http://people.freebsd.org/~dougb/whybind8.html I might be able to buy your arguments for supporting BIND 8 instead of BIND 9 in -STABLE, but not in -CURRENT. Regardless of whether I agree with the points you make here or not, the FreeBSD development model requires that what we import in -current, for the most part, be what we plan to eventually MFC. That factor alone eliminates the possibility of importing BIND 9 at this time. Why? There's no basis for assuming that everything that goes into -current must be MFCd. The -current branch is for our next generation version of the OS with all the new whizzy features we might want and BIND9 is therefore exactly the sort of thing to add to -current, with no intention of ever MFCing it. The requirement is that nothing goes direct into -stable, that it must all go through -current first. that doesn't however imply that everything going into -current must be suitable for MFCing. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Wed, Jun 04, 2003 at 01:33:46PM +0100, Doug Rabson wrote: Interfaces actually can be added at runtime. Existing objects (i.e. objects instantiated before the new interface was added) will continue to work as before. If methods from the new interface are called on old objects, the default method will be called. How can you add an interface at runtime? All of this runtime activity is theoretically possible, i.e. all these structures can be messed around with to add methods to existing objects etc. but wasn't kobj supposed to provide an OO type abstraction and therefore the interfaces and classes are determined at compile time. I'm not aware of any actual code that changes interfaces or classes at runtime and if we added that functionality then I think it would break the abstraction and complicate the usage of kobj for no real gain. Coming back to the central point, I don't understand why we need a cache at all. I don't see the reason for having to support the dispatching of anything other than the methods in the class that the object was instantiated into, so that means a relatively small fixed size method dispatch table. The only argument for that not working is if the class gets extended after the object is created but I'm not convinced yet that that's something to worry about in practice at least at the moment since we don't extend classes anywhere. Even if we added code to make that possible at runtime it would be simple enough to get a miss when looking that method up in older objects and to realloc the method table in the object to accomodate the new methods. Extending the class is something that is likely to be so rare that some performance penalty should that happen would be palatable. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Libthr stable enough for testing
On Thu, May 29, 2003 at 06:28:26PM -0400, James Tanis wrote: On Thu, 29 May 2003 17:39:18 -0400 (EDT) John Baldwin [EMAIL PROTECTED] wrote: It has been committed. Build rtld with WITH_LIBMAP defined and then setup a libmap.conf. -- Alright, I compiled and installed libthr, built rtld WITH_LIBMAP, and created the libmap.conf. I am using the example from the man page to have all programs use the libthr library. As far as I can tell my system is running perfectly fine, but there's nothing particularly special about my setup. Is there a way to find out for sure that the programs are now using libthr? As far as I can tell they should be, but I'd like to have a definitive answer. You can use ldd to check the binaries and it will show how libmap affects the library mappings. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Mon, Jun 02, 2003 at 09:04:11AM -0700, Hiten Pandya wrote: On Mon, Jun 02, 2003 at 08:17:03AM -0700, Terry Lambert wrote: Hiten Pandya wrote: My fingers have been itching to do this since the day phk@ planted this idea in my brain (re: cdevsw initialisations). Basically, it changes the vfsops to use C99 sparse format, just like cdevsw. It removes a lot of junk default initialisations, and duplication. I really dislike the changes to vfs_init(). Specifically, it's not the overhead, so much as it's the implied side effects. And how many times is vfc_register() called? Its not in the patch of an I/O operation or anything. Its just a mount time overhead which will go through -- a one time thing. Consider this going forward: someone adds a new VFSOP to the list of allowable VFSOPs, and the vfs_init() doesn't have any specific code for it. You should look at kobj, it's precisely this sort of dynamic dispatching that it was designed to support. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Tue, Jun 03, 2003 at 08:56:59AM -0600, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Paul Richards [EMAIL PROTECTED] writes: : You should look at kobj, it's precisely this sort of dynamic : dispatching that it was designed to support. Too bad it is a little slow and not MP safe in its implementation. Yes, but it has the right design philosophy for this sort of situation and should be researched as an example of how to solve these sorts of problems. I'm not sure the dispatch is slow enough to cause serious overhead in the context of the tasks being performed and it maybe could be improved a little. I'm not sure that kobj actually needs to be MP safe if the kobj struct is always embedded in a structure at a higher level i.e. a use of kobj in say the device driver code will not interact with it's use by say the linker code so the locking can be done on device driver or linker level structs. I think that's the case currently, though perhaps not something we've defined as part of the API, but there's no reason we could not do so and then that simplifies the use of kobj by avoiding having to add locks right at the very bottom of the structure tree where they wouldn't be needed. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Tue, 2003-06-03 at 18:19, M. Warner Losh wrote:
In message: [EMAIL PROTECTED]
Paul Richards [EMAIL PROTECTED] writes:
: I'm not sure that kobj actually needs to be MP safe if the kobj
: struct is always embedded in a structure at a higher level i.e.
: a use of kobj in say the device driver code will not interact with
: it's use by say the linker code so the locking can be done on device
: driver or linker level structs.
Lock lock data, not code.
Agreed, that's what I said :-) However, I was wondering if locking the
structure that the kobj was embedded in was enough to protect the kobj
itself. As you point out below, that's not the case.
The kobj dispatch tables are shared between all instances of the
object. So if you have two instances of the driver, the driver locks
protect softc for that driver. However, both softc's reference the
kobj structures, directly or indirectly, that are not MP safe. If
instance 1 takes out a lock, that doesn't prevent instance 2 from
getting screwed by the following code:
#define KOBJOPLOOKUP(OPS,OP) do { \
kobjop_desc_t _desc = OP##_##desc; \
kobj_method_t *_ce =\
OPS-cache[_desc-id (KOBJ_CACHE_SIZE-1)]; \
if (_ce-desc != _desc) { \
KOBJOPMISS; \
[1] kobj_lookup_method(OPS-cls-methods, _ce, _desc); \
} else {\
KOBJOPHIT; \
} \
[2] _m = _ce-func; \
} while(0)
Notice how OPS has a cache array. If instance 1 is calling the same
ID as instance 2, modulo KOBJ_CACHE_SIZE, then a collision occurs and
a race happens betwen [1] and [2]:
thread 1thread 2
_ce = ...
_ce = ...
HIT
MISS
set's *_ce via kobj_lookup_method
_m gets set
_m called
_m set
_m called
Notice how thread 1's _m gets set based on the results of the kobj
lookup, and we have a race, even if thread1 and thread2 took out their
driver instance locks.
That means we have to lock the dispatch table before every method is
looked up and hold it until the method returns (the alternative would be
to free it inside the method once it had been called but that'd be a
right mess).
Actually, in practice it doesn't matter, though that's an accident of
the kobjs we currently create. The reason being that the hashing
algorithm is really simple and just lops off some high bits of the
method id so unless we have more than 255 methods they always have an
unique slot. There's no mechanism to change the actual function
associated with the method at the moment. Even if there was, it might
not matter as long as the number of methods was still below 255, because
what you'd actually be doing is, say, changing which attach function to
call when you were indirecting through the attach method entry and by
the time thread 1 gets to actually call the method it might even be
considered correct for it to call the newly mapped one.
However, I take your point and we either have to implement locking of
the method table or redesign the caching mechanism to guarantee that
there's no race. I don't think anything other than _ce is a problem (is
there anything else?).
If we ensured that the cache entry couldn't have collisions (perhaps by
not having a hash and instead indexing a unique position in an array
using the method id) then only the latter race would exist (changing the
function associated with a method) and that might be acceptable
behaviour.
The tradeoff with using an index into an array is that there'd be a
heavy penalty for growing the array if an extra method didn't fit, but
that would be exceptionally rare and with our present usage we'd never
have that happen.
Incidentally, if we implement a lock on the method table we'd
effectively have code locks :-)
--
Tis a wise thing to know what is wanted, wiser still to know when
it has been achieved and wisest of all to know when it is unachievable
for then striving is folly. [Magician]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Mon, 2003-06-02 at 21:04, Paul Richards wrote: The tradeoff with using an index into an array is that there'd be a heavy penalty for growing the array if an extra method didn't fit, but that would be exceptionally rare and with our present usage we'd never have that happen. I'm not sure this is actually a problem after all since the Interface doesn't change and therefore we know a-priori how many methods there can be so we can pre-allocate an array. I wonder why Doug didn't do this, perhaps he thought that there'd be very large interfaces and 255 was a reasonable compromise for a cache. However, in practice we'd save a lot of space per kobj by preallocating the actual number of entries we needed for the Interface instead and then we could do away with the _ce problem. This would actually speed up the dispatch a lot too since we wouldn't have to traverse a list looking for a matching method entry and could call the function directly from the method table. Doug, am I missing something? -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Tue, 2003-06-03 at 22:36, Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], John Baldwin writes: On 02-Jun-2003 Paul Richards wrote: On Mon, 2003-06-02 at 21:04, Paul Richards wrote: The tradeoff with using an index into an array is that there'd be a heavy penalty for growing the array if an extra method didn't fit, but that would be exceptionally rare and with our present usage we'd never have that happen. I'm not sure this is actually a problem after all since the Interface doesn't change and therefore we know a-priori how many methods there can be so we can pre-allocate an array. I thought the point in KOBJ was that it was extensible so you could KLD load stuff which added more methods ? Not exactly. It allows for dynamic binding of methods that implement a specified interface. It gives you 2 things mainly: 1) Not all the interface has to be implemented within a particular object, and things will just work if unimplemented parts of the interface are called on an object. 2) The actual functions called for a particular method can by dynamically changed (or at least they could be, I don't think there's actually any API that does this at the moment so in fact they just get mapped the once when the object is instantiated). The possible methods available in an interface are fixed, they're defined in the .m files. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: fxp0: device timeout with 5.1BETA2
On Tue, 2003-06-03 at 22:00, Tobias Roth wrote: Hi I still get a fxp0: device timeout I get these as well. Is it on irq9 by any chance, along with acpi0 ? -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Tue, 2003-06-03 at 23:09, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Paul Richards [EMAIL PROTECTED] writes: : The possible methods available in an interface are fixed, they're : defined in the .m files. No it isn't. One can add additional interfaces at any time without breaking binary compatibility, so you don't know, a priori, the number of methods in a given interface when you build a client of that interface. I don't think that's true. The interface is defined in the .m file and it's created using makeobjs. You can't do that at runtime because it creates the kobj_desc struct that uniquely represents the existence of that method globally for the whole of the kobj subsystem. The set of all kobj_desc defines all possible methods that can be implemented by an object, and yes you can extend that interface later and previously built modules will still work because they only implement a subset of those possible methods, but the set of all methods that can exist is determined at compile time from the interface definitions A class is then defined which specifies the set of methods that objects instantiated into that class *can* implement (but are note required to). The set of methods in a class is also fixed, since it's basically the method table plus some class fields and the method table is created at compile time. Again though, if you later extend the class older compiled modules will still work because if the method doesn't exist in that older module the default from the kobj_desc will be used or the kobj_error_method will be called (which is safe). So yes you can extend the interface and the class and keep backwards compatibility but that all happens at compile time, and therefore at runtime when the object is instantiated you know the maximum number of methods that the object can possibly call. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: fxp0: device timeout with 5.1BETA2
On Tue, 2003-06-03 at 23:17, Tobias Roth wrote: On Mon, Jun 02, 2003 at 10:59:22PM +0100, Paul Richards wrote: On Tue, 2003-06-03 at 22:00, Tobias Roth wrote: fxp0: device timeout I get these as well. Is it on irq9 by any chance, along with acpi0 ? No. It's on irc 11 device 8 (as dmesg states). All irqs in bios are set to 11 (factory default). ACPI is disabled. Ok, well that's a useful data point (for me anyway) since it means there's a problem with fxp losing interrupts that's not related to some other ACPI problems I've experienced, suggesting that it's therefore a fxp driver problem. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Wed, 2003-06-04 at 00:03, Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Paul Richards wr ites: On Tue, 2003-06-03 at 22:36, Poul-Henning Kamp wrote: I thought the point in KOBJ was that it was extensible so you could KLD load stuff which added more methods ? Not exactly. It allows for dynamic binding of methods that implement a specified interface. It gives you 2 things mainly: The possible methods available in an interface are fixed, they're defined in the .m files. Then I don't see the justification for the hashing caching when it can be resolved at compile time... kobj basically provides us with some OO type functionality. The linkage can't be determined at compile time since you don't know what methods an object is going to implement or what actual function implements a particular method until the object is instantiated, it's sort of late binding. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Wed, Jun 04, 2003 at 10:01:07AM +0100, Doug Rabson wrote: On Tuesday 03 June 2003 12:00 am, Paul Richards wrote: On Tue, 2003-06-03 at 23:09, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Paul Richards [EMAIL PROTECTED] writes: : The possible methods available in an interface are fixed, they're : defined in the .m files. No it isn't. One can add additional interfaces at any time without breaking binary compatibility, so you don't know, a priori, the number of methods in a given interface when you build a client of that interface. I don't think that's true. The interface is defined in the .m file and it's created using makeobjs. You can't do that at runtime because it creates the kobj_desc struct that uniquely represents the existence of that method globally for the whole of the kobj subsystem. The set of all kobj_desc defines all possible methods that can be implemented by an object, and yes you can extend that interface later and previously built modules will still work because they only implement a subset of those possible methods, but the set of all methods that can exist is determined at compile time from the interface definitions A class is then defined which specifies the set of methods that objects instantiated into that class *can* implement (but are note required to). The set of methods in a class is also fixed, since it's basically the method table plus some class fields and the method table is created at compile time. Again though, if you later extend the class older compiled modules will still work because if the method doesn't exist in that older module the default from the kobj_desc will be used or the kobj_error_method will be called (which is safe). So yes you can extend the interface and the class and keep backwards compatibility but that all happens at compile time, and therefore at runtime when the object is instantiated you know the maximum number of methods that the object can possibly call. It is true, I'm afraid. A class is can implement any number of interfaces so knowing the compile-time size of an interface doesn't help you reserve space for a class. The system is intended to have a robust ABI so that e.g. a class implementing an interface can be used safely by a kernel in which new methods have been added to that interface. I though Warner meant adding interfaces at runtime, but he meant build time (I misread that). My comments about interfaces where therefore in rebuttal of the possibility to extend them at runtime. So yes, what Warner asserted is true but as I did also point out above and you've confirmed below, it's the class that determines the methods for an object and that is fixed when the object is instantiated into that class. The class is a concrete definition of a set of methods (at this level, the interfaces are mostly ignored). Objects instantiated in that class use exactly that set of methods, no more, no less. There is no choice involved. You appear to be confusing class and interface here. Given that we agree that the method table is fixed at object instantiation time, why isn't allocating a fixed array appropriate? -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: VFS: C99 sparse format for struct vfsops
On Wed, Jun 04, 2003 at 12:09:00PM +0100, Doug Rabson wrote: On Mon, 2003-06-02 at 21:04, Paul Richards wrote: On Tue, 2003-06-03 at 18:19, M. Warner Losh wrote: Notice how thread 1's _m gets set based on the results of the kobj lookup, and we have a race, even if thread1 and thread2 took out their driver instance locks. That means we have to lock the dispatch table before every method is looked up and hold it until the method returns (the alternative would be to free it inside the method once it had been called but that'd be a right mess). Don't even think about trying to put a mutex into the kobj dispatch I wasn't, I was just pointing out what would be necessary if the cacheing problem wasn't resolved :-) -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: cvs commit: src/sys/contrib/dev/acpica - Imported sources
On Wed, May 28, 2003 at 03:39:44PM -0700, Nate Lawson wrote: On Wed, 28 May 2003, Larry Rosenman wrote: --On Wednesday, May 28, 2003 03:59:24 -0500 Larry Rosenman Ok, with today's sources, I still get a page not present panic for address (0x7) on transistion to battery. as a followup, with this code, I no longer get the panic at ACPI shutdown, just some messages about references. As a further followup, today's update of the ACPI sources get's rid of the ACPI errors in the dmesg, but I STILL have the panic at 0x7 on transition to battery. There's something wrong with your DSDT and/or the Intel acpica interpreter such that reference counting on ns objects is incorrect. Is this going to be released like this? I'm doing my best as a volunteer. If none of us finds the problem before release, the answer is yes. In that case, you should disable acpi or use apm. Or feel free to hunt down the problem yourself. Just a note, but using apm is rarely a solution to using acpi because for a lot of people (me included) the box doesn't work at all without acpi support, interrupt routing and/or timecounter support is screwed to the point that FreeBSD doesn't work. ACPI isn't a power management tool, that's just one small aspect of it. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: policy on GPL'd drivers?
On Wed, May 28, 2003 at 12:04:17PM +0200, Marcin Dalecki wrote: Harti Brandt wrote: MDNO no and again no. This would repeat the same design mistake MDthat is already in Linux. On API level you DO NOT WANT versioning. MDWhat you really want is: type signature cheking. Like for example MDdone through C++ symbol mangling rules. If you can't do it like that MDthen better leave it off as it is. Versioning in itself Type signature checking doesn't help you if the semantics of an API change without type changes. APIs should be semantically and syntactically stable in -STABLE. In -CURRENT they are expected to change. Managing a 3rd party driver for current is a nightmare, but may no be necessary once we have a -STABLE based on FreeBSD5. Sure sure. Perhaps I wasn't clear enough. Versioning doesn't help you *anything* at all, but it is introducing new problems instead. Tons of them in fact, if one looks at the Linux pain in this area. I suggested type signature cheking in C++ style just to show how fundamentally flawed the idea of version cheking is, but I can perfectly life with the situation as it is. Symbol versioning is good to implement multi-flavoured single binary I don't think anyone is talking about symbol versioning. The issue is stamping the API at a particular point in time that shows it behaves in a specified guaranteed way. The module system has all the hooks to deal with versioning. What's missing is not the mechanism but the knowledge, i.e. we don't know that module X won't work with kernel Y becuase we don't know what APIs module X may invoke and whether kernel Y supports them. Some of the suggestions, such as using an MD5 hash, will prevent panics by refusing to load kernels that haven't been rebuilt since the kernel was, but it's a bit of a sledgehammer, since you don't want to have to rebuild all your modules every time you rebuild the kernel and in some cases (third party modules) you won't be able to, so such a mechanism won't work in practice. The only true solution to this is to version the APIs in the kernel and use the module versioning hooks to not load modules if the version isn't the right one. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: policy on GPL'd drivers?
On Wed, 2003-05-28 at 17:48, dave wrote: I don't think anyone is talking about symbol versioning. The issue is stamping the API at a particular point in time that shows it behaves in a specified guaranteed way. The module system has all the hooks to deal with versioning. What's missing is not the mechanism but the knowledge, i.e. we don't know that module X won't work with kernel Y becuase we don't know what APIs module X may invoke and whether kernel Y supports them. snip The only true solution to this is to version the APIs in the kernel and use the module versioning hooks to not load modules if the version isn't the right one. Will this require *any* new infrastructure to implement properly? Or is it simply a matter of maintaining API metadata regarding versions. I think it'd just be a question of maintaining the metadata. There may be a little code missing but I don't think it would take a lot of work to complete the versioning mechanism. Creating all the metadata to actually define and version the APIs is another issue entirely though. Each module can maintain dependency data, stating which versions of other modules it can work with. The kernel would need to create virtual modules that held the faked module version for that API component. That wouldn't be very hard, just a linker set to record the API version in a module version structure. Ideally, whenever possible each API component should be grouped into a module anyway, but when that isn't possible just defining a faked module version should work. The kernel wouldn't then load a module if the kernel didn't support the APIs that the module depended upon. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: gbde Performance - 35Mb/s vs 5.2 MB/s
On Wed, May 28, 2003 at 10:11:19AM +0200, Poul-Henning Kamp wrote: In message [EMAIL PROTECTED], Guido van Rooij writes: On Mon, May 26, 2003 at 02:25:08PM +0200, Heiko Schaefer wrote: Poul gave me the following tip on this list in a mail on Tue, 29 Apr 2003: Remember to set the sectorsize in gbde (gbde init -i) to the fragment size of your filesystem (typically 2048 for ufs), this is critical for performance. If this is so important, why isn't this the default? Because I have no way of knowing that peple will in fact be using UFS/FFS on the GBDE encrypted partition, and even if they do, I have no way of knowing the fragment size they will use. I considered making the sectorsize a mandatory argument, but decided against it. Maybe I was wrong. It might make more sense to mandate an argument that indicates what filesystem they intend to put on top of it and then base the defaults from that. It would be more user friendly. -- Tis a wise thing to know what is wanted, wiser still to know when it has been achieved and wisest of all to know when it is unachievable for then striving is folly. [Magician] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ACPI thermal panics ThinkPad 600X
Don't cross post current and developers. The developers charter says it's for internal management i.e. how we manage the project and not for discussing code issues. It's badly named, we should have called it something like members or admin. On Sat, 2003-02-15 at 02:49, Ruslan Ermilov wrote: ACPI thermal panics my ThinkPad 600X, is anyone interested in a crash dump analysis? Cheers, -- Paul Richards [EMAIL PROTECTED] FreeBSD Services Ltd To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: 5.0 Freezes under high load with SMP.
On Tue, Jan 14, 2003 at 05:19:36PM -0800, wade wrote: I tried this, but when the box froze, the keyboard was completely unresponsive, no numlock, nothing. No interrupts getting serviced. I don't know if it's the same bug or not but writing a CD with burncd lock my box up solid as well. Strangely, I can do an erase pass but as soon as I start a data pass it locks up. I think there's a general problem somewhere though because I've been experiencing lockups when running X and doing heavy disk activity as well. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Current issues
I'm trying to thrash 5.0 a bit but I've run into some rather more basic issues right off. 1) I think I'm suffering from the 1GB memory hang problem. I'm definitely getting the hangs :-) I'll leave this for now since I'll do some more testing tonight and it's already been raised by someone else anyway, so this is just a me too. 2) The IPFW2 syntax is not compatible with IPFW. The latter accepts rules in the form of 1 add ... whereas IPFW2 doesn't understand that format and requires add 1 .. 3) The compiler won't build some C++ packages, evolution being the particular one bothering me at the moment. Is this a known issues that the 5.0 compiler won't build all packages? I will report other problems as I find them :-) Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Current issues
On Tue, Dec 10, 2002 at 04:02:14PM -0500, Joe Marcus Clarke wrote: On Tue, 2002-12-10 at 15:48, Kris Kennaway wrote: On Tue, Dec 10, 2002 at 02:01:08PM +, Paul Richards wrote: 3) The compiler won't build some C++ packages, evolution being the particular one bothering me at the moment. Is this a known issues that the 5.0 compiler won't build all packages? Yes. Evo isn't a C++ application. I haven't seen any build logs for it, either. I'm doing a build on my i386 machine now just to be sure. You're right, it was one of the dependencies, maybe aspell. Anyway, I've got it built now, it was old include files that was the problem since this had originally been a 4.x machine. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Removing perl in make world
On Sat, 2002-07-06 at 03:46, Garance A Drosihn wrote: At 3:05 AM +0100 7/6/02, Paul Richards wrote: Let's start with a premise: No-one running current is using it for anything other than developing FreeBSD. This is assumption is too limiting. It shouldn't be. You're trying to defend a position that the project doesn't support. The -current branch is for FreeBSD developers, or volunteers brave enough to test the -current branch. As such, don't be surprised if it wipes your hard disk clean. People running -current are doing it to test the latest builds. What they *do* to test it is their business. Ie, if I want to install some port in /usr instead of /usr/local, that is something I should be able to do. You're free to do that, but you're not really testing -current then, since you're doing things outside the scope of what -current is designed to support. If you're doing stuff to stress it in some way that's particular to you're testing regime then fine, but that's something you'd need to be aware of and it's not that different to installing a personal, more up to date version of gcc in the base, if you do that you need to be aware that the standard current build system will overwrite your changes. A 'sysclean' target would be the same in my mind. If you're within spec of what -current supports then running that target shouldn't hose you. If you're outside spec then you need to take your own precautions. I'm not suggesting that 'sysclean' be a default part of installworld anymore. I suggested that removing perl should be, but the debate has moved on from that position and I think the sensible thing now would be to create a 'sysclean' target, that could be run as desired that was more comprehensive than just removing perl or other deprecated apps. A more aggressive cleaning target I think solves more problems more effectively and if it's an optional target I can't see any problems. Probably a make.conf knob, like the update target uses, so people can have it on or off as part of the standard build as they desire. I think having it as a make target is a good thing, since it would allow us to easily document that people who truly want to test current should run that target after a build to ensure they're not running old stuff. It keeps the code tied into the build process too. A separate optional script wouldn't be as well integrated into our build setup. -- Paul Richards | FreeBSD Services Ltd| Order 4.6 on DVD now. http://www.freebsd-services.com | To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Removing perl in make world
On Sat, 2002-07-06 at 13:29, Ruslan Ermilov wrote: On Fri, Jul 05, 2002 at 10:45:41AM +0100, Paul Richards wrote: I think we should add a target to make world that checks for the existence of an old base install of Perl and removes it if it exists. As a general principle, if we do things like remove code during -current development then make world needs to cater for that change. The idea of make world is that what you get at the end of it is a pristine install of a snapshot of FreeBSD from the current branch. The bootstrap target was originally added for this purpose, to deal with issues during releases i.e. to bootstrap from one version of current to another. It's got misunderstood along the way to mean bootstrapping tools needed early in the build process. I'd like to resurrect it's original meaning and add code to clean out old versions of Perl. I strongly object to this unless we clearly define the procedure of deleting the old stuff like libraries, headers, programs, etc. I routinely do the cleanup by installing the world into a separate DESTDIR, and comparing it with what is available under /. I don't see a major problem with this except maybe a space. Another approarch is to compare the modification times with the current date; this works almost ok for all things except headers and (now) static libraries plus some miscellaneous stuff that is getting installed with -C. You need to catch up with the rest of this thread, since I think we're moving towards something more acceptable. However, in the particular case of Perl, I think it does highlight a slightly separate issue in that our build process has no concept of deprecating parts of the OS. The 'sysclean' target will accidentally deal with the Perl issue, but I don't think it's wrong to have a target remove a module that has been removed from source since that's what installing the system should really do if the system has opted to deprecate something. -- Paul Richards | FreeBSD Services Ltd| Order 4.6 on DVD now. http://www.freebsd-services.com | To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Removing perl in make world
On Fri, 2002-07-05 at 16:24, Sheldon Hearn wrote: On (2002/07/05 05:22), Terry Lambert wrote: This would not fit in with the rest of the world target, which doesn't clean out stale headers, stale libraries or stale binaries. Special-casing certain things will surprise people. Headers and libraries arguably should be removed, so as to avoid errors; not ports headers or libraries -- which aren't in the installation target paths in the first place -- but things like deprecated system headers, etc.. You and Paul are both pretty out there if you think -current users will graciously accept a new world order in which ports linked dymanically against system libraries won't work between a system upgrade and the next port reinstall. I'm reluctant to suggest something that scares away -current developers, since I don't think enough people are really working on -current these days. However, if you are developing FreeBSD then you do need to make sure you're running -current and not -cruft. The old mantra of -current is for developers still holds and if the development environment becomes ineffective because people are relying on current as a work environment then maybe we should tighten the development environment so it's more suited to those people who are actually working on FreeBSD itself. On my -current box I usually rebuild ports quite regularly anyway because ports get updated pretty frequently. The only real issue with backwards compatibility is the libraries, there's nothing else that should severely impact existing installations. Libraries can easily be dealt with by just moving them to a compat dir. If you want to clean out crap left behind by `make world', just do this: make world rm -r /usr/include# Make world really should overwrite make installincludes # header files! find /bin /sbin /usr/bin /usr/sbin /usr/lib /usr/libexec /usr/share \ -type f -mtime +1 -delete That is precisely what I do periodically (along those lines anyway), and it is often the case that when I do so I find that I've been running -cruft and not -current at all. -- Paul Richards| FreeBSD Services Ltd | Order 4.6 on DVD today! http://www.freebsd-services.com | To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Removing perl in make world
On Sat, 2002-07-06 at 01:07, Garance A Drosihn wrote: At 3:33 PM -0700 7/5/02, Terry Lambert wrote: So, to summarize: Let me summarize my own position. There are a number of files which installworld does install. After an installworld is done, there may be a number of files on a person's hard disk which were not put there by the most recent installworld. For each of those files, the issue is one of intent. 1) Is it there because the administrator explicitly wanted it to be there, for explicit reasons that may be perfectly valid even while testing the latest snapshot of current? 2) or is it left-over cruft from some previous install, and which is only getting in the way of proper testing? If you keep it that simple, instead of writing 200-line summaries of what is going on (and the possible motivations of everyone), then the solution is also simple. The above is just a slight variation from what happens with /etc config files during a new installworld. We should not have anything which automatically blows away those files. We should have an unmergemaster script, which will find those files, and **ASKS THE DEVELOPER** what they want to do for each of those files (or maybe for each set of files). No automatic process is going to be 100% right 100% of the time. I think a -current system is something that should be assumed to be a semi-known environment though. Let's start with a premise: No-one running current is using it for anything other than developing FreeBSD. Given that premise, then there shouldn't be anything in /usr outside of /usr/local, that wasn't put there by make world. Likewise the same should be true of /sbin and /bin. Therefore running, find $listofdirs -newermt $date -delete should be perfectly OK since it's only going to clear out old files that are no longer part of FreeBSD (where $listofdirs is directories that should not be touched other than by make world, and $date is the date of the last install). The only tweak that is necessary is in the case of /usr/lib, where files should be moved to a compat dir and not deleted. I do this periodically on my dev box and it does show up issues. I think it's something we should build into our infrastructure as a step towards a better known environment for testing -current. -- Paul Richards | FreeBSD Services Ltd| Order 4.6 on DVD now. http://www.freebsd-services.com | To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
cdefs and XFree86
The recent changes to /usr/include/sys/cdefs.h have broken the build of XFree86-Server. The problem is with the _XOPEN_SOURCE macro. At line cdefs.h it's checked i.e. #if _XOPEN_SOURCE = 600 but in XFree86 it's defined as #define _XOPEN_SOURCE Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: HEADSUP ATA support for newer SiS chipsets added
--On Monday, December 03, 2001 11:10:47 +0100 SXren Schmidt [EMAIL PROTECTED] wrote: It seems Miklos Niedermayer wrote: I think they are idle (looking at vmstat -i), but i can't be sure. However i have 2 machines here with VIA 82C596 chipset... atapci0: VIA 82C596 ATA66 controller port 0xd800-0xd80f at device 4.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ad0: 28629MB QUANTUM FIREBALLlct20 30 [58168/16/63] at ata0-master UDMA66 524288 bytes transferred in 0.025247 secs (20766367 bytes/sec) It's idle (the LED isn't blinking after/before dd and vmstat -i doesn't show any ata0 activity). Even my Intel PIIX4 ATA33 controller with the same disk performs better (a little)... Hmm, yes that looks somewhat on the low side... Well, two things, the older VIA chips are not the best performers, but I still think it should be better than that, I'll run some tests here, I might have messed up something... Are we talking -current or -stable here ? I'm getting even worse performance. root@lobster# for n in 1 2 3 4 5 do dd if=/dev/ad0 of=/dev/null bs=512k count=1 done 1+0 records in 1+0 records out 524288 bytes transferred in 0.056385 secs (9298353 bytes/sec) 1+0 records in 1+0 records out 524288 bytes transferred in 0.062027 secs (8452580 bytes/sec) 1+0 records in 1+0 records out 524288 bytes transferred in 0.056339 secs (9305947 bytes/sec) 1+0 records in 1+0 records out 524288 bytes transferred in 0.056325 secs (9308271 bytes/sec) 1+0 records in 1+0 records out 524288 bytes transferred in 0.056482 secs (9282398 bytes/sec) root@lobster# atapci0: Intel ICH2 ATA100 controller port 0xb800-0xb80f at device 31.1 on pci0 ad0: 58644MB IBM-DTLA-307060 [119150/16/63] at ata0-master tagged UDMA100 Paul Richards FreeBSD Services Ltd http://www.freebsd-services.com To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Undefined symbol __stderrp
--On Friday, September 28, 2001 19:04:39 -0700 Matthew Jacob [EMAIL PROTECTED] wrote: So, you're gonna be rude and drive away a user because they missed one line? Well, -current is supposed to be for developers and if you don't know C you're probably not a developer. We certainly want to encourage more developers to run -current but we don't really want users to be running it. Paul Richards FreeBSD Services Ltd http://www.freebsd-services.com To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: /usr/games/wtf
--On Tuesday, August 21, 2001 14:18:19 +0700 Max Khon [EMAIL PROTECTED] wrote: hi, there! I would like to add /usr/games/wtf from NetBSD to base system. Any opinions/objections? wtf is it? NAME wtf - translates acronyms for you SYNOPSIS wtf [is] acronym ... husky:~$wtf is pola POLA: principle of least astonishment husky:~$ I can't see any benefits to having this in the base system. Make it a port instead. Paul Richards FreeBSD Services Ltd To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: /usr/games/wtf
--On Tuesday, August 21, 2001 11:37:47 -0400 [EMAIL PROTECTED] wrote: I can't see any benefits to having this in the base system. Make it a port instead. Oh and /usr/games/wargames is such a huge benefit? By that logic all of /usr/games belong as ports. Which I wont argue with at all. Neither will I :-) Paul Richards FreeBSD Services Ltd To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
su, PAM and zsh
There's a strange interaction between su, pam znd zsh. If you su to an account that has zsh as its shell and then hit ctrl-c it will kill the shell that you invoked su from. If you recompile su with -DNOPAM then the problems go away and this doesn't seem to happen with any other shells either. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Lots of interrupts!
My dev box seems to be a bit sick and has been for a day or two. If anything disk intensive is taking place, and it seems to be particularly when gzipping/unzipping files, say when building mozilla during the extract step, things start to crawl and I'm seeing hundreds (500-1200) of interrupts a second for ata0 in vmstat. A network transfer at the same time, just scp a file across my 100M LAN, crawls at around 20k/s with 50-100 interrupts/s. Is anyone else seeing anything like this? Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Fw: Stop annoying message of lnc
Ian Dowse wrote: In message [EMAIL PROTECTED], Mike Smith writes: I don't quite understand Paul's reasoning, though; it's not actually useful to unload/reload parts of a device's bus attachment without unloading/reloading all the downstream parts of the driver. What do you mean by the downstream parts of the driver? I think it makes sense to be able to load/unload the bus specific parts of the driver independently. If you've only got a PCI device installed there's no reason to keep the ISA bus code in the kernel. We're not there yet, but I can see a time where we would want to load/unload the probe functions as we do hardware probing and then only pull in the rest of the driver when we associate that driver to the device. I think the fix should probably be committed and the driver turned into a single monolithic module. Yes, Paul essentially agreed to my doing this as an interim measure until ifconfig is "fixed" to use the module file name rather than the module name when loading drivers. I'll commit the change in a few hours after I have tested that it works. Thanks. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Ethernet entropy harvesting seriously pessimizes performance
"Matthew N. Dodd" wrote: On Mon, 12 Mar 2001, Mark Murray wrote: Lots of security minded people what _all_ the interrupt entropy they can get, and this method gives them that while allowing others to throttle the harvester back. Lots of -CURRENT users want to be able to use their systems to write code without tripping over /dev/random and friends. I hear lots of people objecting to this code and alot of handwaving in response. Choose reasonable defaults already. The -CURRENT cvs tree isn't the proper venue for doing crypto research. Well, I dunno about that. It dovetails into the thread in developers about getting people to use FreeBSD for research and to my mind I think -current probably is a legitimate place for research. As long as the basic -current doctrine of not commiting totally non-functional code is adhered to there's no reason why experimental code can't be tried out in -current. If you don't like the problems that research cause you then -current isn't what you should be running -- it's an old mantra that isn't repeated enough these days. Of course, I'd much prefer it if -current wasn't totally hosed as much as it has been recently but random hasn't caused half the turmoil that some other changes have so it's unfair to pick on it as a major problem. I think Peter gets the award for causing most downtime in -current recently, which is quite a feat given the SMP work taking place :-) Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Patch for FILE problems (was Re: -CURRENT is bad for me...)
David O'Brien wrote: We only bumped due to interface changes in the .MAJOR.MINOR days. The difference is *adding* an interface today does in cause a bump. In the .MAJOR.MINOR days it would require a bump the MINOR number. In both days, an incompatible change in an existing interface require(s)(ed) a MAJOR bump. Yes, that was true. The way we used to do it didn't address all the problems I'm alluding to either but I felt we had more versioning before than we do now. Regardless of that the issues are important enough that I think we should discuss them. There are in my mind two issues: 1) Being able to have the system continue working when a significant interface change occurs. 2) Being able to identify a specific version of a library on a system in order to determine whether it's a rogue version for a particular application. The former I accept works fine now as long as you can take the pain of current. An asthetic requirement to have a nice library number is counter-productive though when it screws the development team so completely that the system is unusable for a week. While developers should accept that they must occasionally suffer considerable pain when running -current it's foolish to not consider alternative methods of working when we run into a problem that causes the project to come to a grinding halt. Most of us don't have rooms full of development boxes, we have all our day to day applications on our development box and having to rebuild it completely is something we accept we may have to do on occasion but it's something we should try and avoid having to happen because it's so wasteful of everyone's time and energy. This library version problem is a non-problem from a technological perspective, it's only a problem from a policy perspective and it's a policy that's based entirely on asthetic requirements. I don't want to see library versions get into the hundreds (unless we adopt that as a convention i.e. 5xx) because we're bumping them all the time but at the same time, if one is necessary then it's necessary, even if it's current. Commercial vendors will skip version numbers in their public releases if their internal development required more than one bump. I don't think the attempt to make the major number bump once per release is a sensible goal. If we have to bump a major number on the stable branch (god forbid, but it may happen one day) then we'll have to deviate from that policy because it'll clash with the version number of -current and therefore I think the policy is flawed because it doesn't consider all the possible scenarios we might have to deal with. The second issue is probably not related to bumping the library version, since I accept your point that we didn't bump major or minor numbers for every change to the library. I think some way of identifying a build of a library would be a good thing though and perhaps we should discuss adding a "properties" field to libraries so we can identify which specific version of a library is installed. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: pkg_update
Leif Neland wrote: On Wed, Feb 07, 2001 at 08:23:35PM -0600, Michael C . Wu wrote: On Thu, Feb 08, 2001 at 01:56:11AM +0100, Leif Neland scribbled: | It seems pkg_update is only usable when installing from packages, not from | ports. Because it is a package update system. If you want to update from the ports, use 'pkg_version -c |sh' Never, ever, *ever* do this. Just installing a new version of a port seems to work. I've actually run into some problems that are on my list of things to look at. The problem is that 'make install' in a port doesn't check dependencies properly, whereas pkg_install does. The way I currently use pkg_update (not ideal I've plans to improve it) is to build the new version and then do 'make package'. This installs the new version, over the top of any previous versions but that doesn't matter because pkg_update cleans all that up. Once 'make package' is finished you can run pkg_update using the newly created package. The problem is that the 'make install' will happily succeed but the pkg_install run via pkg_update will fail because the required dependencies for the package are missing. Of course, by the time the pkg_install fails the pkg_delete has uninstalled all your older versions. This isn't fatal because pkg_update prints out the contents of the +REQUIRES file when it fails so you can do a 'make install' to install the new port version and then copy the dependencies into /var/db/pkg/'pkgname'/+REQUIRES. I know that's not ideal but it prevents you getting totally screwed if the pkg_install fails due to dependencies. Couldn't it be made possible to use just the update-of-dependencies part of pkg_update without doing the pkg_delete/pkg_install bit? Perhaps I'll try... I guess it might be useful to just print out the collated dependencies, just like it does if the pkg_install step fails. It would be pretty trivial to add a switch to add that functionality. That would allow you to get the contents of what the new +REQUIRES file would be like if all the pkg* steps were run but to not actually run them so you can do them by hand. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: pkg_update
Will Andrews wrote: On Wed, Feb 14, 2001 at 03:10:14PM +, Paul Richards wrote: The problem is that 'make install' in a port doesn't check dependencies properly, whereas pkg_install does. Uh, actually, 'make install' does a better job. pkg_install has no clue about substitute dependencies. Ahh, that makes sense. That's what's happening then, 'make install' is succeeding because it's using substitute dependencies but pkg_install fails because the specific packages it expects to see are missing. I misinterpreted the fact that pkg_install was complaining to mean it was being more thorough. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Patch for FILE problems (was Re: -CURRENT is bad for me...)
David O'Brien wrote: On Tue, Feb 13, 2001 at 04:24:00PM +, Paul RichardsF wrote: When we dropped minor numbers I had a worry that we'd run into one of Windows' greatest problems and we have. Applications that are developed and tested to work with a particular library might not work with a different version, How is that??? Because applications expect a stable interface. If an application is developed and tested against a particular behaviour of a library function and that function's behaviour is changed it could and sometimes does break the application. More often than not that sort of failure is a side effect of *fixing* something that was previously broken. It is beter under ELF than a.out in that ld.so isn't making a guess as to which shared libs were compatible and which weren't. The ELF ld.so does not look for shared lib libxyz.so.2, find libxyz.so.3 and decide maybe they are close enough and use it instead. The a.out ld.so would use libxyz.so.2.2 when the binary was compiled and tested with libxyz.so.2.1. Instead what we have now is libxyz.so.3 and libxyz.so.3 which are different from each other. When you've got an application that just broken you've got no way of finding out which version of the library was the working one, or any way of linking against it because it cannot co-exist with the newer version because they've got the same name. This is precisely the problem we've got now. Maybe I didn't make it clear in my last email, it's not the old a.out way of picking the library to link with that was better, it was the finer grained versioning. we're suffering a worst case scenario of this problem now but even "fixes" in new versions can cause applications to break and Don't confuse development (which in years past would have never made it out of the "company's" development machines, with deployed releases. This applies to released code as much as in-development code. When you login to a strange machine and you're trying to diagnose a problem there's no way to know whether the libc they've got installed is of version X or version Y because there's nothing to tell you what sources libc.so.Z was built from, it could be the .Z version with the X fixes or the Y fixes. To be honest, DLLs are better than our scheme from that perspective. While you might screw a load of applications by upgrading a DLL with the same name you can at least look at the version number in the properties to find out which version of that DLL it is. There's no way of looking at a libc.so.5 and say which version of libc.so.5 it is. Although `ident` does provide a slightly cumbersome way of getting some information to help with that when you really need it. we've already seen this many times in this iteration of -current. *Way*, way too many people are using -CURRENT that have no business doing so. I agree with that, I've always been an advocate of raising the barrier of entry to using -current. I think we need some form of version control on libraries so that applications know whether they're linking with the version they're designed for and to be able to keep multiple versions around in the system so all applications continue to work. We have that today and it works very well [in our released product]. I understand the reasoning that Elf doesn't need minor numbers but they served an useful purpose in maintaining application compatibility that we now lack. NO! Please review the rules ld.so in both ELF and a.out varieties uses in finding a desired shared lib. You've missed the point I was trying to make. Our reluctance to bump what we perceive to be a major number is hampering our ability to differentiate between different versions. It has nothing to do with the difference between a.out and Elf library selection, it's a project policy problem. We could just as easily use a minor numbering scheme with Elf to indicate that a version change has occured but not an interface change. To some extent a three figure version number does something along those lines. If we bumped the version number with a bit more abandon this problem wouldn't even be a problem. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: DPT SmartRAID V, VI, Adaptec SCSI RAID driver committed
Mike Smith wrote: I'd like to hear a few more success stories first (only one so far) from people using the kit to add the driver to their 4.x systems. With all the breakage in -current's PCI support at the moment, I don't expect to hear too many people there reporting on it just yet. Maybe I missed it... Where is the kit? I would be happy to install and stress test a machine or two with SMP and without, just didn't realize there was a kit ;) must have been when I was reading my e-mail at like 3 am or something http://people.freebsd.org/~msmith/RAID/index.html#dpt -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] That's a good quote, maybe it should go in fortune. Paul Richards FreeBSD Services Ltd To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make buildworld br0ken in libutil
Mark Murray wrote: Why does crypt need to be in libc? Not even a significant fraction of applications need crypt? Goes for very many libc components. Quite a lot of userland needs libcrypt (not much as a proportion, but a non-insignificant number). This runs counter to my gut instinct of development which is to modularise code. Modularisation is accepted as a goal in all other areas of the tree it doesn't make sense to me why that thinking is being put to one side when it comes to the libraries. Maybe this should move to arch because I guess I'd like to see a actual design discussion as to why the current thinking is to collapse libraries into libc rather than to actually go the other way and modularise the code. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Why no CDR ioctls for SCSI cds?
"Kenneth D. Merry" wrote: On Tue, Aug 22, 2000 at 20:43:15 -0400, Laurence Berland wrote: On a vaguely related topic, after much searching I can't seem to see one way or the other if we can do a complete bit-by-bit copy of a cd with either cdrecord or burncd, though it's possible I'm looking in the wrong place. I think cdrecord can burn CDs in disk-at-once mode, and I think cdrdao (in ports/audio) can do it as well. As far as getting an image, you can use dd to dump off an image of a CD if it is a standard ISO9660 CD. (I've used that method to clone CDs before.) That didn't seem to work when I tried it a couple of days ago. Got a "device not configured" error. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Why no CDR ioctls for SCSI cds?
Alexander Leidinger wrote: On 23 Aug, Paul Richards wrote: On a vaguely related topic, after much searching I can't seem to see one way or the other if we can do a complete bit-by-bit copy of a cd with either cdrecord or burncd, though it's possible I'm looking in the wrong place. I think cdrecord can burn CDs in disk-at-once mode, and I think cdrdao (in ports/audio) can do it as well. As far as getting an image, you can use dd to dump off an image of a CD if it is a standard ISO9660 CD. (I've used that method to clone CDs before.) That didn't seem to work when I tried it a couple of days ago. Got a "device not configured" error. You've done something wrong, it works here. Yep, just tried it again and you're right I was doing something wrong. I was trying to dd from the blank in the CD-RW rather than the master in the CD (blush). BTW: If I need a copy of a data cd I use "cdrecord -v speed=4 dev=0,1,0 -eject -isosize /dev/cd1c" (cd1 is my CD-ROM, cd0 (dev=0,1,0) is my CD burner). That's what I used in the end. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make buildworld br0ken in libutil
Jeroen Ruigrok van der Werven wrote: -On [2822 17:55], Ollivier Robert ([EMAIL PROTECTED]) wrote: According to Jeroen Ruigrok van der Werven: Alternatively the sentiment just rose why we couldn't just collapse the crypt/hash functions of libcrypt into libc. It would make sense. It would make even make more sense to convince the other BSD to do the same (haven't checked recently what they do) and do the merge. I very much agree. Would it be sensible for the regular cypherpunks to discuss this with the NetBSD and OpenBSD brothers? Otherwise I would be willing to open this discussion on the appropriate lists. Is there any current policy on what libc is? It certainly isn't "libc" as required by C and hasn't been for almost ever but there needs to be some rational to its existence otherwise why not fold everything into libc and not bother with any other libraries! A growing libc makes static binaries grow and makes it more difficult to strip out unneeded functionality from a minimalist system install. I'd been inclined to try and move things the other way and strip stuff out of libc into separate libraries but that's obviously not in vogue at the moment. Why does crypt need to be in libc? Not even a significant fraction of applications need crypt? Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Ugly, slow shutdown
David Greenman wrote: Can you give a reason why we'll have to now start coding defensively because our arguments to tsleep() are just "advisory" now? It is not something we "suddenly have to do" it's been The Right Way even since I first sharpened my teeth on unix kernels many years ago. Uh, Poul, I think you're full of it. The previous behavior of tsleep where you can make assumptions about who wakes you and under what conditions is a long and well established idiom. We (the kernel developers of BSD) have always coded to the established kernel programming interface and most of us consider it bad form to check for conditions which can't happen because the kernel API doesn't allow it. For example, we don't check for a NULL return from malloc in the case of !NO_WAIT, because we knew that the code would never do that. There are many other examples of similar assumptions in the kernel. We write the code to be efficient and only check for bogus conditions that might happen. The only exception to this is programatic ASSERTs that do internal consistency checks, but the purpose of those is quite different. In the particular case of sleeping though, a woken process does need to check the condition that it slept on because one of the other processes sleeping on that resource may have had a chance to run first and changed some state. So as a general rule, you shouldn't assume that everything is fine when you return from being asleep because it might not be. I agree with your sentiment about defensive coding in the kernel though. Defensive coding should only be used on the boundary of your interface, i.e. if you're getting non-deterministic data from somewhere, most commonly user behaviour. Everything inside the kernel implementation should be deterministic and therefore there shouldn't be any need for defensive programming. Any cause of errors that defensive programming catches in the kernel can only be caused by bugs and therefore falls into the category of diagnostics. There's actually a lot of defensive programming creeping into the kernel which should really be wrapped in an #ifdef INVARIANTS. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Ugly, slow shutdown
David Greenman wrote: In the particular case of sleeping though, a woken process does need to check the condition that it slept on because one of the other processes sleeping on that resource may have had a chance to run first and changed some state. So as a general rule, you shouldn't assume that everything is fine when you return from being asleep because it might not be. No, that's not true, and there are many examples in the kernel where a bogus wakeup would lead to bad things happening. I recall some code in the advisory locking code, and VM system, that assume that there is only one wakeup event and that the thing causing it assures that certain other things have occured before issuing it. That's just the way it has worked since the dawn of time. I did say "as a general rule". If you know that "by design" nothing else is going to mess with what you're sleeping on before you wake up then you can make tighter optimisations but that's not the general case. There is such a thing as over optimisation though and for the sake of a simple if statement it is probably better to write code that is robust to changes made elsewhere in the system rather than squeeze every inch of performance out of the code, unless there's a real need to optimize in that particular area. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Panic during boot under current
Brian Somers wrote: In article [EMAIL PROTECTED], Archie Cobbs [EMAIL PROTECTED] writes Brian Somers writes: Also (Mark sits beside me at work), is there anyone else out there that actually runs FreeBSD-current under VMWare (irrespective of the host OS) ? This problem has now been traced down to a bug in the lnc driver, where multiple instances are installed. This surfaced as a result of changes made on 16th May. Specifically, the lnc driver declares NLNC softcs. Mark has an isa style ``device lnc0'' in his config and then uses vmware with two configured devices. Actually, PCI cards allocated their own softc dynamically, only ISA cards accessed the softc array. config kernel produces lnc.h which contains ``#define NLNC 1'' and if_lnc_pci.c dives in and starts writing to the second softc. I'll look at fixing this at some point if you don't have time Paul :-I Just got back from the Perl cruise, still catching up with things. This may have broken because Paul changed the softc decl from static to just global (so that the other if_lnc*.c modules can get at it) either that or Mark has been ``just lucky'' so far and the lnc driver has never worked properly with more than one device. This would neatly It was working with more than one device (one PCI and one ISA) the day I left, which was 19 May. I think the changes to the PCI and ISA compatibility code modified something relating to unit number allocation that caused the ISA driver to get a unit number NLNC whereas before the ISA instances would come first and the PCI instances would get unit numbers after the ISA cards, since the PCI part of the driver creates softc structures dynamically that worked fine. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Breaking build world costs $5? (was: Can we please have acurrent that compiles?)
"Brian W. Buchanan" wrote: On Mon, 15 May 2000, Greg Lehey wrote: I see this money scheme as an extension of the "finger pointing" which does nothing to build team spirit. That depends very much on the way it's taken. At the moment, people take the pointy hat voluntarily, not because it's forced on them. It's my expectation that the $5 should be in the the same spirit. Exactly... it's not meant to be a complicated system of fines or to discourage people from developing (at Desktop, we certainly don't want that!), it's just a little bit of good-natured public humiliation to encourage people to think (and maybe test) before they commit. :-) The only problem being that at Desktop you can drop the $5 into the fine box when you leave work but I'd have to walk down the bank and either do a wire transfer or send an international money order. Apart from the hassle involved there's the fact that all told it will cost me in the region of $30-40 to pay the fine! A nice idea but not very practical for the FreeBSD project. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: SMP changes and breaking kld object module compatibility
Bill Fumerola wrote: On Mon, Apr 24, 2000 at 04:46:43AM -0500, Richard Wackerbarth wrote: From the USER's perspective, anything that requires me to as much as reload a module/program that I have already installed "breaks" it. The fact that it is only necessary to recompile it in order to fix it only means that it is easy to fix IF I have the source code. No-one forces you to upgrade you systems. Partial upgrades are something that are nice when they work, but understood when they don't. We don't accept bug reports (typically) when a person hasn't upgraded their world, kernel, and modules. I don't understand why we're accepting preemptive bitching. The trouble is, if you're running a production system then there will be cases where you really must carry out an upgrade, because of a critical bug fix, even though as the admin you really don't want to touch a live system. When you're running a "stable" code base you expect to be able to patch in these bug fixes if they're required with relatively little pain. However, if rebuilding your kernel to incorporate a critical bug fix 3 months down the line means that you're proprietary drivers no longer work then you're stuck between a rock and a hard place. Stable should mean stable and stability means not changing things unless they're critical bug fixes. Development on the stable branches has become far too fast and loose. It *is* affecting the perception of FreeBSD by commercial users who really aren't interested in getting new features on stable branch, in the main if you want new features you wait for the next major release and you go through all the hassles of regression testing your application on the new version. That's not something done lightly or often and breaking the stable branch and preventing the incorporation of genuine fixes, rather than enhancements, is not helping the adoption of FreeBSD by serious commercial users. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: SMP changes and breaking kld object module compatibility
"Brandon D. Valentine" wrote: On Tue, 25 Apr 2000, Daniel C. Sobral wrote: Because if we do not provide a STABLE ABI, we WON'T get third-party (binary only) kernel modules. I'm very divided in this issue. 4.x has just started, and would be seriously impaired if no further improvements to it's SMP get in. On the other hand, if we can't garantee third party vendors a stable ABI, we won't get third party vendors. The number one excuse large third party server vendors use to justify use of NT over Linux on their high end SMP systems is the poor performance of Linux SMP. This is a tremendous opportunity for FreeBSD to take market share. People are seeing the virtues of free, open source operating systems in the server farm and providing top notch SMP support in FreeBSD will help to see that we make further inroads that the Linux guys do. If the BSDi code assists us in improving SMP performance and if the corporate backing helps our PR image, then we could be in for a fun ride. This is the time to start thinking in terms of "What can we do better?" or we're going to lose the battle. Sure, those changes could go into 5.x and be released when RELENG_5 is branched a year from now. But in this business, a year is 6 months too late. Think about that before you object to what appear to be vast improvements in the performance of the RELENG_4 branch while it is just getting off the ground. I totally disagree, my experience is that commercial users upgrade not more than once a year and they expect things to continue working on their chosen branch for that year and to continue to receive bug fix support on that branch (most places upgrade much less often than once a year). "a year is 6 months too late" is true for development branches but it is not the case for production code. Having a happy user base will mean that a FreeBSD 5 with improved SMP will be anticipated and widely adopted when it is released sometime next year (I'd hope). The more we piss off the production users with unnecessary incompatibilities in stable branches the more entrenched they become with their existing releases, regardless of the improvments in later versions, because they don't trust the project to have a professional release strategy. I've seen this happen already and early adopters of 4.0 are going to be really pissed off to find that there is an ABI change in a stable branch. Most commercial users are not developers, and have no interest in anything relating to development. Professional sysadmins are conservative creatures, they expect professional quality code to play by the rules of the industry and those rules are widely accepted as meaning that stable branches do no undergo ABI changes. Such changes are reserved for major upgrades because of the consequences and risks involved. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Stale modules (Re: panic in the morning)
Warner Losh wrote: In message [EMAIL PROTECTED] Alex Zepeda writes: : Perhaps it's time to implement some sort of versioning in the modules to : prevent them from being loaded into the incorrect kernel. In theory that sounds nice, but in -current the kernel ABI changes too quickly for that to be effecitve. The general rule in -current is that the modules must be compiled at the same time as the kernel, or from the same sources. Anything else might work, but is unsupported. I don't know of a good way to serialize (eg assign a serial number to) the sources that would work. Why not just use a UTC timestamp? If your kernel is newer than the module then don't load the module. Maybe make it a sysctl as well so that rapid developer folks can disable it. Even if it was used just as a warning it would remind people to think about the modules. I've lost count of the number of times that I've chased a kernel bug before remembering to rebuild modules. I know a timestamp doesn't guarantess ABI compatibility but it solves the most common problem, which is rebuilding a kernel but forgetting to rebuild the modules. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Stale modules (Re: panic in the morning)
Will Andrews wrote: On Thu, Apr 20, 2000 at 11:56:08AM +0100, Paul Richards wrote: that rapid developer folks can disable it. ITYM "rabid". And I kinda resent that.. ;-) I really did mean rapid, as in those that are installing kernels every 10 mins to test changes. Though, these days there does seem to be a lot of foaming at the mouth :-) Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FreeBSD Build status
"Jordan K. Hubbard" wrote: But does this also check the kernels ? It was my understanding that it only did world/release ? It only does the world/release (and it's the chrooted make release "world build" which is reported on, not the host system's BTW) but could easily add a kernel build just for the benefit of the mailing list folks. It would be trivial to add. As for the lists being tedious and long: I've sorted the content by relevance, and it was my hope that over time they would shrink to zero if we annoyed people enough with them. I think that's too much annoyance, really. I can see most people simply unsubscribing from -current in the face of a mail that long and tedious (and daily) and -current was never created as a place for daemons to talk to people in any case, it was created as a discussion forum. Let's do this, by all means, but let's plase do it elsewhere. Anyone who starts sending daily automated anythings to any of our "top level" mailing lists will classify themselves as a spammer on my and many other people's radar. Not sure I agree with the annoyance issue. It's already got a few people thinking about some of the issues so I think it's useful. If we lose some readers of -current because a report of development problems is too much for them then they probably shouldn't be -current subscribers in the first place. Hackers is supposed to be the mailing list for people generally interested in hacking on FreeBSD, current *should* be for people who want to see long tedious reports on the problems in the current branch. Also, the length of the report should fall dramatically as the problems are addressed and a sudden long report will immediately focus developers attention to the cause. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: RSA library problems
Kris Kennaway wrote: On Thu, 23 Mar 2000, Paul Richards wrote: Because the dlopen() of librsaintl.so fails. Ok, I give up :-) Why would that happen then ? I don't know :-) I stuck a dlerror() in there and the problem is usr/lib/librsaINTL.so: Undefined symbol "BN_mod_exp_mont" Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: FreeBSD random I/O performance issues
Richard Wendland wrote:
I spent a bit of time analysing these results when I first saw them. I
don't think it has anything to do with the cache, it has to do with how
we write out blocks.
One interesting observation is that for non sync, async or noclusterw
mounts ~8750 I/O operations are done, which is 7/8ths of the 10,000
writes. If I change the program to use 16 blocks there are ~9375
I/O operations which is 15/16ths of the 10,000 writes. Guessing,
this is as if writes are forced for all blocks but one.
This is due to a quirk of the clustering algorithm. See below or my
previous email.
With async filesystem mounts very little I/O occurs, and with
noclusterw there are ~10,000 operations matching the number of
writes.
With sync it's ~20,000 operations matching the total of reads
writes. This demonstrates another aspect of the bug, sync behaviour
should cause 10,000 operations; the reads aren't being cached.
This isn't quite true. It's 20,000 *write* operations. I put this down
to the mtime update for each write doubling the number of actual write
operations. No read operations take place, the data *does* come out of
the cache. There's nothing wrong with reading as far as I can tell.
Another aspect of this issue is the effect of changing the seek
blocksize, and write blocksize, by 1 byte each way from 8192, thus
doing block unaligned I/O. In some cases this changes the amount
of I/O recorded by getrusage to zero, and drops elapse time from
half a minute or so to less than 1 second.
Thanks to Paul Richard for noticing this. I've not spent much time
researching this, so can only present my small set of measurements.
To do these tests you have to recompile my test program each time eg
gcc -O4 -DBLOCKSIZE=8191 -DWRITESIZE=8193 seekreadwrite.c
This is because of the fact that if the filesystem block is full it is
written immediately, or rather the clustering code is called
immediately. The rationale is that a full block isn't likely to be
written to again so it might as well be pushed out to disk. Richard's
program deliberately writes full blocks, which is apparently what db
does, so it always forces a write to take place. Given the behaviour of
db it might be more sensible to remove this feature and just mark full
blocks dirty the same as other blocks since it's likely that they will
be written to again shortly if the db record is written to frequently.
The clustering code has a bug in that an old cluster is not pushed out
if the block no is 0 because the code that would do so never gets
reached.
if (lbn == 0)
vp-v_lasta = vp-v_clen = vp-v_cstart = vp-v_lastw = 0;
if (vp-v_clen == 0 || lbn != vp-v_lastw + 1 ||
(bp-b_blkno != vp-v_lasta + btodb(lblocksize))) {
maxclen = vp-v_mount-mnt_iosize_max / lblocksize - 1;
if (vp-v_clen != 0) {
/*
* Next block is not sequential.
*
* If we are not writing at end of file, the process
* seeked to another point in the file since its last
* write, or we have reached our maximum cluster size,
* then push the previous cluster. Otherwise try
* reallocating to make it sequential.
*/
In Richard's program the next block is never sequential so the previous
cluster is always pushed *except* that when the program seeks back to
block zero the
"if (vp-v_clen != 0)" fails and a new cluster is started without
pushing out the previously started one. That dirty block in the previous
cluster then hangs around until it is flushed as dirty blocks normally
would be.
It is the combination of this clustering behaviour and the fact that the
program always writes full blocks that causes the 8750 writes below.
Since the blocks are full file system blocks rather than mark them dirty
they are immediately passed to the clustering code, because they are
never in sequence the clustering code always starts a new cluster and
flushes the previous one except for 1 in every 8 blocks that doesn't
happen because when block 0 is written the previous cluster is not
pushed out but hangs around. The end result is that 7/8 blocks get
written immediately which is 8750/1 writes.
When the write size drops below the filesystem block size then the
clustering code never gets called because the buffers are just marked
dirty and cached.
I think if we fixed the issue of writing out full blocks this behviour
would stop but I also think the clustering code could do with a fix. It
should at least check to see if there is a cluster being built when the
blockno is 0 and push it out. Possibly though it'd be better to not push
out clusters of only one block and just leave them in the cache.
Sorry it's that crude. These results are from a FreeBSD
2.2.7-RELEASE+CAM, ccd stripe=2 (PII 400MHz, 512MB) system,
though exactly the same pattern is apparent with 3.4-STABLE.
"" indicate sub-second
Re: Another current crash (cvs-cur.6183
Stephen Hocking-Senior Programmer PGS SPS Perth wrote:
cvs-cur.6183 appeared to fix the crash I reported under disk activity NFS
but another one has reared its face, when using java with tya15 jit, running
the Together java IDE.
#0 boot (howto=256) at ../../kern/kern_shutdown.c:304
#1 0xc013d7e5 in panic (fmt=0xc0273534 "from debugger")
at ../../kern/kern_shutdown.c:554
#2 0xc01265bd in db_panic (addr=-1071292639, have_addr=0, count=-1,
modif=0xc5988c64 "") at ../../ddb/db_command.c:433
#3 0xc012655d in db_command (last_cmdp=0xc02abc2c, cmd_table=0xc02aba8c,
aux_cmd_tablep=0xc02e4578) at ../../ddb/db_command.c:333
#4 0xc0126622 in db_command_loop () at ../../ddb/db_command.c:455
#5 0xc0128733 in db_trap (type=3, code=0) at ../../ddb/db_trap.c:71
#6 0xc0255cc1 in kdb_trap (type=3, code=0, regs=0xc5988d6c)
at ../../i386/i386/db_interface.c:158
#7 0xc0262690 in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = 16, tf_edi = 0,
tf_esi = -979913536, tf_ebp = -979857996, tf_isp = -979858024,
tf_ebx = -1044942540, tf_edx = 0, tf_ecx = -1070604512, tf_eax = 26,
tf_trapno = 3, tf_err = 0, tf_eip = -1071292639, tf_cs = 8,
tf_eflags = 12870, tf_esp = -1070994081, tf_ss = -1071140376})
at ../../i386/i386/trap.c:549
#8 0xc0255f21 in Debugger (msg=0xc027b1e8 "d_iocmd botch")
I've got a different but I think related panic.
#9 0xc0143280 in panic (fmt=0xc0250460 "vm_page_wakeup: page not
busy!!!")
at ../../kern/kern_shutdown.c:552
#10 0xc01df583 in swp_pager_async_iodone (bp=0xc3236250)
at ../../vm/vm_page.h:346
#11 0xc0167ae6 in biodone (bp=0xc3236250) at ../../kern/vfs_bio.c:2706
#12 0xc0126b8d in dadone (periph=0xc092e380, done_ccb=0xc093f400)
at ../../cam/scsi/scsi_da.c:1230
#13 0xc0122acb in camisr (queue=0xc029def0) at ../../cam/cam_xpt.c:6298
#14 0xc01228dd in swi_cambio () at ../../cam/cam_xpt.c:6201
#15 0xc021188b in doreti_swi ()
A few more details
#11 0xc0167ae6 in biodone (bp=0xc3236250) at ../../kern/vfs_bio.c:2706
2706(*b_iodone) (bp);
#10 0xc01df583 in swp_pager_async_iodone (bp=0xc3236250)
at ../../vm/vm_page.h:346
346 KASSERT(m-flags PG_BUSY, ("vm_page_wakeup: page not busy!!!"));
I've got the dump if you want more diagnostics. It's hitting the KASSERT
on the second of 16 pages in the buf but none of them have PG_BUSY set
and it's only not panicing on the first page because
bp-b_pager.pg_reqpage is 0 and the call to vm_page_wakeup is skipped.
Paul.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: patches for test / review
Alfred Perlstein wrote: * Poul-Henning Kamp [EMAIL PROTECTED] [000320 11:45] wrote: In message [EMAIL PROTECTED], Alfred Perlstein writes: Keeping the currect cluster code is a bad idea, if the drivers were taught how to traverse the linked list in the buf struct rather than just notice "a big buffer" we could avoid a lot of page twiddling and also allow for massive IO clustering ( 64k ) Before we redesign the clustering, I would like to know if we actually have any recent benchmarks which prove that clustering is overall beneficial ? Yes it is really benificial. Yes, I've seen stats that show the degradation when clustering is switched off. Richard Wendlake (who wrote the OS detection code for the Netcraft web server survey) did a lot of testing in this area because of some pathological behavior he was seeing using Gnu's dbm package. Richard, do you want to post a summary of your tests? I'm not talking about a redesign of the clustering code as much as making the drivers that take a callback from it actually traverse the 'union cluster_info' rather than relying on the system to fake the pages being contiguous via remapping. There's nothing wrong with the clustering algorithms, it's just the steps it has to take to work with the drivers. Well, there is something wrong with our clustering algorithm. It always starts a new cluster when the first block of a file is written to. I found this when trying to explain some of the pathological behavior that Richard was seeing. Imagine an algorithm that will write blocks 0,5,2,7,4,1,6,3,0,... The clustering algorithm starts a new cluster if the block is at the beginning of the file, so writing block 0 will always start a new cluster. When block 5 is written out, the clustering code will try and add it to the existing cluster, will fail and so will flush the existing cluster which only has block 0 in it and then start another cluster, with block 5 in it. This continues, with the previous cluster being flushed and a new cluster being created with the current block in it. Eventually, we get to the point where 7 blocks have been flushed and the current cluster contains block 3. When it comes to write out the next block 0 the clustering algorithm doesn't bother trying to add the block to the existing cluster but immediately starts a new one so the cluster with block 3 in it *never gets flushed*. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
Bruce Evans wrote: On Sun, 12 Mar 2000, John Polstra wrote: Sheesh, criticism isn't enough? Now it has to be constructive too? ;-) I guess it could go into machine/limits.h in the "!defined(_ANSI_SOURCE)" section. Bruce might have a better idea. Trying to draw some closure on this discussion. This is the same as putting it in limits.h in the "!defined(_ANSI_SOURCE)" section. Anyway, the names should be FOO_MAX, not MAX_FOO. In C99, there are about 50 *_MAX and *_MIN values defined in stdint.h for Standard C types alone. I would prefer standard maxof() and minof() interfaces that work on any arithmetic type. These can almost be written in portable C, at least in C89 where types are restricted to char, signed char, ..., long double: #define isfloat(type) ((type)0.5 != 0) #define issigned(type) ((type)-1 0) #define isschar(type) (!isfloat(type) issigned(type) sizeof(type) == 1) #define isuchar(type) (!isfloat(type) !issigned(type) sizeof(type) == 1) ... #define maxof(type) ((type)(isschar(type) ? SCHAR_MAX : isuchar(type) ? UCHAR_MAX ...)) Where do you think these macros should go then? Having though about this some more, although the above macros are useful and can be used in a lot of the places where bounds checking needs to be done, there is still probably a need to define certain constants, such as UID_MAX, because the range of valid values may not necessarily be the same as the range of values supported by the type. UID_MAX being a good example, since FreeBSD does not support 32-bit UIDs at the moment even though the kernel does. Most applications are limiting themselves to USHRT_MAX, those applications should be fixed to use UID_MAX, even if we still limit UID_MAX == USHRT_MAX for backwards compatibility. So, sys/syslimits.h seems like the right place for constants like UID_MAX. Any disagreements there? I guess the above macros should really be machine specific if we're going to be pedantic. Would machine/limits.h be an appropriate location for them? Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
Giorgos Keramidas wrote: On Sun, Mar 12, 2000 at 05:59:09AM +, Paul Richards wrote: Are expressions like ((uid_t)0-1) portable/safe ? Maybe that's a better way of approaching this. To get the all-1's number, maybe it's better to use ((uid_t)~0), but that is a rather controversial topic anyway. Well it would be worth deciding on some standard way of dealing with this problem. I've been wandering around the code for the last few days and the same issue crops up all over the place i.e. bounds checking. There are a lot of programs that currently work on alpha only because no-one is hitting the boundary conditions e.g. ipfw checks for rule number limits based upon ULONG_MAX rather than the size of the field that holds the rule number. We could create a new include file that we use for constants that are related to FreeBSD specific types or we can agree on a coding style for performing bounds checking using tricks like ((uid_t)0-1) The code solution is robust in that if the type changes size the code still works. Using constants has the advantage that it is easy to document what the limits are on certain paramters such as IPFW rule numbers since you'll be able to look them up. All we have to do then is write all the bounds checking code :-) Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
Garrett Wollman wrote:
On Mon, 13 Mar 2000 00:06:19 +, Paul Richards [EMAIL PROTECTED] said:
We could create a new include file that we use for constants that are
related to FreeBSD specific types or we can agree on a coding style for
performing bounds checking using tricks like ((uid_t)0-1)
Or we can do it the right way, by assigning to a variable of the
correct type and checking that the value remains unchanged.
I'm not just trying to determine whether a value fits into a type, I'm
also trying to find out what the maximum value the type can hold is.
e.g. in the following code snippet
id = strtoul(p, (char **)NULL, 10);
if (errno == ERANGE) {
warnx("%s max uid value (%lu)", p, ULONG_MAX);
return (0);
}
you also have to print out the maximum value. Not an uncommon
requirement for error reporting.
You're solution doesn't help with the second part.
What I'm looking at is the following.
id = strtoul(p, (char **)NULL, 10);
if ((errno == ERANGE) || (id = ((uid_t)0-1))) {
warnx("%s max uid value (%lu)", p, ((uid_t)0-1));
return (0);
}
or the alternative
id = strtoul(p, (char **)NULL, 10);
if ((errno == ERANGE) || (id = UID_MAX)) {
warnx("%s max uid value (%lu)", p, UID_MAX);
return (0);
}
When you see it written out like that the latter is a lot more
informative. It also provides the flexibility to limit the parameters
max value even if the type allows it to be larger. This is of particular
significance to UIDs which are currently limited to a far smaller value
than would fit in a uid_t.
Paul.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
John Polstra wrote: In article [EMAIL PROTECTED], Paul Richards [EMAIL PROTECTED] wrote: They must not go into limits.h. That header file is defined by the ANSI/ISO C standard. The standard doesn't permit polluting the namespace with extra stuff. Umm, ok. I don't think our limits.h actually has anything in it that meets the ANSI/ISO standard, every line is ifdef'd :-) Where would be a better place for constants like this? Sheesh, criticism isn't enough? Now it has to be constructive too? ;-) I guess it could go into machine/limits.h in the "!defined(_ANSI_SOURCE)" section. Bruce might have a better idea. I don't think machine/limits.h is the right place. These are constants that are definately not architecture dependent. The whole problem at the moment is that the code is abusing architecture dependent constants in lieu of anything better. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
Peter Jeremy wrote:
On 2000-Mar-13 12:01:03 +1100, Paul Richards [EMAIL PROTECTED] wrote:
id = strtoul(p, (char **)NULL, 10);
if ((errno == ERANGE) || (id = UID_MAX)) {
warnx("%s max uid value (%lu)", p, UID_MAX);
return (0);
}
You can do this now. Just add the following:
pid_t UID_MAX = ~0;
somewhere before the code.
I assume you meant uid_t, but anyway, yes that can be done but I'm
looking for a more generic solution since this is a widespread problem
in the userland code.
When you see it written out like that the latter is a lot more
informative. It also provides the flexibility to limit the parameters
max value even if the type allows it to be larger. This is of particular
significance to UIDs which are currently limited to a far smaller value
than would fit in a uid_t.
AFAIK, there's no real hard limit. adduser(8) limits uids to 32000
to prevent portability problems with systems using a signed 16-bit
uid_t (whilst allowing for a few `special' uids near the top of the
range). Other potential limits are ~65000 (16-bit unsigned) and
9 (5 decimal digits).
The actual limit is 4294967295, which is the largest number that will
fit into the the 32-bit uid_t. This does work, I've been playing around
with it for the last few days. At least, it works from a kernel
perspective, there are many applications that screw things up, most of
them use atol to do conversions which limits the UID to half that. If
you stick within the signed 32 bit limits most things seem to work.
It would be nice if there was a system-wide constant that could be
used for this, but it's not obvious where this should be located.
(And note that adduser(8) is a perl script, so it can't use a C
header file without some contortions).
Perl can use C include files (after performing some magic) so that
wouldn't be a problem.
Paul.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
John Polstra wrote: In article [EMAIL PROTECTED], Paul Richards [EMAIL PROTECTED] wrote: John Polstra wrote: I guess it could go into machine/limits.h in the "!defined(_ANSI_SOURCE)" section. Bruce might have a better idea. I don't think machine/limits.h is the right place. These are constants that are definately not architecture dependent. The whole problem at the moment is that the code is abusing architecture dependent constants in lieu of anything better. Hmm, you're right. How about sys/syslimits.h? That sounds ideal :-) I guess my next question is, are there any objections to #define UID_MAX ((uid_t)0-1) which would actually give us the best of both worlds. I assume that gcc optimises the calculation away so it's effectively a constant but automagically of the right size. I can see the flaw in that straight away in that uid_t isn't available in sys/syslimits.h On the other hand, since you may want to limit the MAX_UID (or some other paramter) independently of the size of the type then its probably best to maintain the value manually in any case. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
Peter Jeremy wrote: On 2000-Mar-13 13:14:40 +1100, Paul Richards [EMAIL PROTECTED] wrote: #define UID_MAX ((uid_t)0-1) ... I can see the flaw in that straight away in that uid_t isn't available in sys/syslimits.h Not a problem. C macros are just text expansions. The `uid_t' isn't evaluated unless/until the macro is used - at which point uid_t must be available. Uhh, yeah. It's the middle of the night here I obviously wasn't awake when I said that :-) Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
MAX_UID ?
The fix I applied to pwd_mkdb is an improvement over what was there
before i.e. nothing, but is a poor solution at the moment since it won't
work correctly on the alpha.
The following code snippet is OK on the i386 but on the alpha ULONG_MAX
is 64 bits and so is a totally wrong constant to check a max uid
against.
id = strtoul(p, (char **)NULL, 10);
if (errno == ERANGE) {
warnx("%s max uid value (%lu)", p, ULONG_MAX);
return (0)
}
I think we need a MAX_UID and a MAX_GID to perform checks like this.
Anyone got any objections to adding them to /usr/include/limits.h ?
Paul.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: MAX_UID ?
John Polstra wrote: In article [EMAIL PROTECTED], Paul Richards [EMAIL PROTECTED] wrote: I think we need a MAX_UID and a MAX_GID to perform checks like this. Anyone got any objections to adding them to /usr/include/limits.h ? They must not go into limits.h. That header file is defined by the ANSI/ISO C standard. The standard doesn't permit polluting the namespace with extra stuff. Umm, ok. I don't think our limits.h actually has anything in it that meets the ANSI/ISO standard, every line is ifdef'd :-) Where would be a better place for constants like this? Are expressions like ((uid_t)0-1) portable/safe ? Maybe that's a better way of approaching this. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: The pw command
Kris Kennaway wrote: On Fri, 10 Mar 2000, Paul Richards wrote: Non-root users can use the pw command to get information from the master.passwd file e.g. ps showuser paul paul:*:1000:1000::0:0: Richards:/home/paul:/usr/local/bin/bash % pw showuser kkenn kkenn:*:1000:0::0:0:Kris Kennaway:/home/kkenn:/usr/local/bin/tcsh % grep kkenn /etc/passwd kkenn:*:1000:0::0:0:Kris Kennaway:/home/kkenn:/usr/local/bin/tcsh % ls -l /usr/bin/pw -r-xr-xr-x 1 root wheel 53180 Mar 4 00:49 /usr/sbin/pw* In other words, pw(8) runs with no special privileges and can only read what the user can normally read. Ok, as Mike mentioned in his email pw gets its info from pwd.db, I knew that. What I had assumed was that pwd.db was the same as the contents of /etc/passwd. I spent most of last night trying to work out how a non-root user was getting access to master.passwd based on that flawed assumption :-) My /etc/passwd does not have the class and expiry fields in it. I've deleted it and let it be recreated and it still doesn't have those fields. I've browsed through the code in pwd_mkdb and it looks to me like it deliberately creates the old style /etc/passwd file, which makes sense from a comatibility perspective. So what I'm puzzled about now is how come yours is different and do we think its correct that pwd.db and /etc/passwd have different information in them? Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
The pw command
Non-root users can use the pw command to get information from the master.passwd file e.g. ps showuser paul paul:*:1000:1000::0:0: Richards:/home/paul:/usr/local/bin/bash which shows the class, password expiry and account expiry. I'm not sure whether that's information that should be kept secure but it does seem like 'pw' is the only command that makes it available. The 'chsh' command doesn't show this information except when run as root for instance. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Crypto progress! (And a Biiiig TODO list)
Mark Murray wrote: Mark Murray wrote: I'm very uncomfortable with requiring Yet Another Daemon to manage (and screw up) password checking. Generally speaking, if I wouldn't trust a program with root privileges, I wouldn't trust it with my password, either (for obvious reasons). If "all those" suid programs could be "de-suid'ed", and replaced with a simple "does this username/password pair check out?" daemon/module, would that make you happier? As long as there is some sort of rate limiting system so that it doesn't provide a trivial online brute force password cracking service... Getting this right would be an interesting challenge. :-) Easy to do of the daemon is not on by default, or if it is pretty fascist by default, with lots of options to define the fascism. I think there are risks with this approach and I'm not sure the benefits are clear cut. My main concern is single point of failure. It's only an improvement on what we have now if we can remove suid programs. If you do that then the *only* way to authenticate is via the daemon. If you screw the daemon in some way and you don't already have a root session then your only option is to hit the power switch. At the moment, if you screw login you can su, if you screw su then you can login at the console etc etc. The daemon becomes an obvious target for DOS given how critical it is and any weakness that would cause it to fail would result in a catastrophic system failure. There aren't many (any?) system processes that are that critical. How are you going to deal with single-user boots? I can't really see how you can remove all password related suid programs, only a few of them. The other concern is that at the moment you can't get access to the encrypted password without being root. With a daemon any user can process can access that information. I accept that this might not be a losing position since any user script could run su or password repeatedly but it seems to me that there isn't a *clear* security enhancement here. The daemon approach actually has benfits that I'm keen on that aren't related to security. A single point of access to the data means that the backend can be changed so that passwords can be in a different file or a database, without having to worry about rebuilding all the binaries. You could even split users across different back end systems. It would be interesting to extend the whole idea so that there were username classes e.g. system account, web site user and so on. Then you could use the same database to hold all account information. This is of course what NT does and most people hate that because the web server then has access to the login passwords. If you're going to have a daemon though that any user process can connect to then this is the direction you're moving in so you may as well take the added functionality that comes with it and make it a generic authentication system that can be used for things like web servers. On the whole though, I'm not sure that this is a direction that has a definate win to it. How many binaries are there that have access to the master.passwd file? How many lines of code need to be audited for those binaries, compared to the number of lines of code needed to implement a secure daemon solution? Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: feedback on CD install of 4.0-RC2
Jonathan Lemon wrote: In article local.mail.freebsd-current/[EMAIL PROTECTED] you write: Kai Großjohann wrote: "Jordan K. Hubbard" [EMAIL PROTECTED] writes: I really kinda wish you'd point them to Novice^H^H^H^H^HStandard instead since it does more than be a bit more verbose, it also makes sure that all the appropriate steps are covered and prevents even relatively skilled people from hanging themselves. Does this mean that this option should be called `guided'? I know a little bit about Unix but haven't installed FreeBSD more than five times or so. And I always thought that the novice install meant that I didn't get as many choices... "Guided". I like it. That's *PRECISELY* what this installation option is. There is NO difference in the number of choices available in any of the three types. Guided/Express/Expert. That's my vote. Cool, seconded. I actually use "novice" most of the time, simply because I like the hints that help me not to screw up, and I've been using this option (or equivalent) since 1.1.5 at least. Guided sounds like a better description to me. Possibly, the mistake here is to present the "custom" option as an install method at all. Given that "novice" walks you through the install steps but still allows you to do a customised install, it should perhaps be called "installation", the custom option should be changed to something like "maintenance" because really that's where it's useful in that you can go fix any bit of the system without having to walk through all the installation steps. No new users would try and use "maintenance" to do their installation with. Paul. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
